RFC: AWS Serverless Support #186

New Issue

AtHeartEngineer · 2025-07-08T08:42:06-04:00

AtHeartEngineer commented

2025-07-08 08:42:06 -04:00

Originally created by @txase on 2/14/2025

This draft PR contains a POC of support for deploying an instance of Vaultwarden into an AWS account using entirely "serverless" services (likely falling within the free-tier usage limits as well). I'm looking for feedback and agreement by Vaultwarden maintainers on whether these contributions could be merged into vaultwarden (with further refinement).

Architecture

CloudFront CDN
├─ API Lambda Function
│  ├─ Data S3 Bucket
│  ├─ Aurora DSQL Database
│  └─ Amazon Simple Email Service (SES)
└─ Web-vault static assets S3 Bucket

All of this is implemented in the PR behind feature flags: dsql, s3, and ses. All three can be enabled together via the aws feature flag.

Unimplemented Functionality

I believe all functionality, except as listed below, is functional. But I'm new to vaultwarden and may have missed something along the way. I've not found any significant issues with my own usage due to this missing functionality, however.

Background jobs (e.g. to delete expired data)
Websockets for events

Open Questions / Concerns

Aurora DSQL is in preview. It's mostly compatible with Postgres, but it has a few incompatibilities that could be an issue if they remain at GA:
- Foreign keys (FKs) are not supported. However, after auditing the code I did not find any Postgres functionality that requires FKs (unlike for sqlite and Mysql). It would be nice to have it to feel safer for data consistency, but doesn't appear to be necessary. The seed migration file in this PR is a dump from a local Postgres vaultwarden database that I then stripped FK constraints from. This is a known documented issue with the DSQL preview, and it is unclear if it will be resolved before GA.
- DDL statements cannot be executed in a transaction, so migrations are a bit more fraught. I've yet to have a migration fail, though. This is a known documented issue with the DSQL preview, and it is unclear if it will be resolved before GA.
- Tables can be altered to add rows, but new rows cannot have constraints. This is an undocumented issue I found, and gives me more concern that further migration difficulties will arise over time. Hopefully this will be resolved before GA.
API Lambda Function settings:
- MemorySize is set to 3008 MB. This is the largest possible Function size for new accounts (which is an undocumented AWS limit). Cost is linearly related to this value, but lower values mean longer cold start times, especially when creating TLS connections to external services. For a production environment with a lot of traffic this would be tuned to a minimal amount that achieves desired performance characteristics. Here, I set it to the max because traffic will generally be minimal for private usage, and cost will likely fit under the free tier.
- Timeout is set to 5 minutes. Cold start times seem to average about 3-4 seconds. Warmed functions (or cold functions after the init work) generally take less than a second to complete (if no external services need to be invoked, warmed functions generally complete in under 10 ms). Importing my personal vault (~600 items) took just under 30 seconds. A 5 minute limit seems large, but I don't want to cause issues for anyone importing or exporting a sizable vault.
Attachment size limit: 6 MB. With client changes it would be possible to upload directly from clients to S3 via signed URLs. This is how the official Bitwarden service works, but they use the Azure Blob Storage service instead of AWS S3. Azure uses POST instead of S3's PUT to upload, and although S3 also supports POST with signed URLs the data must be form-encoded. This means existing Bitwarden clients cannot upload directly to S3, and must instead upload through the API Lambda Function. Unfortunately, the AWS Lambda service has a 6 MB size limit for request payloads. Note: This only applies to uploads, downloads are streamed from S3 directly via signed GET URLs which do not have a size limit.

Deployment Instructions

See the aws/README.md file in this PR.

Proposed Plan of Attack

This PR is too large to attempt to review and merge with sanity. With agreement from Vaultwarden maintainers in comments below, I propose developing and merging the following as separate PRs in sequence:

Add the persistent_fs module, migrating existing filesystem functionality into a "local" backend implementation. See src/persistent_fs/mod.rs and src/persistent_fs/local.rs. This would not change any functionality; it would simply rearchitect file access in preparation for the addition of an S3 backend.
Add the S3 persistent FS backend implementation
Add SES support for sending emails
Add DSQL support, with some means for denoting it is beta or unstable functionality (e.g. label the feature flag dsql-beta)
Add a GitHub Actions workflow to build a Lambda function package on release
- In an ideal case, the vaultwarden project would also own an AWS account with regional S3 Buckets to upload releases to. This would simplify deployment, allowing users to deploy to their own AWS accounts without having to download and upload function code packages themselves. I can help set this up if needed.
Add the CloudFormation template and deployment instructions

*Originally created by @txase on 2/14/2025* This draft PR contains a POC of support for deploying an instance of Vaultwarden into an AWS account using entirely "serverless" services (likely falling within the free-tier usage limits as well). I'm looking for feedback and agreement by Vaultwarden maintainers on whether these contributions could be merged into vaultwarden (with further refinement). ### Architecture ``` CloudFront CDN ├─ API Lambda Function │ ├─ Data S3 Bucket │ ├─ Aurora DSQL Database │ └─ Amazon Simple Email Service (SES) └─ Web-vault static assets S3 Bucket ``` All of this is implemented in the PR behind feature flags: `dsql`, `s3`, and `ses`. All three can be enabled together via the `aws` feature flag. ### Unimplemented Functionality I believe all functionality, except as listed below, is functional. But I'm new to vaultwarden and may have missed something along the way. I've not found any significant issues with my own usage due to this missing functionality, however. * Background jobs (e.g. to delete expired data) * Websockets for events ### Open Questions / Concerns * Aurora DSQL is in preview. It's mostly compatible with Postgres, but it has a few incompatibilities that could be an issue if they remain at GA: * Foreign keys (FKs) are not supported. However, after auditing the code I did not find any Postgres functionality that requires FKs (unlike for sqlite and Mysql). It would be nice to have it to feel safer for data consistency, but doesn't appear to be necessary. The seed migration file in this PR is a dump from a local Postgres vaultwarden database that I then stripped FK constraints from. This is a known documented issue with the DSQL preview, and it is unclear if it will be resolved before GA. * DDL statements cannot be executed in a transaction, so migrations are a bit more fraught. I've yet to have a migration fail, though. This is a known documented issue with the DSQL preview, and it is unclear if it will be resolved before GA. * Tables can be altered to add rows, but new rows cannot have constraints. This is an undocumented issue I found, and gives me more concern that further migration difficulties will arise over time. Hopefully this will be resolved before GA. * API Lambda Function settings: * MemorySize is set to 3008 MB. This is the largest possible Function size for new accounts (which is an undocumented AWS limit). Cost is linearly related to this value, but lower values mean longer cold start times, especially when creating TLS connections to external services. For a production environment with a lot of traffic this would be tuned to a minimal amount that achieves desired performance characteristics. Here, I set it to the max because traffic will generally be minimal for private usage, and cost will likely fit under the free tier. * Timeout is set to 5 minutes. Cold start times seem to average about 3-4 seconds. Warmed functions (or cold functions after the init work) generally take less than a second to complete (if no external services need to be invoked, warmed functions generally complete in under 10 ms). Importing my personal vault (~600 items) took just under 30 seconds. A 5 minute limit seems large, but I don't want to cause issues for anyone importing or exporting a sizable vault. * Attachment size limit: 6 MB. With client changes it would be possible to upload directly from clients to S3 via signed URLs. This is how the official Bitwarden service works, but they use the Azure Blob Storage service instead of AWS S3. Azure uses `POST` instead of S3's `PUT` to upload, and although S3 also supports `POST` with signed URLs the data must be form-encoded. This means existing Bitwarden clients cannot upload directly to S3, and must instead upload through the API Lambda Function. Unfortunately, the AWS Lambda service has a 6 MB size limit for request payloads. *Note: This only applies to uploads, downloads are streamed from S3 directly via signed `GET` URLs which do not have a size limit.* ### Deployment Instructions See the aws/README.md file in this PR. ### Proposed Plan of Attack This PR is too large to attempt to review and merge with sanity. With agreement from Vaultwarden maintainers in comments below, I propose developing and merging the following as separate PRs in sequence: - [x] Add the `persistent_fs` module, migrating existing filesystem functionality into a "local" backend implementation. See `src/persistent_fs/mod.rs` and `src/persistent_fs/local.rs`. This would not change any functionality; it would simply rearchitect file access in preparation for the addition of an S3 backend. - [x] Add the S3 persistent FS backend implementation - [ ] Add SES support for sending emails - [ ] Add DSQL support, with some means for denoting it is beta or unstable functionality (e.g. label the feature flag `dsql-beta`) - [ ] Add a GitHub Actions workflow to build a Lambda function package on release - [ ] In an ideal case, the vaultwarden project would also own an AWS account with regional S3 Buckets to upload releases to. This would simplify deployment, allowing users to deploy to their own AWS accounts without having to download and upload function code packages themselves. I can help set this up if needed. - [ ] Add the CloudFormation template and deployment instructions

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github/vaultwarden#186