github/infisical
1
0
Fork 0
mirror of https://github.com/Infisical/infisical.git synced 2026-01-09 15:38:03 -05:00
Code Issues Packages Projects Releases Wiki Activity

feat: add auto redeploy for daemonset and statefulset

Browse Source
This commit is contained in:
Sheen Capadngan
2025-02-27 16:26:43 +09:00
parent f4bd48fd1d
commit 51f29e5357
3 changed files with 238 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files

141
docs/integrations/platforms/kubernetes/infisical-dynamic-secret-crd.mdx
View File

@@ -3,18 +3,21 @@ sidebarTitle: "InfisicalDynamicSecret CRD"
title: "Using the InfisicalDynamicSecret CRD"
description: "Learn how to generate dynamic secret leases in Infisical and sync them to your Kubernetes cluster."
---
## Overview
The **InfisicalDynamicSecret** CRD allows you to easily create and manage dynamic secret leases in Infisical and automatically sync them to your Kubernetes cluster as native **Kubernetes Secret** resources.
This means any Pod, Deployment, or other Kubernetes resource can make use of dynamic secrets from Infisical just like any other K8s secret.
## Overview
The **InfisicalDynamicSecret** CRD allows you to easily create and manage dynamic secret leases in Infisical and automatically sync them to your Kubernetes cluster as native **Kubernetes Secret** resources.
This means any Pod, Deployment, or other Kubernetes resource can make use of dynamic secrets from Infisical just like any other K8s secret.
This CRD offers the following features:
- **Generate a dynamic secret lease** in Infisical and track its lifecycle.
- **Write** the dynamic secret from Infisical to your cluster as native Kubernetes secret.
- **Automatically rotate** the dynamic secret value before it expires to make sure your cluster always has valid credentials.
- **Optionally trigger redeployments** of any workloads that consume the secret if you enable auto-reload.
### Prerequisites
- A project within Infisical.
- A [machine identity](/docs/documentation/platform/identities/overview) ready for use in Infisical that has permissions to create dynamic secret leases in the project.
- You have already configured a dynamic secret in Infisical.
@@ -77,16 +80,19 @@ spec:
```
Apply the InfisicalDynamicSecret CRD to your cluster.
```bash
kubectl apply -f dynamic-secret-crd.yaml
```
After applying the InfisicalDynamicSecret CRD, you should notice that the dynamic secret lease has been created in Infisical and synced to your Kubernetes cluster. You can verify that the lease has been created by doing:
```bash
kubectl get secret <managed-secret-name> -o yaml
```
After getting the secret, you should should see that the secret has data that contains the lease credentials.
```yaml
apiVersion: v1
data:
@@ -102,7 +108,7 @@ kind: Secret
If you are fetching secrets from a self-hosted instance of Infisical set the value of `hostAPI` to
` https://your-self-hosted-instace.com/api`
When `hostAPI` is not defined the operator fetches secrets from Infisical Cloud.
When `hostAPI` is not defined the operator fetches secrets from Infisical Cloud.
<Accordion title="Advanced use case">
If you have installed your Infisical instance within the same cluster as the Infisical operator, you can optionally access the Infisical backend's service directly without having to route through the public internet.
@@ -120,36 +126,45 @@ kind: Secret
<Accordion title="leaseTTL">
The `leaseTTL` is a string-formatted duration that defines the time the lease should last for the dynamic secret.
The format of the field is `[duration][unit]` where `duration` is a number and `unit` is a string representing the unit of time.
The format of the field is `[duration][unit]` where `duration` is a number and `unit` is a string representing the unit of time.
The following units are supported:
- `s` for seconds (must be at least 5 seconds)
- `m` for minutes
- `h` for hours
- `d` for days
The following units are supported:
<Note>
The lease duration at most be 1 day (24 hours). And the TTL must be less than the max TTL defined on the dynamic secret.
</Note>
</Accordion>
- `s` for seconds (must be at least 5 seconds)
- `m` for minutes
- `h` for hours
- `d` for days
<Note>
The lease duration at most be 1 day (24 hours). And the TTL must be less than the max TTL defined on the dynamic secret.
</Note>
</Accordion>
<Accordion title="managedSecretReference">
The `managedSecretReference` field is used to define the Kubernetes secret where the dynamic secret lease should be stored. The required fields are `secretName` and `secretNamespace`.
```yaml
spec:
managedSecretReference:
secretName: <secret-name>
secretNamespace: default
```
```yaml
spec:
managedSecretReference:
secretName: <secret-name>
secretNamespace: default
```
<Accordion title="managedSecretReference.secretName">
The name of the Kubernetes secret where the dynamic secret lease should be stored.
</Accordion>
{" "}
<Accordion title="managedSecretReference.secretNamespace">
The namespace of the Kubernetes secret where the dynamic secret lease should be stored.
</Accordion>
<Accordion title="managedSecretReference.secretName">
The name of the Kubernetes secret where the dynamic secret lease should be
stored.
</Accordion>
{" "}
<Accordion title="managedSecretReference.secretNamespace">
The namespace of the Kubernetes secret where the dynamic secret lease should
be stored.
</Accordion>
<Accordion title="managedSecretReference.creationPolicy">
Creation polices allow you to control whether or not owner references should be added to the managed Kubernetes secret that is generated by the Infisical operator.
@@ -165,32 +180,36 @@ kind: Secret
</Tip>
This field is optional.
</Accordion>
<Accordion title="managedSecretReference.secretType">
Override the default Opaque type for managed secrets with this field. Useful for creating kubernetes.io/dockerconfigjson secrets.
This field is optional.
</Accordion>
</Accordion>
<Accordion title="leaseRevocationPolicy">
The field is optional and will default to `None` if not defined.
The field is optional and will default to `None` if not defined.
The lease revocation policy defines what the operator should do with the leases created by the operator, when the InfisicalDynamicSecret CRD is deleted.
The lease revocation policy defines what the operator should do with the leases created by the operator, when the InfisicalDynamicSecret CRD is deleted.
Valid values are `None` and `Revoke`.
Valid values are `None` and `Revoke`.
Behavior of each policy:
- `None`: The operator will not override existing secrets in Infisical. If a secret with the same key already exists, the operator will skip pushing that secret, and the secret will not be managed by the operator.
- `Revoke`: The operator will revoke the leases created by the operator when the InfisicalDynamicSecret CRD is deleted.
Behavior of each policy:
- `None`: The operator will not override existing secrets in Infisical. If a secret with the same key already exists, the operator will skip pushing that secret, and the secret will not be managed by the operator.
- `Revoke`: The operator will revoke the leases created by the operator when the InfisicalDynamicSecret CRD is deleted.
```yaml
spec:
leaseRevocationPolicy: Revoke
```
```yaml
spec:
leaseRevocationPolicy: Revoke
```
</Accordion>
<Accordion title="dynamicSecret">
@@ -205,29 +224,37 @@ kind: Secret
secretsPath: <secrets-path>
```
<Accordion title="dynamicSecret.secretName">
The name of the dynamic secret.
</Accordion>
{" "}
<Accordion title="dynamicSecret.projectId">
The project ID of where the dynamic secret is stored in Infisical.
</Accordion>
<Accordion title="dynamicSecret.secretName">
The name of the dynamic secret.
</Accordion>
<Accordion title="dynamicSecret.environmentSlug">
The environment slug of where the dynamic secret is stored in Infisical.
</Accordion>
{" "}
<Accordion title="dynamicSecret.secretsPath">
The path of where the dynamic secret is stored in Infisical. The root path is `/`.
</Accordion>
<Accordion title="dynamicSecret.projectId">
The project ID of where the dynamic secret is stored in Infisical.
</Accordion>
{" "}
<Accordion title="dynamicSecret.environmentSlug">
The environment slug of where the dynamic secret is stored in Infisical.
</Accordion>
{" "}
<Accordion title="dynamicSecret.secretsPath">
The path of where the dynamic secret is stored in Infisical. The root path is
`/`.
</Accordion>
</Accordion>
<Accordion title="authentication">
The `authentication` field dictates which authentication method to use when pushing secrets to Infisical.
The available authentication methods are `universalAuth`, `kubernetesAuth`, `awsIamAuth`, `azureAuth`, `gcpIdTokenAuth`, and `gcpIamAuth`.
The `authentication` field dictates which authentication method to use when pushing secrets to Infisical.
The available authentication methods are `universalAuth`, `kubernetesAuth`, `awsIamAuth`, `azureAuth`, `gcpIdTokenAuth`, and `gcpIamAuth`.
<Accordion title="universalAuth">
The universal authentication method is one of the easiest ways to get started with Infisical. Universal Auth works anywhere and is not tied to any specific cloud provider.
@@ -246,7 +273,7 @@ kind: Secret
spec:
universalAuth:
credentialsRef:
secretName: <secret-name>
secretName: <secret-name>
secretNamespace: <secret-namespace>
```
@@ -282,6 +309,7 @@ kind: Secret
name: <secret-name>
namespace: <secret-namespace>
```
</Accordion>
<Accordion title="awsIamAuth">
@@ -316,12 +344,12 @@ kind: Secret
azureAuth:
identityId: <machine-identity-id>
```
</Accordion>
<Accordion title="gcpIamAuth">
The GCP IAM machine identity authentication method is used to authenticate with Infisical. The identity ID is stored in a field in the InfisicalSecret resource. This authentication method can only be used both within and outside GCP environments.
[Read more about Azure Auth](/documentation/platform/identities/gcp-auth).
Valid fields:
- `identityId`: The identity ID of the machine identity you created.
- `serviceAccountKeyFilePath`: The path to the GCP service account key file.
@@ -334,6 +362,7 @@ kind: Secret
identityId: <machine-identity-id>
serviceAccountKeyFilePath: </path-to-service-account-key-file.json>
```
</Accordion>
<Accordion title="gcpIdTokenAuth">
The GCP ID Token machine identity authentication method is used to authenticate with Infisical. The identity ID is stored in a field in the InfisicalSecret resource. This authentication method can only be used within GCP environments.
@@ -349,11 +378,11 @@ kind: Secret
gcpIdTokenAuth:
identityId: <machine-identity-id>
```
</Accordion>
</Accordion>
<Accordion title="tls">
This block defines the TLS settings to use for connecting to the Infisical
instance.
@@ -376,11 +405,11 @@ kind: Secret
secretNamespace: default
key: ca.crt
```
</Accordion>
</Accordion>
### Applying the InfisicalDynamicSecret CRD to your cluster
Once you have configured the `InfisicalDynamicSecret` CRD with the required fields, you can apply it to your cluster. After applying, you should notice that a lease has been created in Infisical and synced to your Kubernetes cluster.
@@ -396,7 +425,7 @@ To address this, we've added functionality to automatically redeploy your deploy
#### Enabling auto redeploy
To enable auto redeployment you simply have to add the following annotation to the deployment that consumes a managed secret
To enable auto redeployment you simply have to add the following annotation to the deployment, statefulset, or daemonset that consumes a managed secret.
```yaml
secrets.infisical.com/auto-reload: "true"

28
docs/integrations/platforms/kubernetes/infisical-secret-crd.mdx
View File

@@ -547,12 +547,14 @@ The `managedSecretReference` field is deprecated and will be removed in a future
Replace it with `managedKubeSecretReferences`, which now accepts an array of references to support multiple managed secrets in a single InfisicalSecret CRD.
Example:
```yaml
managedKubeSecretReferences:
- secretName: managed-secret
secretNamespace: default
creationPolicy: "Orphan"
managedKubeSecretReferences:
- secretName: managed-secret
secretNamespace: default
creationPolicy: "Orphan"
```
</Note>
<Accordion title="managedKubeSecretReferences">
@@ -666,13 +668,13 @@ The example below assumes that the `BINARY_KEY_BASE64` secret is stored as a bas
The resulting managed secret will contain the decoded value of `BINARY_KEY_BASE64`.
```yaml
managedKubeSecretReferences:
secretName: managed-secret
secretNamespace: default
template:
includeAllSecrets: true
data:
BINARY_KEY: "{{ decodeBase64ToBytes .BINARY_KEY_BASE64.Value }}"
managedKubeSecretReferences:
secretName: managed-secret
secretNamespace: default
template:
includeAllSecrets: true
data:
BINARY_KEY: "{{ decodeBase64ToBytes .BINARY_KEY_BASE64.Value }}"
```
</Accordion>
@@ -866,7 +868,7 @@ To address this, we added functionality to automatically redeploy your deploymen
#### Enabling auto redeploy
To enable auto redeployment you simply have to add the following annotation to the deployment that consumes a managed secret
To enable auto redeployment you simply have to add the following annotation to the deployment, statefulset, or daemonset that consumes a managed secret.
```yaml
secrets.infisical.com/auto-reload: "true"
@@ -948,4 +950,4 @@ metadata:
type: Opaque
```
</Accordion>
</Accordion>

138
k8-operator/packages/controllerhelpers/controllerhelpers.go
View File

@@ -27,6 +27,18 @@ func ReconcileDeploymentsWithManagedSecrets(ctx context.Context, client controll
return 0, fmt.Errorf("unable to get deployments in the [namespace=%v] [err=%v]", managedSecret.SecretNamespace, err)
}
listOfDaemonSets := &v1.DaemonSetList{}
err = client.List(ctx, listOfDaemonSets, &controllerClient.ListOptions{Namespace: managedSecret.SecretNamespace})
if err != nil {
return 0, fmt.Errorf("unable to get daemonSets in the [namespace=%v] [err=%v]", managedSecret.SecretNamespace, err)
}
listOfStatefulSets := &v1.StatefulSetList{}
err = client.List(ctx, listOfStatefulSets, &controllerClient.ListOptions{Namespace: managedSecret.SecretNamespace})
if err != nil {
return 0, fmt.Errorf("unable to get statefulSets in the [namespace=%v] [err=%v]", managedSecret.SecretNamespace, err)
}
managedKubeSecretNameAndNamespace := types.NamespacedName{
Namespace: managedSecret.SecretNamespace,
Name: managedSecret.SecretName,
@@ -39,6 +51,7 @@ func ReconcileDeploymentsWithManagedSecrets(ctx context.Context, client controll
}
var wg sync.WaitGroup
// Iterate over the deployments and check if they use the managed secret
for _, deployment := range listOfDeployments.Items {
deployment := deployment
@@ -54,6 +67,34 @@ func ReconcileDeploymentsWithManagedSecrets(ctx context.Context, client controll
}
}
// Iterate over the daemonSets and check if they use the managed secret
for _, daemonSet := range listOfDaemonSets.Items {
daemonSet := daemonSet
if daemonSet.Annotations[AUTO_RELOAD_DEPLOYMENT_ANNOTATION] == "true" && IsDaemonSetUsingManagedSecret(daemonSet, managedSecret) {
wg.Add(1)
go func(deployment v1.DaemonSet, managedSecret corev1.Secret) {
defer wg.Done()
if err := ReconcileDaemonSet(ctx, client, logger, daemonSet, managedSecret); err != nil {
logger.Error(err, fmt.Sprintf("unable to reconcile daemonset with [name=%v]. Will try next requeue", deployment.ObjectMeta.Name))
}
}(daemonSet, *managedKubeSecret)
}
}
// Iterate over the statefulSets and check if they use the managed secret
for _, statefulSet := range listOfStatefulSets.Items {
statefulSet := statefulSet
if statefulSet.Annotations[AUTO_RELOAD_DEPLOYMENT_ANNOTATION] == "true" && IsStatefulSetUsingManagedSecret(statefulSet, managedSecret) {
wg.Add(1)
go func(statefulSet v1.StatefulSet, managedSecret corev1.Secret) {
defer wg.Done()
if err := ReconcileStatefulSet(ctx, client, logger, statefulSet, managedSecret); err != nil {
logger.Error(err, fmt.Sprintf("unable to reconcile statefulset with [name=%v]. Will try next requeue", statefulSet.ObjectMeta.Name))
}
}(statefulSet, *managedKubeSecret)
}
}
wg.Wait()
return 0, nil
@@ -94,6 +135,53 @@ func IsDeploymentUsingManagedSecret(deployment v1.Deployment, managedSecret v1al
return false
}
func IsDaemonSetUsingManagedSecret(daemonSet v1.DaemonSet, managedSecret v1alpha1.ManagedKubeSecretConfig) bool {
managedSecretName := managedSecret.SecretName
for _, container := range daemonSet.Spec.Template.Spec.Containers {
for _, envFrom := range container.EnvFrom {
if envFrom.SecretRef != nil && envFrom.SecretRef.LocalObjectReference.Name == managedSecretName {
return true
}
}
for _, env := range container.Env {
if env.ValueFrom != nil && env.ValueFrom.SecretKeyRef != nil && env.ValueFrom.SecretKeyRef.LocalObjectReference.Name == managedSecretName {
return true
}
}
}
for _, volume := range daemonSet.Spec.Template.Spec.Volumes {
if volume.Secret != nil && volume.Secret.SecretName == managedSecretName {
return true
}
}
return false
}
func IsStatefulSetUsingManagedSecret(statefulSet v1.StatefulSet, managedSecret v1alpha1.ManagedKubeSecretConfig) bool {
managedSecretName := managedSecret.SecretName
for _, container := range statefulSet.Spec.Template.Spec.Containers {
for _, envFrom := range container.EnvFrom {
if envFrom.SecretRef != nil && envFrom.SecretRef.LocalObjectReference.Name == managedSecretName {
return true
}
}
for _, env := range container.Env {
if env.ValueFrom != nil && env.ValueFrom.SecretKeyRef != nil && env.ValueFrom.SecretKeyRef.LocalObjectReference.Name == managedSecretName {
return true
}
}
}
for _, volume := range statefulSet.Spec.Template.Spec.Volumes {
if volume.Secret != nil && volume.Secret.SecretName == managedSecretName {
return true
}
}
return false
}
// This function ensures that a deployment is in sync with a Kubernetes secret by comparing their versions.
// If the version of the secret is different from the version annotation on the deployment, the annotation is updated to trigger a restart of the deployment.
func ReconcileDeployment(ctx context.Context, client controllerClient.Client, logger logr.Logger, deployment v1.Deployment, secret corev1.Secret) error {
@@ -121,6 +209,56 @@ func ReconcileDeployment(ctx context.Context, client controllerClient.Client, lo
return nil
}
func ReconcileDaemonSet(ctx context.Context, client controllerClient.Client, logger logr.Logger, daemonSet v1.DaemonSet, secret corev1.Secret) error {
annotationKey := fmt.Sprintf("%s.%s", DEPLOYMENT_SECRET_NAME_ANNOTATION_PREFIX, secret.Name)
annotationValue := secret.Annotations[constants.SECRET_VERSION_ANNOTATION]
if daemonSet.Annotations[annotationKey] == annotationValue &&
daemonSet.Spec.Template.Annotations[annotationKey] == annotationValue {
logger.Info(fmt.Sprintf("The [daemonSetName=%v] is already using the most up to date managed secrets. No action required.", daemonSet.ObjectMeta.Name))
return nil
}
logger.Info(fmt.Sprintf("DaemonSet is using outdated managed secret. Starting re-deployment [daemonSetName=%v]", daemonSet.ObjectMeta.Name))
if daemonSet.Spec.Template.Annotations == nil {
daemonSet.Spec.Template.Annotations = make(map[string]string)
}
daemonSet.Annotations[annotationKey] = annotationValue
daemonSet.Spec.Template.Annotations[annotationKey] = annotationValue
if err := client.Update(ctx, &daemonSet); err != nil {
return fmt.Errorf("failed to update daemonSet annotation: %v", err)
}
return nil
}
func ReconcileStatefulSet(ctx context.Context, client controllerClient.Client, logger logr.Logger, statefulSet v1.StatefulSet, secret corev1.Secret) error {
annotationKey := fmt.Sprintf("%s.%s", DEPLOYMENT_SECRET_NAME_ANNOTATION_PREFIX, secret.Name)
annotationValue := secret.Annotations[constants.SECRET_VERSION_ANNOTATION]
if statefulSet.Annotations[annotationKey] == annotationValue &&
statefulSet.Spec.Template.Annotations[annotationKey] == annotationValue {
logger.Info(fmt.Sprintf("The [statefulSetName=%v] is already using the most up to date managed secrets. No action required.", statefulSet.ObjectMeta.Name))
return nil
}
logger.Info(fmt.Sprintf("StatefulSet is using outdated managed secret. Starting re-deployment [statefulSetName=%v]", statefulSet.ObjectMeta.Name))
if statefulSet.Spec.Template.Annotations == nil {
statefulSet.Spec.Template.Annotations = make(map[string]string)
}
statefulSet.Annotations[annotationKey] = annotationValue
statefulSet.Spec.Template.Annotations[annotationKey] = annotationValue
if err := client.Update(ctx, &statefulSet); err != nil {
return fmt.Errorf("failed to update statefulSet annotation: %v", err)
}
return nil
}
func GetInfisicalConfigMap(ctx context.Context, client client.Client) (configMap map[string]string, errToReturn error) {
// default key values
defaultConfigMapData := make(map[string]string)