Cirrus
The operator that runs web applications on Community Cloud — one WebApplication resource for an app's database, bucket, queue, hostnames and GitLab releases, and DnsProviders for publishing them.
What Cirrus does
Cirrus runs web applications on Community Cloud. A WebApplication says what an application needs, and Cirrus builds it out of the services the cluster already runs:
| Asked for | What Cirrus makes | Through |
|---|---|---|
| always | A Deployment and a Service | Kubernetes |
hostnames |
A ListenerSet on cc-letsencrypt-web-gateway, a route, an HTTP→HTTPS redirect, and a certificate |
The Gateway API, cert-manager |
postgres: {} |
A role and a database in the namespace's own Postgres, cirrus-postgres |
CloudNativePG |
bucket: {} |
A bucket, an S3 user limited to it, and its keys | The SeaweedFS operator |
rabbitmq: {} |
A vhost, and a user limited to it | RabbitMQ's topology operator |
migrations |
A Job, run with each new image before the rollout | Kubernetes |
source.gitlab |
A project webhook; each successful pipeline on ref rolls out |
GitLab's API |
It also runs ExternalDNS for each DnsProvider, so a namespace's hostnames are put in DNS without anyone editing records.
Meteorologist is the web console for all of this: it creates and edits WebApplications through a form, and sets up namespaces, GitLab connections and DNS.
Installing it
Cirrus is the installer's cirrus package. It runs in the community-cloud namespace, and enabling it brings in the services it builds on — cert-manager, CloudNativePG, SeaweedFS and the RabbitMQ operator:
"packages": { "cirrus": true }
GitLab's webhooks reach Cirrus at values.domain.cirrus. If they can't, Cirrus still finds new pipelines by asking GitLab every ten minutes.
A WebApplication
apiVersion: community-cloud.technology/v1alpha1
kind: WebApplication
metadata:
name: shop
namespace: acme
spec:
image:
repository: registry.gitlab.com/acme/shop
tag: main
port: 3000
healthCheck:
path: /up
hostnames: [shop.acme.example]
postgres: {}
bucket:
quota: 5Gi
rabbitmq: {}
migrations:
command: ["bin/rails", "db:migrate"]
source:
gitlab:
project: acme/shop
ref: main
tokenSecret:
name: gitlab-token # key "token" by default
imageTag: "{commit_sha}" # or "{short_sha}", "{ref}", combined freely
The least there is, an image on two hostnames with no source:
apiVersion: community-cloud.technology/v1alpha1
kind: WebApplication
metadata:
name: brochure
namespace: marketing
spec:
image:
repository: ghcr.io/acme/brochure
tag: "2.4.1"
port: 80
hostnames:
- acme.example
- www.acme.example
spec also takes replicas, env, resources and imagePullSecrets, as a Deployment would.
Checking on it
kubectl get webapp shows the deployed image, readiness and address. kubectl describe webapp shop shows its conditions and what each is waiting on:
| Condition | What it means |
|---|---|
Provisioned |
The database, bucket, queue and routes it asked for exist |
RolledOut |
The newest release is running. False names the migration Job that failed |
WebhookRegistered |
GitLab has a webhook for the project |
ReleaseChecked |
Cirrus could ask GitLab for the newest release. It tries again within a couple of minutes, and meanwhile runs the last release it found |
Ready |
Both Provisioned and RolledOut |
What the application is given
Nobody copies passwords around. Each service the app asks for reaches it as environment variables:
- Postgres: the Secret
<app>-postgres, loaded whole, withDATABASE_URL,PGHOST,PGPORT,PGDATABASE,PGUSERandPGPASSWORD. The role and the database are both called<app>, with any-as_. - Bucket:
S3_ENDPOINT,S3_BUCKET,S3_REGION,S3_ACCESS_KEY_ID,S3_SECRET_ACCESS_KEY, and theAWS_*equivalents that AWS's SDKs read on their own. The bucket is called<namespace>-<app>. Address it by path, not by virtual host. - RabbitMQ: the Secret
<app>-rabbitmq, loaded whole, withAMQP_URLandRABBITMQ_HOST,_PORT,_USERNAME,_PASSWORDand_VHOST.
Anything in spec.env comes after these, so it wins.
Postgres
Each namespace has a Postgres of its own rather than a share of cc-postgres: a CloudNativePG Cluster, cirrus-postgres, made when the first application in the namespace asks for a database and shared by every later one. Keeping it in the namespace keeps it with the application. A namespace pinned to a region has every pod in it pinned there, Postgres's included, so an application never queries a database in another region.
Cirrus makes the Cluster once and then leaves it alone. Grow its volume, add instances or set up backups as on any CloudNativePG Cluster, and Cirrus won't put them back. It isn't owned by any application, so it outlives the last one; delete it by hand when the namespace is done with it.
The namespace
Two of Community Cloud's capabilities are labels on a namespace. Cirrus checks for them rather than adding them itself:
| Label | Needed for |
|---|---|
whereWereGoing: "weDontNeedRoads" |
hostnames — the cluster Gateway refuses listeners from any other namespace |
storage.community-cloud.technology/buckets: "true" |
bucket — SeaweedFS refuses buckets from any other namespace |
Without the label, Provisioned is False and says which label is missing. Meteorologist shows these as switches on each namespace.
Releases from GitLab
GitLab has no webhook for an image being pushed, so Cirrus listens for successful pipelines on the tracked ref instead. When one arrives:
- Cirrus checks the hook's token, which is the application's own and is kept in
<app>-cirrus-webhook. - It works out the tag from
imageTagand asks GitLab's registry API for the tag's digest. An image outside the project's GitLab registry is rolled out by tag. - It runs the migration Job,
<app>-migrate-<hash>, with the new image. The old version keeps serving while the Job runs. - Once the Job completes, the Deployment gets the new image, by digest.
If the migration fails, RolledOut is False with the Job's name, and the old version stays. Each image's migrations run once; to rerun them, delete the Job.
Webhooks can get lost, so Cirrus also checks for the newest successful pipeline every ten minutes. Until a release arrives, the application runs spec.image.
The GitLab token in tokenSecret needs the api scope and the Maintainer role on the project, so Cirrus can manage the project's webhook and read its registry. A group access token is usually the best choice: it reaches every project in the group and keeps working whoever leaves.
Deleting an application
Everything in the application's namespace is owned by the WebApplication and is removed with it. What's in cc-office — the RabbitMQ vhost, user and permission — and the GitLab webhook are removed by Cirrus's finalizer.
Data is kept. CloudNativePG keeps the database, the namespace's Postgres stays, and SeaweedFS keeps the bucket. Removing a postgres, bucket or rabbitmq block from a live WebApplication stops Cirrus maintaining those resources but doesn't delete them.
Publishing DNS
A DnsProvider is one DNS provider account, the domains it holds, and the namespaces whose hostnames it publishes. It's cluster-scoped, since which namespace may write which domain is the cluster's decision, not a namespace's:
apiVersion: community-cloud.technology/v1alpha1
kind: DnsProvider
metadata:
name: acme
spec:
provider: cloudflare # any ExternalDNS provider
domains: [acme.example]
excludeDomains: [] # parts given to another provider
namespaces: [acme] # one ExternalDNS each
credentialsSecret: cirrus-dns-acme # the default, in external-dns
target: 203.0.113.7 # optional: what records point at
policy: sync # or upsert-only, which never deletes
For each namespace Cirrus runs an ExternalDNS in external-dns, cirrus-dns-<provider>-<namespace>, that reads only that namespace's routes and writes only the provider's domains. Every key of the credentials Secret becomes one of its environment variables, named as the provider's documentation names them. The credentials live in external-dns, where no application runs, so access to a namespace doesn't mean being able to read the token:
A DnsProvider runs nothing, and says why in its Ready condition, when:
Invalid— it has no provider or domains, or an unknown policy.DomainConflict— another DnsProvider publishing different namespaces holds the same domain, or one containing it without excluding it.MissingCredentials— the credentials Secret isn't inexternal-dns.
DnsProviders sit beside the installer's own dns section without touching its records. Use the installer for the cluster's own domains, and DnsProviders for the domains of the people using it — which can then be added from Meteorologist without re-running a bundle. Don't give the same domain to both: neither sees the other's.
Deleting a DnsProvider removes its instances. With the default sync policy, the records they made are left behind, since nothing is left to delete them.
Not yet
- Redis. The shared Redis is divided by database number, and handing out numbers safely needs a design of its own.
- Forges other than GitLab.
- Leader election. Cirrus runs as one replica.
The operator's own README, with its configuration and development notes, is operators/cirrus in the repository.