Cloud Services

• Cloud Storage
• Secrets Manager
• Cloud SQL
• GKE
• BigQuery
• IAM
• Pub/Sub (for data ingestion into Superwise)

The following APIs must be enabled in the project

• compute.googleapis.com
• container.googleapis.com
• iam.googleapis.com
• cloudresourcemanager.googleapis.com
• sqladmin.googleapis.com
• servicenetworking.googleapis.com
• secretmanager.googleapis.com
• bigquerydatatransfer.googleapis.com

Requirements

Networking

• A subnet for GKE:
  • Range: /24 or larger
  • Secondary ranges:
    • Pods: /16 or larger
    • Services: /16 or larger
      The services range cannot be increased after the cluster creation
• A subnet CIDR for GKE control plane (will be created by GKE):
  • Range: /28
• The following DNS records for accessing the Superwise Platform (the IP would be provided after the installation is completed):
  • app.<**DOMAIN_NAME**>
  • api.<**DOMAIN_NAME**>
  • *.app.<**DOMAIN_NAME**> - a wildcard DNS for accessing applications.
• A TLS certificate that matches the hostnames above, stored as a Kubernetes secret.

Firewall

• Ingress:
  • Load balancer (UI/API access): 80, 443, 15021
• Egress:
  • Outbound internet access is required

GKE

Version: 1.24 or newer (up to 1.29)

Nodes

• A minimum of 3 nodes (e2-standard-4) is required to run Superwise.
• Node autoscaling is recommended with a 1 node per zone configuration, across 3 zones.
• A maximum of 6 nodes (2 per zone) can be set without affecting performance.

Firewall

• admission/webhook controllers ports: 443, 6443, 8443, 9090, 9443, 15017

Network policy

Calico Kubernetes Network policy should be enabled for control plane and nodes in this cluster.

CloudSQL

Type: PostgreSQL

Version: 14.x

Tier: db-custom-2-7680

Features

• Auto storage increase
• Automated backups (recommended)
• HA - regional

Database Flags

• temp_file_limit: 2147483647
• max_connections: 200000

Cloud Storage

A Cloud Storage bucket is required for Superwise to store data.

A multi-region bucket with versioning is recommended.

Public access prevention should be enforced.

IAM

A GCP service account with the following permissions is required:

• Project level:
  • roles/bigquery.admin
• Bucket level:
  • roles/storage.objectAdmin
  • roles/storage.legacyBucketReader

🚧

Pay attention!

A service account key is also required, stored as a Kubernetes secret.

GKE workload identity

Enabling GKE workload identity for this service account is recommended.
When using this option for authentication, the Kubernetes service accounts that requires access to Google Cloud APIs must be allowed to impersonate the Google service account.
This can be done by adding the roles/iam.workloadIdentityUser role to the service account.

To use GKE workload identity for Superwise, the following service accounts must be allowed:

• The Superwise service account. The default value is superwise/superwise.
• argo-workflows/argo-workflows-workflow-controller
• argo-workflows/argo-workflows-server

For using GKE workload identity when adding GCS sources to Superwise, the following Kubernetes service accounts must be allowed:

• triggermesh/triggermesh-controller
• superwise/googlecloudstoragesource-adapter

Secrets

Superwise utilizes Kubernetes secrets for loading resources credentials.
These secrets can be created directly in Kubernetes or synced from other secret providers (like GCP Secrets Manager) using a variety of tools.

External Secrets Operator is recommended for this purpose as it is simple to use and supports most secret providers.
For setting up External Secrets Operator in an EKS cluster, check out the installation and GCP Secrets Manager provider guide.
The GKE Workload Identity module can be used for creating a service account that can be used for authentication. The roles/secretmanager.secretAccessor role should be assigned to this service account.