Cloud Services

S3
Secrets Manager
RDS
EKS
Redshift
IAM
SQS (for data ingestion into Superwise)

Requirements

Networking

A minimum of two private subnets for EKS
- One subnet per availability zone
- Range: /24 or larger
A minimum of two subnets for RDS:
- One subnet per availability zone
- Range: /28 or larger
A minimum of two subnets for Redshift:
- One subnet per availability zone
- Range: /28 or larger
- The subnets require outbound access to connect to the S3 bucket
- When using Enhanced VPC routing, a VPC endpoint for accessing S3 is required.
  The security group that is assigned to the Redshift cluster should have an egress rule to allow S3 traffic to the endpoint.
The following DNS records for accessing the Superwise Platform (the IP would be provided after the installation is completed):
- app.<**DOMAIN_NAME**>
- api.<**DOMAIN_NAME**>
A TLS certificate that matches the hostnames above, stored as a Kubernetes secret.
The subnet in which the load balancer is deployed must have one of the following tags:
- kubernetes.io/role/internal-elb: 1 for internal (private IP) load balancer in a private subnet
- kubernetes.io/role/elb: 1 for external (public IP) load balancer in a public subnet

Firewall

Ingress:
- Load balancer (UI/API access): 80, 443, 15021
Egress:
- Outbound internet access is required

EKS

Version: 1.25 or newer (up to 1.29)

Nodes

A minimum of 3 nodes (t3.xlarge/m5.xlarge) is required to run Superwise.
Node autoscaling is recommended with a 1 node per zone configuration, across 3 zones.
A maximum of 6 nodes (2 per zone) can be set without affecting performance.

Firewall

Enable egress traffic to 0.0.0.0/0 on all ports and protocols (outbound internet access).
Enable ingress traffic to the private subnets CIDR blocks for inter-cluster communication.
Enable ingress traffic from the cluster security group on ports 443-9443 for cluster control plane to nodes communication.

Network policy

Kubernetes network policies should be enabled for this cluster.

Components

Since EKS does not provide these capabilities out of the box, the following components must be installed on the cluster:

The AWS IAM Terraform module can be used for creating the EKS roles that can be used for authentication.

RDS

Engine: PostgreSQL

Version: 14.x

Class: db.t3.large/db.m5.large

Features

Storage autoscaling
Automated backups (recommended)
HA - multi AZ

Redshift

Node type: ra3.xlplus

Number of nodes: 2

Features

Automated snapshots (recommended)
Enhanced VPC routing
Availability zone relocation

Parameters

wlm_json_configuration: [{\"query_concurrency\":15}]
require_ssl: true
use_fips_ssl: false
max_concurrency_scaling_clusters: 3
enable_case_sensitive_identifier: true

IAM

Redshift loads data from Superwise S3 bucket files.
For this purpose, an IAM role for Redshift must be created and assigned to the cluster.

Assign the following policies to the Redshift role:

Assume role policy (Trust relationships)

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "redshift.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Policy for S3 bucket access

{
    "Statement": [
        {
            "Action": [
                "s3:GetObject"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::<BUCKET_NAME>/*"
        },
        {
            "Action": [
                "s3:ListBucket"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::<BUCKET_NAME>"
        }
    ],
    "Version": "2012-10-17"
}

arn:aws:iam::aws:policy/AmazonRedshiftAllCommandsFullAccess: allow Redshift to execute commands for loading data from other AWS services

S3

An S3 bucket is required for Superwise to store data.

Enabling file versioning is recommended.

Public access prevention should be enforced.

IAM

An EKS role (or IAM user) with the following permissions is required:

{
    "Statement": [
        {
            "Action": [
                "s3:PutObject",
                "s3:ListMultipartUploadParts",
                "s3:GetObject",
                "s3:DeleteObject",
                "s3:AbortMultipartUpload"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::<BUCKET_NAME>/*"
            ],
            "Sid": "S3ReadWrite"
        },
        {
            "Action": "s3:ListBucket",
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::<BUCKET_NAME>"
            ],
            "Sid": "S3List"
        }
    ],
    "Version": "2012-10-17"
}

EKS Role

Using an EKS role is recommended.
When using this option for authentication, the Kubernetes service accounts that requires access to AWS APIs must be added as trusted entities in the role.

To use EKS role for Superwise, the following service accounts must be allowed:

The Superwise service account. The default value is superwise:superwise.
argo-workflows:argo-workflows-workflow-controller
argo-workflows:argo-workflows-server

For using EKS role when adding S3 sources to Superwise, the following Kubernetes service accounts must be allowed:

triggermesh:triggermesh-controller
superwise:awss3source-adapter

Secrets

Superwise utilizes Kubernetes secrets for loading resources credentials.
These secrets can be created directly in Kubernetes or synced from other secret providers (like AWS Secrets Manager) using a variety of tools.

External Secrets Operator is recommended for this purpose as it is simple to use and supports most secret providers.
For setting up External Secrets Operator in an EKS cluster, check out the installation and AWS Secrets Manager provider guide.
The AWS IAM Terraform module can be used for creating an EKS role that can be used for authentication.