Cloud Spectra Gateway -- Architecture v1.0.0

This page describes the internal architecture of Cloud Spectra Gateway at the level an enterprise architecture review expects: where the software runs, how packets and requests flow through it, how it is configured and operated, and how it isolates failure domains and scales. Cloud Spectra Gateway is delivered through the AWS Marketplace and deploys entirely inside your own AWS account.

1. Design principles

Cloud Spectra Gateway is built on a small number of principles that shape every other design decision. The product replaces metered AWS networking and LLM-API spend with a fixed EC2 cost -- Your Cloud, Off the Meter -- without introducing a vendor-operated control plane or moving your data out of your account.

In your own AWS account

The gateway is an EC2 appliance that runs in the customer's account, in the customer's VPC, under an IAM instance role that the customer can inspect. There is nothing to "connect to" outside your account boundary for the data plane to function.

No vendor control plane

There is no Cloud Spectra-operated SaaS backend that your traffic or configuration passes through. The management dashboard, the configuration API, and the data plane all run on the appliance itself; configuration is persisted to AWS SSM Parameter Store inside your account. If the vendor disappeared tomorrow, the gateway you already deployed keeps routing traffic.

graph LR
    subgraph TYPICAL["Typical SaaS network/AI gateway"]
        direction TB
        TC["Your VPC
workloads"] -->|traffic + config| VCP["Vendor control plane
(outside your account)"]
        VCP --> TINT["Internet / LLM APIs"]
    end
    subgraph CS["Cloud Spectra Gateway"]
        direction TB
        CC["Your VPC
workloads"] --> CGW["Cloud Spectra appliance
(EC2 in YOUR account)"]
        CGW --> CINT["Internet / LLM APIs"]
        CGW -.config.-> SSM["SSM Parameter Store
(YOUR account)"]
    end
    style VCP fill:#fecaca,stroke:#ef4444,color:#991b1b
    style CGW fill:#d1fae5,stroke:#10b981,color:#065f46
    style SSM fill:#dbeafe,stroke:#3b82f6,color:#1e3a8a

Data sovereignty

Because the appliance lives in your account, packets, proxied HTTP, firewall logs, and -- for the AI Gateway -- prompts, completions, and the cache all stay within your account's boundary. Outbound calls go directly from your appliance to their destination (the internet, or an LLM provider you configure). Nothing transits a Cloud Spectra-operated service on the way.

Fixed cost vs metered

AWS managed networking services bill per-hour and per-GB. By running the equivalent functions on EC2 you pay for compute you control: a fixed instance cost (optionally on Spot) plus the Marketplace software fee, with no per-GB data-processing meter on the appliance itself.

2. Tier model

One AMI ships every capability; the compiled product catalog in the backend decides what an instance may activate. There are three tiers on one cumulative ladder. The enforcement rank is Platform(1) < Network(2) < Security(3): a feature gated at rank N is available at rank N and every higher rank.

The two Compute worker tiers

The CPU and GPU worker stacks are AMI-only capabilities of the Platform tier. Their value is the pre-installed stack in the image, not a Cloud Spectra data plane: they gate no configurable backend feature and have no runtime settings. You launch the AMI as a worker node and it joins your cluster.

• The CPU worker stack ships a lean Kubernetes worker AMI: the containerd CRI runtime, kubeadm/kubelet ready to join an existing cluster, and Calico pod networking (CNI) pre-installed.
• The GPU worker stack ships that same worker stack plus the GPU layer: a pinned NVIDIA driver, the CUDA toolkit and NVIDIA container toolkit, the vLLM (OpenAI-compatible) model-serving runtime, and the NVIDIA device plugin so Kubernetes can schedule GPUs.

ElasticApps: a separate axis

ElasticApps is not a smaller network tier -- it answers a different question. Instead of forwarding other workloads' traffic, it takes one stateful open-source application and makes it cost-elastic through vertical right-sizing, Spot, a sticky EBS volume, a stable endpoint, and multi-AZ placement. Because its rank (3) sits below Network, the three network tiers are strict supersets: a Network, Security, or AI Gateway deployment can also run an elastic app. Section 9 covers the architecture.

The three network tiers

Network -> Security -> AI Gateway are cumulative supersets of one another, and are what the rest of this page describes.

• Network (10 features) -- Sticky Secondary ENI; Sticky EBS Volume; Per-Slot Route Table; Source NAT; Destination NAT / port forwarding; Network Load Balancer (IPVS L4); TLS termination (ACM); Per-AZ Auto Scaling; vertical + horizontal scaling behind a Gateway Load Balancer; forward HTTP proxy + caching (Squid).
• Security (10 features) -- Suricata IDS/IPS (inline); nftables firewall rules; domain / URL filtering; URL-category filtering; ET Open threat detection; multi-source community threat feeds; TLS inspection (SSL bump); SIEM export (OCSF/CEF/syslog); antivirus scanning (ClamAV ICAP); L7 App-ID (nDPI). See the data plane.
• AI Gateway (3 features) -- AI LLM Proxy with response caching; AI Assistant; AI Semantic Cache. See section 6.

Sticky EBS is the one universal primitive: its floor is Platform, so every tier has it. Sticky Secondary ENI is a networking-appliance mechanism and is gated to the Network tier. Both are opt-in and default off.

3. Deployment topology

A deployment consists of public-facing networking (an Elastic IP for a stable endpoint, a Gateway Load Balancer for horizontal scale) in front of a per-AZ fleet of appliance instances managed by EC2 Auto Scaling. Each Availability Zone runs its own Auto Scaling Group and egresses through its own elastic network interface (ENI), so steady-state traffic never crosses an AZ boundary and never incurs cross-AZ data charges.

New-VPC vs existing-VPC models

The product ships two base CloudFormation templates, plus a standalone-AMI path:

The full topology below shows a two-AZ existing-VPC deployment. Private workloads route through the gateway ENI in their own AZ; the appliances themselves egress to the internet via the public subnet.

graph TD
    EIP["Elastic IP
(stable endpoint)"]
    IGW["Internet Gateway"]
    GWLB["Gateway Load Balancer
(GENEVE)"]

    subgraph VPC["Customer VPC"]
        direction TB
        subgraph AZA["Availability Zone A"]
            direction TB
            PUBA["Public subnet A"]
            PRIA["Private subnet A"]
            ENIA["Gateway ENI A"]
            ASGA["Auto Scaling Group A
(1..N appliance instances)"]
            WLA["Private workloads A"]
            PRIA --> WLA
            WLA -->|"default route 0.0.0.0/0"| ENIA
            ENIA --- ASGA
            ASGA --- PUBA
        end
        subgraph AZB["Availability Zone B"]
            direction TB
            PUBB["Public subnet B"]
            PRIB["Private subnet B"]
            ENIB["Gateway ENI B"]
            ASGB["Auto Scaling Group B
(1..N appliance instances)"]
            WLB["Private workloads B"]
            PRIB --> WLB
            WLB -->|"default route 0.0.0.0/0"| ENIB
            ENIB --- ASGB
            ASGB --- PUBB
        end
        SSM["SSM Parameter Store
(configuration)"]
    end

    PUBA --> IGW
    PUBB --> IGW
    EIP --- IGW
    GWLB -. horizontal scale .- ASGA
    GWLB -. horizontal scale .- ASGB
    ASGA -.reads/writes config.-> SSM
    ASGB -.reads/writes config.-> SSM
    IGW --> INET["Internet / upstream APIs"]

    style EIP fill:#dbeafe,stroke:#3b82f6,color:#1e3a8a
    style GWLB fill:#ede9fe,stroke:#8b5cf6,color:#5b21b6
    style ASGA fill:#d1fae5,stroke:#10b981,color:#065f46
    style ASGB fill:#d1fae5,stroke:#10b981,color:#065f46
    style SSM fill:#fef3c7,stroke:#f59e0b,color:#92400e

Sticky Secondary ENI Network tier

Route stability across instance replacement can be pinned to the interface rather than the instance. The Sticky Secondary ENI feature -- opt-in and default off, available from the Network tier up -- pre-provisions an ENI per Auto Scaling Group slot with a fixed private IP, and attaches it to whichever instance currently occupies that slot. When the ASG replaces an instance -- a health-check failure, a Spot interruption, a vertical resize -- the ENI (with its security group, its routes, and its disabled source/destination check) detaches from the outgoing instance and re-attaches to the incoming one at the same address.

The architectural consequence, where the feature is enabled, is that your route tables never have to be rewritten. A route that pointed at the slot ENI yesterday still points at a live gateway today, because the identity workloads depend on is the interface, not the instance behind it. Each non-master instance gets an additional recyclable secondary ENI for its slot; the master keeps its own dedicated re-homing data-plane ENI.

Per-Slot Route Table Network tier

Paired with the sticky ENI, and likewise opt-in, is a dedicated route table per slot, carrying a 0.0.0.0/0 route to that slot's ENI. You associate a private subnet with the slot route table once, at deploy time, and the association is thereafter permanent: the gateway maintains the route inside the table, and the ENI the route targets is itself stable across replacements.

Where this pairing is in use, it keeps the control plane out of your subnets' route tables during normal operation. Instance churn is absorbed one level down -- at the ENI attachment -- rather than being propagated upward as route rewrites, which is both faster and a much smaller IAM grant than letting the appliance mutate arbitrary route tables.

4. Data plane

The data plane is where customer packets are forwarded, source-NATed, inspected, filtered, and load-balanced. It runs in the Linux kernel and in user-space services on each appliance instance. A private instance's outbound packet takes the following path.

flowchart LR
    SRC["Private instance
(in AZ A)"] -->|"default route"| ENI["Per-AZ gateway ENI A"]
    ENI --> NFT["nftables
(stateful allow/deny)"]
    NFT -->|"queued for inspection"| SUR["Suricata IDS/IPS
(inline, NFQUEUE)
ET Open + community feeds"]
    SUR -->|"verdict: accept"| APPID["App-ID (nDPI)
L7 application identity"]
    APPID -->|"application allowed"| PROXY{"Forward HTTP
proxy? (Squid)"}
    PROXY -->|"not proxied"| SNAT["Source NAT (sNAT)"]
    PROXY -->|"proxied"| BUMP["TLS inspection
(ssl-bump, optional)"]
    BUMP --> FILT["Domain / URL /
category filtering"]
    FILT --> ICAP["ClamAV via ICAP
(RESPMOD scan)"]
    ICAP -->|"clean + cached"| SNAT
    SNAT --> OUT["Public subnet -> IGW -> Internet"]
    SUR -.->|"verdict: drop"| DROP["Dropped + logged"]
    NFT -.->|"deny rule match"| DROP
    APPID -.->|"blocked application"| DROP
    FILT -.->|"blocked domain/category"| DROP
    ICAP -.->|"infected"| DROP
    DROP -.->|"events"| SIEM["SIEM export
(OCSF / CEF / syslog)"]

    style ENI fill:#d1fae5,stroke:#10b981,color:#065f46
    style NFT fill:#dbeafe,stroke:#3b82f6,color:#1e3a8a
    style SUR fill:#ede9fe,stroke:#8b5cf6,color:#5b21b6
    style APPID fill:#ede9fe,stroke:#8b5cf6,color:#5b21b6
    style BUMP fill:#ede9fe,stroke:#8b5cf6,color:#5b21b6
    style FILT fill:#ede9fe,stroke:#8b5cf6,color:#5b21b6
    style ICAP fill:#ede9fe,stroke:#8b5cf6,color:#5b21b6
    style PROXY fill:#fef3c7,stroke:#f59e0b,color:#92400e
    style SIEM fill:#fef3c7,stroke:#f59e0b,color:#92400e
    style DROP fill:#fecaca,stroke:#ef4444,color:#991b1b

Outbound path stage by stage

1. Default route to the per-AZ ENI. The private subnet's route table sends 0.0.0.0/0 to the gateway ENI in the same AZ.
2. nftables. Stateful allow/deny policy (source/destination CIDR, port, protocol, connection state) is enforced in the Linux kernel before any further processing. Security
3. Suricata. Accepted packets are handed inline to Suricata via NFQUEUE for intrusion detection/prevention; Suricata returns an accept or drop verdict at the data plane. Security
4. App-ID. A standalone nDPI daemon identifies the application behind the flow, so egress policy can be written against applications rather than ports. Security
5. Forward proxy (optional). If a workload is configured to use the Squid forward proxy, HTTP/HTTPS flows pass through it for authentication, response caching, domain and category filtering, optional TLS inspection, antivirus scanning, and bandwidth limiting.
6. Source NAT. The packet is source-NATed to the appliance's address and leaves via the public subnet and internet gateway, presenting the stable Elastic IP to the internet.

Return path

Reply traffic returns to the appliance's connection-tracking state, is reverse-NATed back to the originating private instance, and is delivered over the same per-AZ ENI. The kernel conntrack table keeps the flow pinned to the instance that established it, so a long-lived connection is handled coherently for its lifetime.

sequenceDiagram
    autonumber
    participant W as Private workload (AZ A)
    participant G as Appliance (AZ A)
    participant I as Internet endpoint
    W->>G: SYN to 0.0.0.0/0 via ENI A
    Note over G: nftables -> Suricata -> (proxy) -> sNAT
    G->>I: SYN (source = appliance / EIP)
    I-->>G: SYN-ACK
    Note over G: conntrack maps reply -> original flow
    G-->>W: SYN-ACK (reverse NAT to workload)
    W->>G: data ...
    G->>I: data ... (same flow, same instance)

Inbound and load-balanced paths

The data plane also supports inbound and in-appliance load balancing:

• Destination NAT / port forwarding (dNAT) forwards inbound TCP to private targets.
• In-appliance L4 load balancing uses Linux IPVS, kept in sync with an AWS Network Load Balancer target set.
• TLS termination is handled by HAProxy on port 443 using an AWS Certificate Manager (ACM) certificate, with an HTTP-to-HTTPS redirect frontend on port 8180 (8181 if 8180 is the configured virtual port).

Security enforcement stages Security tier

The Security tier adds inline inspection and enforcement to the same packet path, at two distinct places: in the kernel path (nftables, then Suricata over NFQUEUE, then the App-ID daemon), and in the proxy path (TLS inspection, then filtering, then the ICAP antivirus scan). Every one of these stages runs on the appliance in your account -- no packet, file, or event is sent to a vendor for analysis, and no per-sensor vendor licence is required for any of it.

TLS inspection (SSL bump) Security tier

Inspection stages can only act on what they can read, and most egress is TLS. TLS inspection solves that with transparent ssl-bump: the proxy terminates the client's TLS session using a certificate it mints on the fly, opens its own session to the origin, and hands the cleartext to the inline IPS and the proxy filters. By default the signing CA is self-managed on the appliance; AWS Private CA is an opt-in alternative. Either way the private keys are generated and held in your account and never leave it.

Trust distribution is the deployment consideration. In v1 the CA is auto-trusted onto EC2 instances via SSM; Kubernetes pods, containers, and on-premises clients must be given the CA out of band, through whatever mechanism already manages their trust stores.

IDS/IPS and threat feeds Security tier

Suricata runs inline on the NFQUEUE path and returns a verdict per packet, so a match can drop the traffic rather than merely record it. What it matches against is supplied from two free, in-account sources:

• ET Open threat detection. The Emerging Threats (ET Open) rule sets, cached in the AMI so a fresh instance is protected from first boot rather than after a rule download.
• Multi-source community threat feeds. Additional free open-source inline-IPS feeds -- abuse.ch, CINS Army, and the Spamhaus DROP list -- combined into the same enforcement path.

Both carry $0 vendor licensing and are consumed entirely in-account, which is the architectural point: threat coverage that does not add a per-sensor meter on top of the instance cost, and does not require shipping your traffic to a vendor to be evaluated.

Domain, URL, and category filtering Security tier

Destination policy is enforced at two granularities. Domain / URL filtering is policy-driven egress control on both the firewall and the Squid proxy path, for the destinations you name explicitly. URL-category filtering covers what you cannot enumerate by hand: whole categories such as adult, gambling, malware, and anonymizer are blocked using the free UT1 category lists.

Category filtering operates at the domain level, so it deliberately needs no TLS decryption -- the destination name is visible without it. That makes it the low-friction control to turn on first: it works on HTTPS egress before you have taken on the CA-distribution work that TLS inspection requires.

Antivirus scanning (ClamAV ICAP) Security tier

Files downloaded through the forward proxy are scanned in-account. The proxy hands each response to ClamAV over ICAP (a c-icap RESPMOD service); an infected object is blocked before it reaches the requesting workload. The whole exchange is local to the appliance, so files never leave your account to be scanned -- the difference between this and a cloud AV service is precisely that no copy of your data is uploaded anywhere.

Because it operates on the proxy's view of the response body, HTTPS downloads are only scannable when TLS inspection is enabled; otherwise only cleartext HTTP is covered.

L7 App-ID (nDPI) Security tier

Port numbers are a poor proxy for intent, so application identity is determined by deep packet inspection instead. A standalone nDPI daemon -- deliberately a separate process from Suricata, with nDPI shipped as a dynamically linked LGPL shared library -- classifies each flow's application and feeds that into egress policy.

Architecturally this buys two things: hard blocks on applications that exist to evade network policy (Tor, BitTorrent, anonymizing proxies), and signals on traffic that merely looks wrong -- DGA-style domain generation, self-signed certificates, command-and-control patterns -- which are flagged for the operator rather than silently allowed because they used port 443.

SIEM export (OCSF / CEF / syslog) Security tier

Enforcement decisions are only useful if they reach the place your security team already looks. IDS, firewall, and audit events are streamed to an external SIEM -- Splunk, Microsoft Sentinel, IBM QRadar, or anything that accepts these formats -- in the vendor-neutral OCSF, CEF, or syslog encodings.

The export runs from the appliance directly to the destination you configure, so it stays consistent with the rest of the model: the events are generated in your account and delivered to a system you own, with no Cloud Spectra service in the path and no lock-in to a proprietary event schema.

5. Control plane

The control plane is how operators configure the gateway and how the fleet keeps itself consistent. It is entirely in-account: an Angular dashboard served over HTTPS, a configuration REST API, configuration persisted to SSM Parameter Store, and the per-AZ Auto Scaling Group lifecycle.

graph TD
    OP["Operator / Terraform"] -->|"HTTPS (nginx, dashboard.port; default 9443)"| DASH["Angular dashboard + Config API
(on the appliance)"]
    DASH -->|"reads/writes"| SSM["SSM Parameter Store
(desired configuration)"]
    SSM -->|"poll for changes"| INST["Appliance instances
(all AZs)"]
    INST -->|"apply"| DP["Data-plane services
(nftables, Suricata, Squid,
IPVS, HAProxy, sNAT/dNAT)"]
    ASG["Per-AZ Auto Scaling Groups"] -->|"launch / replace / scale"| INST
    INST -->|"new instance reads config on boot"| SSM

    style DASH fill:#dbeafe,stroke:#3b82f6,color:#1e3a8a
    style SSM fill:#fef3c7,stroke:#f59e0b,color:#92400e
    style INST fill:#d1fae5,stroke:#10b981,color:#065f46
    style ASG fill:#ede9fe,stroke:#8b5cf6,color:#5b21b6

Dashboard and configuration API

The management dashboard is an Angular single-page application served over HTTPS by nginx on the port set by spec.dashboard.port (CF parameter dashboard0000port), which defaults to 9443 -- 443 is deliberately left free for HAProxy TLS termination or a load balancer, and an operator who does not need that can set dashboard.port to 443 to serve on the standard HTTPS port. Every dashboard action is backed by the configuration REST API (port 8080), so the same operations are scriptable. The Terraform provider (cloudspectra/cloudspectra, installed via a one-time ~/.terraformrc network-mirror block) drives that API as well, so the gateway can be managed as code.

Configuration in SSM Parameter Store

Desired configuration is the source of truth and is stored in SSM Parameter Store in your account. The dashboard and API write configuration there; appliance instances read it. This decoupling is what makes the fleet stateless: any instance can be replaced, and a freshly launched instance reads current configuration on boot and converges to it.

Per-AZ Auto Scaling Group lifecycle

Each AZ's Auto Scaling Group launches, health-checks, and replaces instances independently. Because desired state lives in SSM rather than on any single box, the lifecycle is simple: a new instance boots from the AMI, reads configuration, programs its data plane, and (behind the Gateway Load Balancer) begins taking traffic. A terminated instance is replaced without operator action.

6. AI Gateway architecture Security tier

The AI Gateway is an OpenAI-compatible reverse proxy for LLM traffic, exposed on port 8090. Clients point their OpenAI base URL at the gateway; it applies caching, meters tokens, and routes each request to Amazon Bedrock, OpenAI, or Anthropic. The interface follows the OpenAI API reference, so existing SDKs work by changing only the base URL.

Caching layers above routing

Two cache layers sit in front of routing, so the most expensive operation -- calling a model -- is skipped whenever possible:

• Exact-match response cache: identical requests return a stored response with no upstream call.
• Semantic cache: an embedding-based similar-prompt cache that raises hit rates beyond exact match by matching prompts that are equivalent in meaning, not just byte-identical.

flowchart TD
    REQ["Client request
(OpenAI-compatible, :8090)"] --> AUTH["Auth + token metering"]
    AUTH --> EXACT{"Exact-match
cache hit?"}
    EXACT -->|"yes"| HIT["Return cached response"]
    EXACT -->|"no"| SEM{"Semantic
cache hit?"}
    SEM -->|"yes"| HIT
    SEM -->|"no"| ROUTE["Routing layer
(by requested model name)"]
    ROUTE --> REMOTE["Provider
Bedrock / OpenAI / Anthropic"]
    REMOTE --> STORE["Store in caches"]
    STORE --> RESP["Response to client"]
    HIT --> RESP

    style EXACT fill:#dbeafe,stroke:#3b82f6,color:#1e3a8a
    style SEM fill:#dbeafe,stroke:#3b82f6,color:#1e3a8a
    style REMOTE fill:#ede9fe,stroke:#8b5cf6,color:#5b21b6

Routing

In v1, routing is by requested model name to a configured provider: Amazon Bedrock, OpenAI, or Anthropic. Because the proxy and both cache layers run on the appliance in your account, a prompt that is served from cache never leaves your account at all, and an uncached prompt goes directly from your appliance to the provider you chose -- never through a Cloud Spectra service.

Cached vs uncached request, side by side

sequenceDiagram
    autonumber
    participant C as Client (OpenAI SDK)
    participant G as AI Gateway (:8090)
    participant K as Cache (exact + semantic)
    participant M as Model (Bedrock / OpenAI / Anthropic)
    Note over C,M: Uncached request
    C->>G: POST /v1/chat/completions
    G->>K: lookup (exact, then semantic)
    K-->>G: miss
    G->>M: forward request
    M-->>G: completion
    G->>K: store response
    G-->>C: completion (+ token usage)
    Note over C,M: Subsequent equivalent request
    C->>G: POST /v1/chat/completions
    G->>K: lookup
    K-->>G: hit
    G-->>C: cached completion (no model call)

AI Assistant Security tier

The AI tier also turns the gateway's own operational surface into something you can ask questions of. The AI Assistant is an in-dashboard assistant, backed by Amazon Bedrock, that answers questions about gateway status, configuration, and troubleshooting. It runs in your account; no vendor control plane sees your data.

Architecturally it is a consumer of the same components documented elsewhere on this page rather than a new plane: it is served by the dashboard on spec.dashboard.port (default 9443), it reads the gateway's own status and the configuration in SSM Parameter Store, and it reaches Bedrock through the instance's own credentials -- the same Bedrock access path the LLM proxy uses, and the same path your security team can review in the IAM role. The assistant is gated by the Security tier entitlement; it has no separate runtime setting, and it is not in the traffic path -- an instance below Security simply does not expose it.

Local AI inference (vLLM) Roadmap

The planned design serves a model addressed as local/<model> in-account by vLLM on GPU instances, OpenAI-compatible like the remote providers, with automatic fallback to Bedrock. On a pre-first-token failure of the local model (for example, capacity pressure), an overflow policy would govern what happens next:

7. High availability & scaling

Availability and scale come from three independent mechanisms: per-AZ isolation, horizontal scale behind the Gateway Load Balancer, and live vertical resize -- all fronted by a stable Elastic IP endpoint.

Per-AZ isolation

Each Availability Zone is its own failure domain: a dedicated subnet, gateway ENI, and Auto Scaling Group. An incident confined to one AZ does not take down egress for workloads in other AZs, and keeping each AZ's traffic in-zone avoids cross-AZ data charges.

Horizontal scale (GWLB)

Within an AZ, the Gateway Load Balancer distributes flows across the instances in that AZ's Auto Scaling Group using the GENEVE protocol. Adding instances increases aggregate throughput; the GWLB spreads flows so no single instance is a throughput ceiling for the AZ.

Vertical resize and the stable endpoint

Instance size can be changed live to give each instance more CPU and network bandwidth. Throughout scale-out, scale-in, and resize, the Elastic IP provides a stable public endpoint, so external dependencies (allow-lists, DNS, partner integrations) see one unchanging address.

graph TD
    EIP["Stable Elastic IP
(unchanging endpoint)"] --> GWLB["Gateway Load Balancer (GENEVE)"]
    GWLB --> I1["Instance 1"]
    GWLB --> I2["Instance 2"]
    GWLB --> I3["Instance N (scale out)"]
    subgraph SCALE["Scaling dimensions"]
        H["Horizontal: add/remove instances
(GWLB spreads flows)"]
        V["Vertical: resize instance type
(more CPU / bandwidth)"]
        Z["Per-AZ: independent ASG + ENI
per Availability Zone"]
    end
    GWLB -.- H
    I1 -.- V
    GWLB -.- Z
    style EIP fill:#dbeafe,stroke:#3b82f6,color:#1e3a8a
    style GWLB fill:#ede9fe,stroke:#8b5cf6,color:#5b21b6
    style I1 fill:#d1fae5,stroke:#10b981,color:#065f46
    style I2 fill:#d1fae5,stroke:#10b981,color:#065f46
    style I3 fill:#d1fae5,stroke:#10b981,color:#065f46

8. IAM & data-residency model

This section covers how the appliance is permitted to act and where your data lives: least-privilege execution inside your account, data that stays in your account, and encryption in transit. For the inline inspection and enforcement features of the Security tier, see Security enforcement stages in the data plane.

Least-privilege instance role

The appliance runs under an IAM instance role scoped to the AWS actions it actually needs -- for example, reading and writing its configuration parameters in SSM Parameter Store, managing the network interfaces and routes it operates, and, where enabled, accessing ACM for the TLS certificate and Bedrock for AI routing. The role is created in your account by the CloudFormation template and is fully visible to your security team for review.

graph LR
    ROLE["IAM instance role
(least privilege, in your account)"] --> SSMP["SSM Parameter Store
(read/write own config)"]
    ROLE --> NET["EC2 networking
(ENIs, routes it operates)"]
    ROLE --> ACMR["ACM
(TLS certificate, when enabled)"]
    ROLE --> BR["Amazon Bedrock
(AI routing, when enabled)"]
    style ROLE fill:#dbeafe,stroke:#3b82f6,color:#1e3a8a
    style SSMP fill:#fef3c7,stroke:#f59e0b,color:#92400e
    style NET fill:#d1fae5,stroke:#10b981,color:#065f46
    style ACMR fill:#ede9fe,stroke:#8b5cf6,color:#5b21b6
    style BR fill:#ede9fe,stroke:#8b5cf6,color:#5b21b6

Cross-account / home-account role (base vs operational)

The least-privilege instance role above is intentionally minimal. From it the gateway boots, elects a leader, associates its Elastic IP, completes its Auto Scaling launch hook, and runs outbound source NAT on its primary interface. Most other capabilities are deliberately moved out of the base role into a separately deployed operational IAM role -- the cross-account / home-account role. The backend's single AWS access path assumes this operational role and, by design, refuses to fall back to the base instance role -- so a feature whose operational role is not deployed is simply inert until it is.

The operational stack creates one inline-policy role per feature, named <name>-cross-account-roleN (the gateway rebuilds the name at runtime to assume it). Every such role trusts only two principals in your home account -- the gateway instance role and the CloudFormation handler role -- gated by an external ID and, optionally, your AWS Organizations ID. The same template is deployed once in your home account and once in each member account you manage; the home-account stack additionally grants the gateway instance role permission to assume those roles. Permissions are updated over time through CloudFormation Change Sets the gateway stages and you execute -- never by the gateway editing IAM at runtime.

graph LR
    INST["Gateway instance role
(base, minimal)"] -->|"boot, NAT,
master EIP"| BASEOK["Boot-to-ready"]
    INST -->|"sts:AssumeRole
+ external ID"| OPS["Operational roles
<name>-cross-account-roleN"]
    CFH["CF handler role"] -->|"sts:AssumeRole"| OPS
    OPS --> FEAT["Most features
(NAT data plane, GWLB,
DNS, scaling, AI Gateway)"]
    OPS -.->|"member accounts"| SPOKE["Spoke account roles
(same template)"]
    style INST fill:#dbeafe,stroke:#3b82f6,color:#1e3a8a
    style CFH fill:#dbeafe,stroke:#3b82f6,color:#1e3a8a
    style BASEOK fill:#d1fae5,stroke:#10b981,color:#065f46
    style OPS fill:#fef3c7,stroke:#f59e0b,color:#92400e
    style FEAT fill:#d1fae5,stroke:#10b981,color:#065f46
    style SPOKE fill:#ede9fe,stroke:#8b5cf6,color:#5b21b6

In-account execution and where data lives

All processing happens on the appliance in your account. Configuration lives in your SSM Parameter Store; firewall and proxy logs, NAT state, and AI prompts/completions/caches reside on resources you own. There is no off-account control plane that your traffic or configuration transits.

Encryption in transit

The management dashboard and configuration API are served over HTTPS by nginx on spec.dashboard.port (default 9443), with TLS terminated using the per-instance certificate generated on the appliance at first boot into /etc/nginx/certs; port 80 only redirects to the dashboard port. ACM-backed TLS termination is a separate, workload-facing feature: for the services you configure the gateway to front, HAProxy terminates TLS on port 443 using an ACM-managed certificate.

9. ElasticApps architecture Platform tier

The ElasticApps tier reuses the same in-account, SSM-backed control model described above, but inverts the topology. Where the network gateway is a horizontally scaled, per-AZ fleet behind a Gateway Load Balancer, ElasticApps runs a single stateful application on one instance and makes it cost-elastic by changing which instance that is. The design goal is a stateful app that is expensive to run always-on and hard to scale horizontally (a database, a search or analytics engine, a dashboard) getting the cost profile of an elastic service, with no clustering and no application-level load balancer.

Topology: a multi-AZ singleton

The runtime is a single Auto Scaling Group with min=max=desired=1 that spans several Availability Zone subnets. Because there is only one box, no traffic crosses an AZ boundary, so spanning AZs is a pure win: more Spot capacity pools to draw from and resilience to a single-AZ failure. There is no Gateway Load Balancer -- a single instance has nothing to load-balance across. The application's state lives on a dedicated EBS volume, and its address is a stable Elastic IP plus a managed private Route 53 DNS name.

graph TD
    EIP["Elastic IP + private DNS
(stable endpoint)"]
    IGW["Internet Gateway"]
    subgraph VPC["Customer VPC"]
        direction TB
        subgraph AZA["Availability Zone A"]
            INST["ElasticApps instance
(selected app + nginx :443)"]
            VOL["Sticky EBS volume
(app data directory)"]
            INST --- VOL
        end
        subgraph AZB["Availability Zone B"]
            SPARE["Spare capacity /
failover target"]
        end
        ASG["Singleton ASG
(min=max=desired=1, spans A+B)"]
        SSM["SSM Parameter Store
(app + scaling config)"]
    end
    ASG --- INST
    ASG -. "can place in" .- SPARE
    EIP --> INST
    INST --> IGW
    IGW --> INET["Internet / clients"]
    INST -.reads config.-> SSM
    style INST fill:#dbeafe,stroke:#3b82f6,color:#1e3a8a
    style VOL fill:#fef3c7,stroke:#f59e0b,color:#92400e
    style EIP fill:#d1fae5,stroke:#10b981,color:#065f46
    style ASG fill:#ede9fe,stroke:#8b5cf6,color:#5b21b6
    style SSM fill:#fef3c7,stroke:#f59e0b,color:#92400e

The right-sizing control loop

Vertical resize is driven by a control-plane reconcile loop, not by AWS Auto Scaling adjusting a desired count (the count is pinned at one). The loop reads the desired application and scaling policy from SSM, compares live load against the instance ladder thresholds, and, when a move is warranted, replaces the instance at the new size. The replacement is done by publishing a new launch-template version and letting the ASG relaunch the singleton on it -- deliberately not an in-place StopInstances/ModifyInstanceAttribute/StartInstances, which would fight the ASG health check. The sticky volume detaches from the outgoing instance and re-attaches to the incoming one; on-demand versus Spot is selected by the ASG mixed-instances policy, so the same path also switches purchase options.

flowchart TD
    POLL["Control-plane reconcile loop"] --> READ["Read desired app +
scaling policy (SSM)"]
    READ --> METRIC{"Live load vs
ladder thresholds?"}
    METRIC -->|"above scale-up"| BIG["Target = next larger rung"]
    METRIC -->|"below scale-down"| SMALL["Target = next smaller rung"]
    METRIC -->|"within band"| HOLD["No change"]
    BIG --> LT["New launch-template version
(target instance type)"]
    SMALL --> LT
    LT --> REPL["ASG relaunches the singleton
detach volume -> re-attach -> terminate old"]
    REPL --> ENDP["Re-associate Elastic IP + upsert DNS
refresh cert if the endpoint moved"]
    ENDP --> POLL
    HOLD --> POLL
    style REPL fill:#dbeafe,stroke:#3b82f6,color:#1e3a8a
    style ENDP fill:#d1fae5,stroke:#10b981,color:#065f46
    style LT fill:#ede9fe,stroke:#8b5cf6,color:#5b21b6
    style HOLD fill:#fef3c7,stroke:#f59e0b,color:#92400e

State and endpoint durability

The property that makes the frequent instance replacement safe is that nothing unique lives on the instance. The data directory is on the sticky EBS volume, which is re-attached rather than rebuilt. The certificate's subject alternative name is built on the managed DNS name rather than the private IP, so it survives a move between AZs (where the private IP changes) without reissue. On every move the control plane re-associates the Elastic IP and upserts the private DNS record, so external clients and in-VPC clients keep using one unchanging address. On a Spot interruption the replacement is biased to the volume's AZ so the zero-loss re-attach is the normal path; a cross-AZ restore from snapshot is the escalation (roadmap).

Where ElasticApps sits relative to the network tiers

The application catalog is baked into the gateway AMI as dormant software; tier entitlement -- enforced by the compiled product catalog in the backend, not by a separate image -- governs what an instance may actually activate and use. The elasticity primitives ElasticApps depends on are universal; the network-appliance primitives are gated higher, so the network tiers are a strict superset that can also run an elastic app.

Operationally, ElasticApps follows the same base-versus-operational IAM split described in section 8. The base instance role stays minimal; the sticky-volume attach/detach, Elastic IP association, DNS upsert, and (roadmap) snapshot actions are granted through a dedicated, selectively scoped cross-account / home-account role rather than added to the base template.

10. Fleet monitoring Roadmap

The planned design follows the same principle as the rest of the product: it would run entirely in your account, so no telemetry leaves your boundary.

Master-hosted scrape

In the planned design, each gateway instance runs a lightweight metrics exporter (host and NIC metrics). The primary (leader) instance -- the same instance that holds the master Elastic IP -- runs a Prometheus server that discovers every instance across all per-AZ Auto Scaling Groups (by tag) and scrapes them, relabeling each series with its ASG, instance, Availability Zone, and Region. A Grafana instance on the primary would serve the dashboards through the existing dashboard reverse proxy at /grafana/. Pull-based scrape is deliberate: the primary already reaches every instance, there would be a single time-series database, and nothing would be pushed onto the data-plane path.

graph TD
    OP["Operator
(dashboard HTTPS,
dashboard.port)"] -->|"/grafana/ (auth-proxy SSO)"| GRAF["Grafana
(primary only, loopback)"]
    GRAF --> PROM["Prometheus TSDB
(primary only, resource-capped)"]
    subgraph FLEET["Fleet across all per-AZ ASGs"]
        NE1["exporter
(instance, AZ A)"]
        NE2["exporter
(instance, AZ B)"]
        NE3["exporter
(instance, AZ N)"]
    end
    DISC["Fleet discovery
(tag-based, cross-ASG)"] -.targets.-> PROM
    PROM -->|"scrape (pull)"| NE1
    PROM -->|"scrape (pull)"| NE2
    PROM -->|"scrape (pull)"| NE3
    style GRAF fill:#dbeafe,stroke:#3b82f6,color:#1e3a8a
    style PROM fill:#ede9fe,stroke:#8b5cf6,color:#5b21b6
    style NE1 fill:#d1fae5,stroke:#10b981,color:#065f46
    style NE2 fill:#d1fae5,stroke:#10b981,color:#065f46
    style NE3 fill:#d1fae5,stroke:#10b981,color:#065f46
    style DISC fill:#fef3c7,stroke:#f59e0b,color:#92400e

Guardrails on the data plane

Because the primary instance is also a data-plane node, the monitoring stack is designed to be bounded so it can never compete with NAT, firewall, or AI traffic: Prometheus and Grafana would run only on the primary (every other instance runs just the exporter); both are resource-capped (CPU and memory limits); and the time-series database has a bounded retention window. Gating the server components to the single primary also means they start in exactly one place, not on every instance.

What would be collected

The exporter is designed to publish standard host metrics (CPU, memory, disk, filesystem, network) plus AWS ENA allowance counters via the ethtool collector -- bandwidth, packets-per-second, connection-tracking, and link-local allowance drops that are otherwise invisible to CloudWatch and are the clearest signal that an instance is network-bound. A textfile metric would also publish the effective data-processing price per GB versus the AWS NAT and firewall meters, driving a cost-savings dashboard. See Fleet monitoring & analytics in the User Guide.

Alerting and remote-write export of these metrics to an external store (Grafana Cloud, Datadog, or a self-hosted backend) are a later step on the same roadmap.