CloudVpc -- Quick Start v1.0.0

From an empty AWS account to Oracle Cloud machines serving as members of an AWS VPC, verified and torn down.

This quick start builds a complete CloudVpc deployment from one Terraform configuration, aws-asg-oci-pool: an Auto Scaling group of relays in a new AWS VPC, and an instance pool of workers in Oracle Cloud Infrastructure (OCI) that wear the relays' private addresses. You prove it works by sending requests through an AWS load balancer that are answered in OCI, scale it, and tear it down.

If you have not read it yet, the Overview explains what a relay and a worker are.

1. What you will build

  AWS -- new dual-stack VPC, two Availability Zones          OCI -- new VCN, IPv4 + IPv6
  +-----------------------------------------------+         +-----------------------------+
  |  Application Load Balancer  (port 80)         |         |                             |
  |        |                                      |         |                             |
  |        v  one target group                    |         |                             |
  |  relay Auto Scaling group (t4g.small, 2..6)   |         |  instance pool (A1.Flex)    |
  |    relay 1  10.43.0.x  ======== WireGuard ==============  worker 1 wears 10.43.0.x   |
  |    relay 2  10.43.1.x  ======== WireGuard ==============  worker 2 wears 10.43.1.x   |
  +-----------------------------------------------+         |  demo app on port 8080      |
                                                            +-----------------------------+
  • Each relay claims a rendezvous address from a pre-allocated IPv6 block in its subnet. The workers poll that block, enrol with a relay that answers, and wear its private IPv4 address.
  • Each worker runs a small demo web application on port 8080. The load balancer sends requests to the relays' private addresses; the relays forward them down the tunnel and the OCI workers answer.
  • The OCI pool follows the AWS Auto Scaling group: change the group's desired capacity and the pool grows or shrinks to match. There is no controller in this topology.
Costs. This creates billable resources in both clouds: two t4g.small relays, an Application Load Balancer, a DynamoDB table, two OCI VM.Standard.A1.Flex instances (1 OCPU and 6 GB each), plus Cloud Spectra usage on the relays (see Pricing). Tear it down when you are finished (step 8).

2. Prerequisites

Tools on your workstation

ToolWhy
Terraform 1.5 or laterApplies the configuration.
AWS CLI v2, with credentials for the target accountThe AWS provider uses the credentials; the verification commands call the CLI.
OCI credentials in ~/.oci/config (or the standard OCI_* environment variables)The OCI provider uses them.
curl and jqThe verification command uses them.

Accounts and permissions

  • AWS: permission to create a VPC with an IPv6 block, subnets, an internet gateway, security groups, an Application Load Balancer, a launch template and an Auto Scaling group, IAM roles, an instance profile and an IAM user with an access key, and a DynamoDB table.
  • OCI: a compartment where you can create a VCN, an instance configuration and an instance pool. The configuration also creates a dynamic group and a policy at tenancy level, so the workers can resize their own pool. That needs tenancy administrator rights, and OCI accepts it only in your tenancy's home region. If either is a problem, see OCI IAM without tenancy rights.

From Cloud Spectra

  • A licence key (it starts with CSTW1-). Sign in on your account page, choose the Developer or Enterprise lane and complete checkout; the key is shown once when checkout completes, so save it. A relay without a valid licence starts but forwards nothing.
  • The CloudVpc Terraform configurations for your release -- the cloudvpc/aws-asg-oci-pool configuration and the modules it uses -- and the relay image ID (the arm64 Cloud Spectra Gateway AMI) for the AWS Region you will use. Request both from support@cloudspectra.ai.
Enterprise lane: every relay needs at least 4 vCPUs, so also set relay_instance_type = "t4g.xlarge" in step 2. The default t4g.small has 2 vCPUs and is refused on that lane. Developer lane: an account may run at most 16 vCPUs, which is eight t4g.small relays.

Values to collect

ValueWhere to find it
OCI tenancy OCIDOCI Console, Tenancy details. Use the tenancy OCID itself: availability domains are listed against the tenancy, and a compartment OCID returns an empty list.
OCI compartment OCIDThe compartment the workers and their network go into.
Your OCI home regionOCI Console, Tenancy details. Use it as oci_region unless you set up the OCI IAM by hand.
Your workstation's public IPv4 addressFor example curl -s https://checkip.amazonaws.com. It opens the load balancer, and recovery SSH, to you only.

3. Configure

1Change into the configuration directory

cd cloudvpc/aws-asg-oci-pool
cp terraform.tfvars.example terraform.tfvars

terraform.tfvars.example lists every variable the configuration declares, with its default; the required ones are at the top. Keep the directory layout you received: the configuration refers to its modules by relative path.

2Set the values

Replace the contents of terraform.tfvars with the following, using your own values:

# --- required ------------------------------------------------------------
license_key          = "CSTW1-..."                          # your licence key
delivery_url         = "https://api.gateway.cloudspectra.ai"
package_version      = "1.0.0"                              # an exact version, never "latest"
oci_tenancy_ocid     = "ocid1.tenancy.oc1..aaaa..."
oci_compartment_ocid = "ocid1.compartment.oc1..aaaa..."

# Required by the configuration but not used by the install path:
# workers fetch the client through the Cloud Spectra downloader.
client_apt_base_url  = "https://downloads.cloudspectra.ai/gateway/apt"

# --- the relay image, and where workers download the client --------------
relay_ami_id         = "ami-0123456789abcdef0"              # arm64 Cloud Spectra Gateway AMI, your Region
public_artifact_base = "https://downloads.cloudspectra.ai"

# --- placement and naming ------------------------------------------------
name       = "cloudvpc-demo"                                # unique per deployment in these accounts
aws_region = "us-east-1"
oci_region = "us-ashburn-1"                                 # your OCI home region

# --- reach the demo application from your workstation --------------------
lb_internal = false
admin_cidrs = ["203.0.113.10/32"]                           # your public IPv4, as a /32
Keep terraform.tfvars and the Terraform state private. The licence key is a sensitive variable, so Terraform does not print it, but it is recorded in the state and placed on each worker so the worker can download the client.

What the less obvious settings do:

VariableEffect
delivery_urlThe Cloud Spectra API. Relays fetch their datapath from it, and the workers' downloader exchanges your licence there for a short-lived download token.
public_artifact_baseThe public host the workers download the Cloud Spectra downloader from. Set it as shown; the configuration's default is not the release host.
relay_ami_idThe relay image. The relays in this configuration are Graviton (arm64) instances, so the image must be arm64.
namePrefix for every resource the configuration creates, including IAM names, so two deployments in one account need different names.
lb_internalDefaults to true (an internal load balancer). Set to false here so you can reach the demo application from your workstation.
admin_cidrsWho may reach the load balancer and the relays' recovery SSH ports. Empty by default, so nothing is open until you add a range.
oci_ssh_public_keyOptional. The SSH key installed on the relays and the workers for debugging. When it is unset, ~/.ssh/id_rsa.pub is used if it exists.

4. Deploy

3Initialise and apply

terraform init
terraform apply

Review the plan and confirm. Terraform creates the AWS VPC, the relay Auto Scaling group, the load balancer and the peer map table, then the OCI network and the worker pool.

4Let the fleet pair

When terraform apply returns, the instances are still booting. Each relay starts and claims its rendezvous address; each worker downloads the client, finds a relay over IPv6, enrols and brings its tunnel up. Watch the relays claim their addresses:

eval "$(terraform output -raw watch_claims)"

The table lists each network interface in the VPC with its IPv6 rendezvous address and its private IPv4 address. A relay with an IPv6 address listed has claimed its slot.

5. Verify

5Check the load balancer targets

eval "$(terraform output -raw watch_targets)"

Every relay should be registered and healthy. The health check is the relay's own health endpoint, and the Auto Scaling group replaces a relay that keeps failing it.

6Send requests through AWS and see OCI answer

terraform output -raw app_url
eval "$(terraform output -raw verify)"

Each line reports the cloud that served the request and the AWS instance the worker presents itself as:

{"served":"oci","as":"i-0a1b2c3d4e5f60718"}
{"served":"oci","as":"i-0f9e8d7c6b5a41302"}

Two things in that output are the product:

  • served is oci: the request entered an AWS load balancer and was answered by a machine in OCI.
  • as is an AWS instance ID -- its relay's. The worker read it from the VPC's instance metadata service through the tunnel. To the VPC, the worker is that instance.
Nothing answering? Give the workers a few minutes after the relays are healthy, then see Troubleshooting. The most common causes are a wrong relay_ami_id (for example an x86 image), public_artifact_base left at its default, and missing OCI IAM.

6. Scale (optional)

7Change one number

The relay Auto Scaling group is the only place the fleet size is decided:

aws autoscaling set-desired-capacity \
  --auto-scaling-group-name "$(terraform output -raw asg_name)" --desired-capacity 3

A third relay launches and claims an address; the workers read the group's size, grow the OCI pool to match, and the new worker pairs with the new relay. Run verify again to see a third instance ID. Scale back to 2 the same way: the worker whose relay left removes itself from the pool. relay_min and relay_max (2 and 6 by default) bound the capacity.

7. Tear down

8Destroy everything this configuration created

terraform destroy

This removes the OCI pool and network, the relays, the load balancer, the peer map table and the VPC. If the destroy stops with DependencyViolation and a message that the network has some mapped public address(es), the relays are still terminating and their public addresses are still mapped, so the internet gateway cannot be detached yet. Wait a minute and run terraform destroy again.

Destroying the deployment does not cancel your Cloud Spectra subscription; manage that on your account page.

8. Next steps

  • Topologies -- run the workers in DigitalOcean, or join them to an Amazon EKS cluster as Kubernetes nodes.
  • Scaling on pair load -- let the relay group scale on the load of each relay-worker pair.
  • Configuration reference -- every input and output of this configuration.
  • FAQ -- common questions and known limitations.