Skip to main content

Run a Public Fullnode on GCP

This tutorial explains how to configure and deploy a public fullnode to connect to the Aptos devnet using Google Cloud (GCP). Running a public fullnode in the cloud usually provides better stability and availability compared to running it on your laptop. If you are looking to deploy a production grade public fullnode, we recommend you to deploy it on the cloud.

Alternative methods for running a public fullnode

Read Public Fullnode if you want other options for deployment. Using cloud comes with a cost, and it varies depending on how you configure it.

Prerequisites​

You can run the commands in this guide to deploy your FullNode on Google Kubernetes Engine from any machine you want: From a VM on GCP, or Google Cloud Shell, or your personal computer.

The following packages are pre-installed with Cloud Shell. Make sure to review the documentation around ephemeral mode if you choose to use Cloud Shell.

However, if you are running the installation from your laptop or another machine, you need to install:

After you have installed the gcloud CLI, log into GCP using gcloud:

gcloud auth login --update-adc
Already have a GCP account set up?

If you already have a GCP account setup, skip to Getting Started. If you do not have a GCP account, then follow the below sections to create and configure your GCP account.

GCP setup​

Sign up for the 90-day free trial​

Google Cloud offers a [90 day 300 free trial for every new user](https://cloud.google.com/free/docs/gcp-free-tier/#free-trial). These 300 are given as credits to your account and you can use them to get a sense of Google Cloud products. Be aware that you will need to add payment information when signing up for the free trial. This is for identity verification purposes and will not incur charges until you upgrade to a paid account and run out of credits.). Some GCP feature such as GPUs and Windows servers are not available in the free trial.

Sign up for GCP $300 credits

Sign up for the $300 in credits here.

Create a new GCP project​

Enable billing, upgrade your account​

You will still be able to use the free trial credits, but enabling billing allows you to have full access to all the features of GCP and not experience any interruption to your nodes.

Upgrade your account by following the steps outlined here.

Additional GCP resources​

This should be enough to get your GCP setup ready to start deploying your fullnode. But if you are brand new to GCP, you may want to check out some of our quickstart guides and Google Cloud Skills Boost.

Getting started​

Before you proceed

From here on, this guide assumes that you have already set up your GCP account, and have created a new project for deploying Aptos public fullnode. If you don't, check out the instructions above for GCP Setup.

You can deploy a public fullnode on GCP by using the Aptos fullnode Terraform module.

  1. Create a working directory for your configuration.

    • Choose a workspace name e.g. devnet. Note: This defines the Terraform workspace name, which in turn is used to form resource names.
    export WORKSPACE=devnet
    • Create a directory for the workspace
    mkdir -p ~/$WORKSPACE

  2. Create a storage bucket for storing the Terraform state on Google Cloud Storage. Use the console or this gcs command to create the bucket. The name of the bucket must be unique. See the Google Cloud Storage documentation here: https://cloud.google.com/storage/docs/creating-buckets#prereq-cli.

gsutil mb gs://BUCKET_NAME
# for example
gsutil mb gs://<project-name>-aptos-terraform-dev
  1. Create Terraform file called main.tf in your working directory:
cd ~/$WORKSPACE
touch main.tf
  1. Modify the main.tf file to configure Terraform and create a public fullnode from the Terraform module.

Note: If you are using a different version of Terraform, you will need to use the tfenv command to change the required version.

You can find the Docker image tag at https://hub.docker.com/r/aptoslabs/validator/tags?page=1&ordering=last_updated&name=devnet

Example content for main.tf:

terraform {
  required_version = "~> 1.3.6"
  backend "gcs" {
    bucket = "BUCKET_NAME" # bucket name created in step 2
    prefix = "state/fullnode"
  }
}

module "fullnode" {
  # download Terraform module from aptos-labs/aptos-core repo
  source        = "github.com/aptos-labs/aptos-core.git//terraform/fullnode/gcp?ref=main"
  region        = "us-central1"  # Specify the region
  zone          = "c"            # Specify the zone suffix
  project       = "gcp-fullnode" # Specify your GCP project ID
  fullnode_name = "BUCKET_NAME" #bucket name created in step 2
  era           = 1              # bump era number to wipe the chain
  image_tag     = "devnet" # Specify the docker image tag
}
  1. Initialize Terraform in the same directory of your main.tf file:
terraform init

This will download all the Terraform dependencies into the .terraform folder.

  1. Create a new Terraform workspace to isolate your environments:
terraform workspace new $WORKSPACE
# This command will list all workspaces
terraform workspace list
  1. Apply the configuration:
terraform apply

This might take a while to finish (10 - 20 minutes), Terraform will create all the resources on your cloud account.

Validation​

Once Terraform apply finished, you can follow this section to validate your deployment.

  1. Configure your Kubernetes client to access the cluster you just deployed:
gcloud container clusters get-credentials aptos-$WORKSPACE --zone <region_zone_name> --project <project_name>
# for example:
gcloud container clusters get-credentials aptos-devnet --zone us-central1-a --project aptos-fullnode
  1. Check that your public fullnode pods are now running (this may take a few minutes):
kubectl get pods -n aptos

You should see this:

NAME                       READY   STATUS    RESTARTS   AGE
devnet0-aptos-fullnode-0   1/1     Running   0          56s
  1. Get your public fullnode IP:
kubectl get svc -o custom-columns=IP:status.loadBalancer.ingress -n aptos

You should see this:

[map[ip:104.198.36.142]]
  1. Check the REST API, make sure that the ledger version is increasing:
curl http://<IP>/v1
# Example command syntax: curl http://104.198.36.142/v1

You should see this:

{"chain_id":25,"epoch":"22","ledger_version":"9019844","oldest_ledger_version":"0","ledger_timestamp":"1661620200131348","node_role":"full_node","oldest_block_height":"0","block_height":"1825467"}

  1. To verify the correctness of your public fullnode, as outlined in the section Verify the correctness of your FullNode, you will need to:

    • Set up a port-forwarding mechanism directly to the aptos pod in one ssh terminal, and
    • Test it in another ssh terminal.

    Follow the below steps:

    • Set up the port-forwarding to the aptos-fullnode pod. Use kubectl get pods -n aptos to get the name of the pod:

      kubectl port-forward -n aptos <pod-name> 9101:9101
      # for example:
      kubectl port-forward -n aptos devnet0-aptos-fullnode-0 9101:9101

    • Open a new ssh terminal. Execute the following curl calls to verify the correctness:

      curl -v http://0:9101/metrics 2> /dev/null | grep "aptos_state_sync_version{type=\"synced\"}"

      curl -v http://0:9101/metrics 2> /dev/null | grep "aptos_connections{direction=\"outbound\""

    • Exit port-forwarding when you are done by entering control-c in the terminal.

Configure identity and seed peers​

Static identity​

If you want to configure your node with a static identity, first see the Network Identity For Fullnode for how to generate the keys, and then follow the below instructions to configure your Terraform file.

  1. Generate your own private key, and extract peer id, following the guide Creating a static identity for a fullnode.

  2. Modify the main.tf to add fullnode_identity in fullnode_helm_values. This will configure the keys for public fullnode, for example:

module "fullnode" {
  # download Terraform module from aptos-labs/aptos-core repo
  source        = "github.com/aptos-labs/aptos-core.git//terraform/fullnode/gcp?ref=main"
  region        = "us-central1"  # Specify the region
  zone          = "c"            # Specify the zone suffix
  project       = "gcp-fullnode" # Specify your GCP project name
  era           = 1              # bump era number to wipe the chain
  image_tag     = "devnet"       # Specify the docker image tag to use

  fullnode_helm_values = {
    chain = {
      name = "devnet"
    }
    # create fullnode from this identity config, so it will always have same peer id and address
    fullnode_identity = {
      type = "from_config"
      key = "B8BD811A91D8E6E0C6DAC991009F189337378760B55F3AD05580235325615C74"
      peer_id = "ca3579457555c80fc7bb39964eb298c414fd60f81a2f8eedb0244ec07a26e575"
    }
  }
}
  1. Apply Terraform changes:
terraform apply

Add upstream seed peers​

You can add upstream seed peers to allow your node state sync from a specific. This is helpful when the public fullnode is not able to connect to the network due to congestion.

  1. Obtain the upstream peer id information. You can either use the one listed in the Connecting your fullnode to seed peers, or grab one from the Aptos Discord #advertise-full-node channel; these are the nodes hosted by our community.

  2. Modify the main.tf to add seeds for devnet in fullnode_helm_values. This will configure the upstream seeds for public fullnode. For example:

module "fullnode" {
    # download Terraform module from aptos-labs/aptos-core repo
    source        = "github.com/aptos-labs/aptos-core.git//terraform/fullnode/gcp?ref=main"
    region        = "us-central1"  # Specify the region
    zone          = "c"            # Specify the zone suffix
    project       = "gcp-fullnode" # Specify your GCP project name
    era           = 1              # bump era number to wipe the chain
    image_tag     = "dev_5b525691" # Specify the docker image tag to use

    fullnode_helm_values = {
      # add a list of peers as upstream
      aptos_chains = {
        devnet = {
          seeds = {
            "bb14af025d226288a3488b4433cf5cb54d6a710365a2d95ac6ffbd9b9198a86a" = {
            addresses = ["/dns4/pfn0.node.devnet.aptoslabs.com/tcp/6182/noise-ik/bb14af025d226288a3488b4433cf5cb54d6a710365a2d95ac6ffbd9b9198a86a/handshake/0"]
            role = "Upstream"
            },
            "7fe8523388084607cdf78ff40e3e717652173b436ae1809df4a5fcfc67f8fc61" = {
            addresses = ["/dns4/pfn1.node.devnet.aptoslabs.com/tcp/6182/noise-ik/7fe8523388084607cdf78ff40e3e717652173b436ae1809df4a5fcfc67f8fc61/handshake/0"]
            role = "Upstream"
            },
            "f6b135a59591677afc98168791551a0a476222516fdc55869d2b649c614d965b" = {
            addresses = ["/dns4/pfn2.node.devnet.aptoslabs.com/tcp/6182/noise-ik/f6b135a59591677afc98168791551a0a476222516fdc55869d2b649c614d965b/handshake/0"]
            role = "Upstream"
            }
          }
        }
      }
    }
  }
  1. Apply Terraform changes:
terraform apply

Check logging​

To check the logs of the pod, use the following commands:

# Get a list of the pods
kubectl get pods -n aptos

# Get logs of the pod
kubectl logs <pod-name> -n aptos
# for example:
kubectl logs devnet0-aptos-fullnode-0 -n aptos

When using GKE, the logs of the cluster and pod will automatically show up in the Google Cloud console. From the console menu, choose Kubernetes Engine. From the side menu, choose Workloads. You will see all the pods from the cluster listed.

GKE Workloads screenshot

The devnet0-aptos-fullnode is the pod that is running the aptos fullnode container. Click on the pod to view details. You will see some metrics and other details about the pod.

GKE Workloads Pod screenshot

Click the LOGS tab to view the logs directly from the pod. If there are errors in the pod, you will see them here.

GKE Workloads Pod Logs screenshot

Click the open in new window icon to view the logs in the Log Explorer. This screen allows advanced searching in the logs.

GKE Workloads Pod Logs Explorer screenshot

Other logging insights are available in the Logs Dashboard

GKE Workloads Pod Logs Dashboard screenshot

Additional features are available through Cloud Logging, including creating log-based metrics, logging sinks and log buckets.

Check monitoring​

Google cloud captures many metrics from the cluster and makes them easily viewable in the console. From the console menu, choose Kubernetes Engine. Click on the cluster that aptos is deployed to. Click on the Operations link at the top right. Click on the Metrics sub-tab to view specific cluster metrics.

GKE Monitoring metrics screenshot

Click the View in Cloud Monitoring link at the top to view the built-in GKE dashboard for the cluster.

GKE Monitoring dashboard screenshot

Google Cloud Monitoring has many other features to easily monitor the cluster and pods. You can configure uptime checks for the services and configure alerts for when the metrics reach a certain threshold.