透過 Terraform 建立私有 GKE Cluster

讓我們一起來花個十分鐘，我會手把手地教你使用 Terraform ，建立私有的 GKE Cluster ！

設定 GCP

建立 Service Account 與 Key

建立 Service Account

首先我們會需要先到 Google Cloud 的頁面上，建立一個 Service Account 給 Terraform 使用，位置可以參考上圖的第 1 2 3 步驟，如果你已經有 Service Account 的話，你可以跳過這個步驟。

權限的部分，我先給到 Editor ，因為我只是要 POC 而已，如果你是要建立在 Production 的話，記得翻一下文件，把權限設定的剛剛好就好。

建立 Key

第 4 步驟是生成一把 Key ，這把 Key 稍等一下在寫 Terraform 的時候會用到。

建立 GKE

1. 設定變數

我們開始撰寫 Terraform 啦！

我們先來設定一些變數，提供等下讓你可以快速複製貼上將 GKE 建立起來。

# GCP 的 Project ID
variable "gcp_project_id" {
  default = "alan-zhan-proj"
}

# GKE 的建立位置
variable "gcp_region" {
  default = "asia-east1"
}

variable "subnet_cidr" {
  default = "10.0.0.0/24"
}

variable "pod_cidr" {
  default = "10.0.8.0/21"
}

variable "service_cidr" {
  default = "10.0.1.0/24"
}

# master 的 CIDR 必須是 /28
variable "master_cidr" {
  default = "10.0.2.0/28"
}

2. 設定 Provider

再來建立一個 provider.tf 檔案，並且將下面內容填入。

terraform {
  required_providers {
    google = {
      source = "hashicorp/google"
    }
  }
}

provider "google" {
  project = var.gcp_project_id
  region  = var.gcp_region
  # 這邊填入你的剛剛下載下來的 key
  credentials = "./xxx.json"
}

3. Init Project

第二步，透過 terraform init 將 Terraform 專案初始化。

terraform init

Initializing the backend...

Initializing provider plugins...
- Finding latest version of hashicorp/google...
- Installing hashicorp/google v4.56.0...
- Installed hashicorp/google v4.56.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

4. 設定 Service API

Private GKE 在建立的時候，會需要 Compute 以及 Container 這兩個 Service ，所以我們建立一個 service.tf ，並且把下面內容填入。

resource "google_project_service" "compute" {
  project = var.gcp_project_id
  service = "compute.googleapis.com"
}

resource "google_project_service" "container" {
  project = var.gcp_project_id
  service = "container.googleapis.com"
}

5. 設定 Network

建立一個 network.tf ，並且設定 Network 相關的設定。

resource "google_compute_network" "gke" {
  name = "gke"
  auto_create_subnetworks = false
  delete_default_routes_on_create = false
  mtu = 1460
  routing_mode = "REGIONAL"
}

resource "google_compute_subnetwork" "gke" {
  name          = "gke-subnet"
  ip_cidr_range = var.subnet_cidr
  region        = var.gcp_region
  network       = google_compute_network.gke.id
  private_ip_google_access = true

  secondary_ip_range {
    range_name    = "k8s-pod-range"
    ip_cidr_range = var.pod_cidr
  }
  secondary_ip_range {
    range_name    = "k8s-service-range"
    ip_cidr_range = var.service_cidr
  }
}

resource "google_compute_router" "gke" {
  name    = "gke"
  region  = var.gcp_region
  network = google_compute_network.gke.id
}

resource "google_compute_router_nat" "gke" {
  name   = "nat"
  router = google_compute_router.gke.name
  region = var.gcp_region

  source_subnetwork_ip_ranges_to_nat = "LIST_OF_SUBNETWORKS"
  nat_ip_allocate_option             = "MANUAL_ONLY"

  subnetwork {
    name                    = google_compute_subnetwork.gke.id
    source_ip_ranges_to_nat = ["ALL_IP_RANGES"]
  }

  nat_ips = [google_compute_address.gke.self_link]
}

resource "google_compute_address" "gke" {
  name         = "nat"
  address_type = "EXTERNAL"
  network_tier = "PREMIUM"
  region = var.gcp_region

  depends_on = [google_project_service.compute]
}

6. Service Account

建立 service-account.tf ，建立一個給 GKE node 專用的 service account 。

resource "google_service_account" "gke" {
  account_id = "kubernetes"
}

7. GKE

建立 container.tf ，最後我們來建立 GKE 吧！不過我會順便建立 spot 的 node 。

在 Terraform 建立 GKE 的過程中，一定會建立預設的 Node ，不管你是不是有設定 remove_default_node_pool = true，在 GKE 建立成功後才會刪除預設的 Node。

resource "google_container_cluster" "gke" {
  name                     = "primary"
  location                 = var.gcp_region
  remove_default_node_pool = true
  initial_node_count       = 1
  network                  = google_compute_network.gke.self_link
  subnetwork               = google_compute_subnetwork.gke.self_link
  logging_service          = "logging.googleapis.com/kubernetes"
  monitoring_service       = "monitoring.googleapis.com/kubernetes"
  networking_mode          = "VPC_NATIVE"

  # 選填，可以客制 node 的位置
#  node_locations = [
#    ""
#  ]

  addons_config {
    http_load_balancing {
      disabled = true
    }
    horizontal_pod_autoscaling {
      disabled = true
    }
  }

  release_channel {
    channel = "REGULAR"
  }

#  workload_identity_config {
     # 這個是給 Pod 在做 
#    workload_pool = "xxx.svc.id.goog"
#  }

  ip_allocation_policy {
    cluster_secondary_range_name  = "k8s-pod-range"
    services_secondary_range_name = "k8s-service-range"
  }

  private_cluster_config {
    enable_private_nodes    = true
    enable_private_endpoint = false
    master_ipv4_cidr_block  = var.master_cidr
  }
}

resource "google_container_node_pool" "general" {
  name       = "general"
  cluster    = google_container_cluster.gke.id
  node_count = 1

  management {
    auto_repair  = true
    auto_upgrade = true
  }

  node_config {
    preemptible  = false
    machine_type = "e2-small"

    labels = {
      role = "general"
    }

    service_account = google_service_account.gke.email
    oauth_scopes = [
      "https://www.googleapis.com/auth/cloud-platform"
    ]
  }
}

resource "google_container_node_pool" "gke" {
  name    = "spot"
  cluster = google_container_cluster.gke.id

  management {
    auto_repair  = true
    auto_upgrade = true
  }

  autoscaling {
    min_node_count = 0
    max_node_count = 10
  }

  node_config {
    preemptible  = true
    machine_type = "e2-small"

    labels = {
      team = "spot"
    }

    taint {
      key    = "instance_type"
      value  = "spot"
      effect = "NO_SCHEDULE"
    }

    service_account = google_service_account.gke.email
    oauth_scopes = [
      "https://www.googleapis.com/auth/cloud-platform"
    ]
  }
}

8. Terraform Apply

透過 terraform apply 來建立 GKE 。完成啦～

Create Private GKE Via Terraform

總結

透過 Terraform 建立 GKE 就是快速又方便，也順便複習了 Google Cloud 的相關服務，而且我們還可以透過 Terraform 不斷複製成功又快速的去建立 GKE。

最後有興趣的朋友可以到這邊複製貼上： Github Source Code

歡迎到我的 Facebook Alan 的筆記本留言，順手給我個讚吧！你的讚將成為我持續更新的動力，感謝你的閱讀，讓我們一起學習成為更好的自己。