Deploy production-grade Gitlab on GKE with Terraform and Helm

Introduction

This written workshop describes how to deploy a scalable, reliable, and observable GitLab instance on Google Kubernetes Engine (GKE).
GitLab is an open source application that helps its users plan, store, test, and deploy their source code. Its core functionality is that of a source code management platform, but it also includes features for project management, continuous integration, and continuous deployment.

In this tutorial, you deploy the following architecture using Terraform. You use GKE for GitLab’s application components and Google Cloud managed services where possible to decrease the operational burden of running GitLab. You use Helm as the mechanism to install and configure the GitLab application components in GKE.

Architecture

The following table maps the Google Cloud products used in this tutorial to their use case in GitLab.

Assumptions and Prerequisites

• You have basic knowledge of GCP
• Have basic knowledge of Kubernetes
• You have Terraform v0.12x installed in your local machine
•  Understanding of Cloud Identity and Access Management (IAM), Google Kubernetes Engine, and Cloud SQL.
• You must have a Service Account with the following project roles:owner

Objectives

This guide walks you through how to the following tasks:

1. ✅Use Terraform to create a GKE Kubernetes cluster
2. ✅Create backing datastores to host GitLab data
3. ✅Install GitLab in a GKE cluster with Helm
4. ✅Monitor Gitlab with Prometheus and StackDriver
5. ✅Learn and use HCL (HashiCorp Language) and Terraform best practices

Software Dependencies

• Terraform 0.12.x
• terraform-provider-google plugin v1.8.0

Before you begin

1. Sign in to your Google Account.If you don’t already have one, sign up for a new account.
2. In the Cloud Console, on the project selector page, select or create a Cloud project Go to the project selector page
3. Make sure that billing is enabled for your Google Cloud project. Learn how to confirm billing is enabled for your project.

⭐️ Note: If you don’t plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project

When you finish this lab, you can avoid continued billing by deleting the resources you created. For more information, see Cleaning up section.

Creating prerequisite resources

Before deploy Gitlab instance on GKE, you need to enable the following APIs

⭐️ Note: You run all the terminal commands in this tutorial from Cloud Shell.

1. Open Cloud Shell: Open Cloud Shell
2. In Cloud Shell, enable the Google Kubernetes Engine, Service Networking, Resource Manager, and Redis APIs:

gcloud services enable container.googleapis.com \
servicenetworking.googleapis.com \
cloudresourcemanager.googleapis.com \
redis.googleapis.com

Deploy Gitlab on GKE

1- Source Code

The first step is to obtain the source code from Github.
This will clone the sample repository and make it the current directory:

git clone https://github.com/AymenSegni/gitlab-gke-tf-helm.git
cd gitlab-gke-tf-helm

2- Project structure

The directory that holds the Terraform configuration files for this lab has a special tree structure. Obviously, there are 2 main subfolders: deployment and modules. In order to see the source code structure you can run tree

1. /deployment: Terraform deployment execution
2. main.tf: main file for this module, contains all the resources to create
3. variables.tf: all the variables for the module
4. output.tf: the outputs of the module
5. providers.tf: project providers (gcp, kubernetes, helm)
6. versions.tf: required versions
7. /values.yaml.tpl: Helm values. Learn more about values in helm

To learn more about Terraform, Terraform modules and Terraform best practices you can visit my previous lab

3- Define the bricks of Gitlab on GKE architecture with terraform

Basically, we define all the resources to create in src/modules/giltab_gke/main.tf. By running the Terraform deployment in : src/deployment/ we will create the following Cloud services:

Configure Cloud IAM

Create an IAM service account for GitLab. Then add the storage.admin role to it so that it can access Cloud Storage.

Networking

1. Create an external IP address that can be used to reach your GitLab instance
2. Create a private IP address range that can be assigned to your Cloud SQL for PostgreSQL database instance so that it can’t be reached from outside of your VPC network

Cloud Storage

Create the buckets used by GitLab

Create the PostgreSQL instance and database

1. Create the Cloud SQL database that GitLab will use to store most of its metadata. This creates a privately accessible PostgreSQL database in the default network, with 4 CPU cores and 15 GB of RAM. The disk size of the database is set to increase automatically as needed.
2. Create a database user and password for GitLab to authenticate to the database
3. Create the database that GitLab will use in your PostgreSQL instance

Create the Redis instance

Create the Redis instance. For more information on sizing the Redis instance, see the GitLab requirements documentation

Create and configure the GKE cluster

1. Create the GKE cluster that you will use to host the GitLab application’s components. The Alias IP feature must be enabled in the cluster so that it can talk to the Redis instance.
2. Create a Storage Class so that GitLab can store its Git repositories on SSD Persistent Disks.
3. Create a secret in your cluster that holds the PostgreSQL password.
4. Create a secret to hold your Cloud Storage credentials.
5. Create a secret to hold the additional parameters that Rails needs in order to configure the large file storage, artifacts, upload, and packages features.

Configuring and installing GitLab with Helm provider

In this section, you use Helm  Terraform provider to deploy GitLab from the GitLab Chart repository. Helm is a package manager you can use to configure and deploy Kubernetes apps.

1-The GitLab chart requires custom configuration so that it can leverage the external datastores that you previously provisioned.

2-Install the chart by using Helm

Configure monitoring

You can monitor the GitLab installation by using the Stackdriver Prometheus integration. This integration deploys a Prometheus server to scrape metrics from GitLab. A sidecar container then sends metrics from Prometheus to Stackdriver for longer term storage, dashboarding, and alerting.

1- Clone the Prometheus integration source code:

git clone https://github.com/stackdriver/stackdriver-prometheus-sidecar
cd stackdriver-prometheus-sidecar/kube/full


2- Configure the installation by setting the following environment variables:

export KUBE_NAMESPACE=default
export KUBE_CLUSTER=gitlabexport GCP_REGION=us-central1
export GCP_PROJECT=$(gcloud config get-value project)
export SIDECAR_IMAGE_TAG=release-0.4.2

3- Install the Prometheus integration:

./deploy.sh

4- Deploy Gitlab on GKE with Terraform

After getting the source code, perform the following commands

cd gitlab-gke-tf-helm #the project folder
cd src/deployment
terraform init # to get the plugins
terraform plan # to see the infrastructure plan
terraform apply # to apply the infrastructure build

Use the production ready Gitlab

During the execution of the terraform apply command, the module outputs will display 2 important values:

1. gitlab_url: URL where you can access your GitLab instance
2. root_password_instructionsInstructions: for getting the root user’s password for initial setup

Now that GitLab is up and running, connect to it and set up the initial user password.
Visit the GitLab URL in your browser from step 1 and log in as the root user with the password from step 2. The following screenshot shows the login page.

You are now ready to use GitLab. Visit the GitLab documentation to learn more about how to interact with it. For more information on customizing your installation, visit the GitLab Chart documentation.

Clean up

1. In the Cloud Console, go to the Manage resources page.Go to the Manage resources page
2. In the project list, select the project that you want to delete and then click Delete delete.
3. In the dialog, type the project ID and then click Shut down to delete the project.

That’s all folks!

That’s all for this lab, thanks for reading 🙏

Be the first to be notified when a new article, running it on Cloud or Kubernetes experiment is published.
Don’t miss the next article!