Category: GoLang

Go is a statically typed, compiled programming language designed at Google by Robert Griesemer, Rob Pike, and Ken Thompson. Go is syntactically similar to C, but with memory safety, garbage collection, structural typing, and CSP-style concurrency.

Deploying a containerized Go app on Kubernetes
06
Feb
2021

Deploying a containerized Go app on Kubernetes

Introduction

Kubernetes is an open-source container orchestrator built by Google that helps you run, manage, and scale containerized applications on the cloud.

It has everything to automate the deployment, scaling, and management of modern applications. Some notable features of Kubernetes are:

  • Horizontal auto-scaling
  • Service discovery and Load balancing
  • Rolling updates with zero downtime
  • Self-healing mechanisms (using health-checks)
  • Secret and configuration management

All the major cloud providers (Google Cloud, AWS, Azure, DigitalOcean, etc) have managed Kubernetes platforms. This means that you can easily switch between different cloud platforms without making any changes in your architecture.

In this article, You’ll learn how to deploy, manage, and scale a simple Go web app on Kubernetes.

We’ll deploy the app on a local kubernetes cluster created using minikube. Minikube is a tool that lets you set up a single-node Kubernetes cluster inside a VM on your local machine. It’s great for learning and playing with Kubernetes.

Building a sample web application in Go

Let’s build a simple Go web app to deploy on Kubernetes. Fire up your terminal and create a new folder for the project:

$ mkdir go-kubernetes

Next, Initialize Go modules by running the following command

$ cd go-kubernetes
$ go mod init github.com/changeit/go-kubernetes # Change the module path as per your Github username

Now, Create a file named main.go and copy the following code –

package main

import (
	"context"
	"fmt"
	"log"
	"net/http"
	"os"
	"os/signal"
	"syscall"
	"time"

	"github.com/gorilla/mux"
)

func handler(w http.ResponseWriter, r *http.Request) {
	query := r.URL.Query()
	name := query.Get("name")
	if name == "" {
		name = "Guest"
	}
	log.Printf("Received request for %s\n", name)
	w.Write([]byte(fmt.Sprintf("Hello, %s\n", name)))
}

func healthHandler(w http.ResponseWriter, r *http.Request) {
	w.WriteHeader(http.StatusOK)
}

func readinessHandler(w http.ResponseWriter, r *http.Request) {
	w.WriteHeader(http.StatusOK)
}

func main() {
	// Create Server and Route Handlers
	r := mux.NewRouter()

	r.HandleFunc("/", handler)
	r.HandleFunc("/health", healthHandler)
	r.HandleFunc("/readiness", readinessHandler)

	srv := &http.Server{
		Handler:      r,
		Addr:         ":8080",
		ReadTimeout:  10 * time.Second,
		WriteTimeout: 10 * time.Second,
	}

	// Start Server
	go func() {
		log.Println("Starting Server")
		if err := srv.ListenAndServe(); err != nil {
			log.Fatal(err)
		}
	}()

	// Graceful Shutdown
	waitForShutdown(srv)
}

func waitForShutdown(srv *http.Server) {
	interruptChan := make(chan os.Signal, 1)
	signal.Notify(interruptChan, os.Interrupt, syscall.SIGINT, syscall.SIGTERM)

	// Block until we receive our signal.
	<-interruptChan

	// Create a deadline to wait for.
	ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
	defer cancel()
	srv.Shutdown(ctx)

	log.Println("Shutting down")
	os.Exit(0)
}

The app uses gorilla mux library for routing. It also has /health and /readiness endpoints apart from the / endpoint. You’ll find out what is the use of these endpoints in the later section.

Let’s now build and run the app locally:

$ go build
$ ./go-kubernetes
2019/07/27 11:51:58 Starting Server

$ curl localhost:8080?name=Rajeev
Hello, Rajeev

Dockerizing the Go application

To deploy our app on Kubernetes, we need to first containerize it. Create a file named Dockerfile inside the project’s folder and add the following configurations in the Dockerfile.

# Dockerfile References: https://docs.docker.com/engine/reference/builder/

# Start from the latest golang base image
FROM golang:latest as builder

# Add Maintainer Info
LABEL maintainer="Rajeev Singh <rajeevhub@gmail.com>"

# Set the Current Working Directory inside the container
WORKDIR /app

# Copy go mod and sum files
COPY go.mod go.sum ./

# Download all dependancies. Dependencies will be cached if the go.mod and go.sum files are not changed
RUN go mod download

# Copy the source from the current directory to the Working Directory inside the container
COPY . .

# Build the Go app
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .


######## Start a new stage from scratch #######
FROM alpine:latest  

RUN apk --no-cache add ca-certificates

WORKDIR /root/

# Copy the Pre-built binary file from the previous stage
COPY --from=builder /app/main .

# Expose port 8080 to the outside world
EXPOSE 8080

# Command to run the executable
CMD ["./main"] 

I’m not gonna go into the details of the Dockerfile here. Please check out the article Building Docker Containers for Go Applications to learn more.

Building and pushing the docker image to docker hub

Let’s build and push the docker image of our Go app on docker hub so that we can later use this image while deploying the app on Kubernetes –

# Build the docker image
$ docker build -t go-kubernetes .

# Tag the image
$ docker tag go-kubernetes modulename/go-hello-world:1.0.0

# Login to docker with your docker Id
$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don\'t have a Docker ID, head over to https://hub.docker.com to create one.
Username (username): password
Password:
Login Succeeded

# Push the image to docker hub
$ docker push modulename/go-hello-world:1.0.0

Creating a Kubernetes deployment

All right! Let’s now create a Kubernetes deployment for our app. Deployments are a declarative way to instruct Kubernetes how to create and update instances of your application. A deployment consists of a set of identical, indistinguishable Pods.

Pod represents a unit of deployment, i.e. a single instance of your application in Kubernetes, which might consist of either a single container or a small number of containers that are tightly coupled and that share resources.

When it comes to managing Pods, deployments abstract away the low-level details like what node is the Pod running on. Pods are tied to the lifetime of the node. So when the node dies, so does the Pod. It’s the job of the deployment to ensure that the current number of Pods equals the desired number of Pods.

We specify the details of the number of Pods, what containers to run inside the Pod, how to check if the Pod is healthy or not, in a so-called manifest file. It’s a simple yaml file with a bunch of configurations containing the desired state of our application.

k8s-deployment.yml

---
apiVersion: apps/v1
kind: Deployment                 # Type of Kubernetes resource
metadata:
  name: go-hello-world           # Name of the Kubernetes resource
spec:
  replicas: 3                    # Number of pods to run at any given time
  selector:
    matchLabels:
      app: go-hello-world        # This deployment applies to any Pods matching the specified label
  template:                      # This deployment will create a set of pods using the configurations in this template
    metadata:
      labels:                    # The labels that will be applied to all of the pods in this deployment
        app: go-hello-world 
    spec:                        # Spec for the container which will run in the Pod
      containers:
      - name: go-hello-world
        image: modulename/go-hello-world:1.0.0 
        imagePullPolicy: IfNotPresent
        ports:
          - containerPort: 8080  # Should match the port number that the Go application listens on
        livenessProbe:           # To check the health of the Pod
          httpGet:
            path: /health
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 5
          periodSeconds: 15
          timeoutSeconds: 5
        readinessProbe:          # To check if the Pod is ready to serve traffic or not
          httpGet:
            path: /readiness
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 5
          timeoutSeconds: 1    

I’ve added comments alongside each configuration in the above deployment manifest file. But I want to talk more about some of them.

Notice the configuration replicas: 3 in the above file. It instructs Kubernetes to run 3 instances of our application at any given time. If an instance dies, Kubernetes automatically spins up another instance.

Let’s also talk about the livenessProbe and readinessProbe. Sometimes a container on a pod can be running but the application inside of the container might be malfunctioning. For instance, if your code was deadlocked.

Kubernetes has built-in support to make sure that your application is running correctly with user implemented application health and readiness checks.

Readiness probes indicate when an application is ready to serve traffic. If a readiness check fails then the container will be marked as not ready and will be removed from any load balancers.

Liveness probes indicate a container is alive. If a liveness probe fails multiple times, then the container will be restarted.

Starting a local Kubernetes cluster using Minikube and deploying the app

You’ll need to install and set up kubectl (Kubernetes command-line tool) and Minikube to proceed further. Please follow the instructions on the official Kubernetes website to install kubectl and minikube.

Once the installation is complete, type the following command to start a Kubernetes cluster:

$ minikube start

Let’s now deploy our app to the minikube cluster by applying the deployment manifest using kubectl.

$ kubectl apply -f k8s-deployment.yml
deployment.apps/go-hello-world created

That’s it! The deployment is created. You can get the deployments like this:

$ kubectl get deployments
NAME             READY   UP-TO-DATE   AVAILABLE   AGE
go-hello-world   3/3     3            3           25s

You can type the following command to get the pods in the cluster:

$ kubectl get pods
NAME                              READY   STATUS    RESTARTS   AGE
go-hello-world-69b45499fb-7fh87   1/1     Running   0          37s
go-hello-world-69b45499fb-rt2xj   1/1     Running   0          37s
go-hello-world-69b45499fb-xjmlq   1/1     Running   0          37s

Pods are allocated a private IP address by default and cannot be reached outside of the cluster. You can use the kubectl port-forward command to map a local port to a port inside the pod like this:

$ kubectl port-forward go-hello-world-69b45499fb-7fh87 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

You can now interact with the Pod on the forwarded port:

$ curl localhost:8080
Hello, Guest

$ curl localhost:8080?name=Rajeev
Hello, Rajeev

You can also stream the Pod logs by typing the following command:

$ kubectl logs -f go-hello-world-69b45499fb-7fh87
2019/07/27 06:12:09 Starting Server
2019/07/27 06:15:42 Received request for Guest
2019/07/27 06:16:02 Received request for Rajeev

Creating a Kubernetes Service

The port-forward command is good for testing the pods directly. But in production, you would want to expose the pod using services.

Pods can be restarted for all kinds of reasons like failed liveliness checks, readiness checks or they can be killed if the node they are running on dies.

Instead of relying on the Pods IP addresses which change, Kubernetes provides services as stable endpoint for pods. The pods that the service exposes are based on a set of labels. If Pods have the correct labels, they are automatically picked up and exposed by our services.

The level of access the service provides to the set of pods depends on the service type which can be:

  • ClusterIP: Internal only.
  • NodePort: Gives each node an external IP that’s accessible from outside the cluster and also opens a Port. A kube-proxy component that runs on each node of the Kubernetes cluster listens for incoming traffic on the port and forwards them to the selected pods in a round-robin fashion.
  • LoadBalancer: Adds a load balancer from the cloud provider which forwards traffic from the service to the nodes within it.

Let’s expose our Pods by creating a service. Add the following configurations in the k8s-deployment.yml file:

---
apiVersion: v1
kind: Service                    # Type of kubernetes resource
metadata:
  name: go-hello-world-service   # Name of the resource
spec:
  type: NodePort                 # A port is opened on each node in your cluster via Kube proxy.
  ports:                         # Take incoming HTTP requests on port 9090 and forward them to the targetPort of 8080
  - name: http
    port: 9090
    targetPort: 8080
  selector:
    app: go-hello-world         # Map any pod with label `app=go-hello-world` to this service

Let’s now apply the above configurations by typing the following command:

$ kubectl apply -f k8s-deployment.yml
deployment.apps/go-hello-world unchanged
service/go-hello-world-service created

A service is created for exposing the Pods. You can get the list of services in the kubernetes cluster like this:

$ kubectl get services
NAME                     TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
go-hello-world-service   NodePort    10.111.51.170   <none>        9090:32550/TCP   35s
kubernetes               ClusterIP   10.96.0.1       <none>        443/TCP          13h

Type the following command to get the URL for the service in the minikube cluster:

$ minikube service go-hello-world-service --url
http://192.168.99.100:32550

That’s all! You can now interact with the service on the above URL:

$ curl http://192.168.99.100:32550
Hello, Guest

$ curl http://192.168.99.100:32550?name=Rajeev
Hello, Rajeev

Scaling a Kubernetes deployment

You can scale the number of Pods by increasing the number of replicas in the kubernetes deployment manifest and applying the changes using kubectl.

You can also use kubectl scale command to increase the number of pods:

$ kubectl scale --replicas=4 deployment/go-hello-world
deployment.extensions/go-hello-world scaled

$ kubectl get pods
NAME                              READY   STATUS    RESTARTS   AGE
go-hello-world-69b45499fb-7fh87   1/1     Running   0          112m
go-hello-world-69b45499fb-hzb6v   1/1     Running   0          10s
go-hello-world-69b45499fb-rt2xj   1/1     Running   0          112m
go-hello-world-69b45499fb-xjmlq   1/1     Running   0          112m

Deleting Kubernetes resources

Deleting a Pod

$ kubectl delete pod go-hello-world-69b45499fb-7fh87
pod "go-hello-world-69b45499fb-7fh87" deleted

Deleting a Service

$ kubectl delete service go-hello-world-service
service "go-hello-world-service" deleted

Deleting a Deployment

$ kubectl delete deployment go-hello-world
deployment.extensions "go-hello-world" deleted

Stopping and Deleting the Minikube cluster

Stopping the minikube kubernetes cluster

$ minikube stop

Deleting the minikube kubernetes cluster

$ minikube delete

Conclusion

That’s all in this article. You can find the complete code in the following Github repository.

As always, thanks for reading. I hope you found the article useful. Let me know your thoughts in the comment section below.

A Guide on How to Write a Clean Code
18
Mar
2021

A Guide on How to Write a Clean Code

In this article, I’m going to talk about writing clean code in general and then end up with some examples. As an Android developer having a clean code has always been challenging for me which needs lots of effort and that’s just coding and coding.

Rules We Should Follow

  1. Ignore duplication (imply DRY principle-Don’t Repeat Yourself)
  2. Minimize entities, classes, and functions (avoid repetition)
  3. It should be readable and simple
  4. Be testable as it makes your code flexible and maintainable
  5. Follow SOLIDprinciples

S = Single-responsibility principle: A class should only have one purpose.

O = Open-closed principleA class should be open for extension, but closed for modification.

L = Liskov substitution principleAbstraction should be able to provide all needs of child class.

I = Interface segregation principleSmall interface is better than big one.

D = Dependency Inversion Principle: A class should depend on abstraction, not implementation.

6. Be careful with dependencies

As many as possible try to have one-directional dependency. When dependency goes in multiple directions, things get much more complicated which makes hard to update and change.

7. Don’t hardcode

Define constant or use variables instead of hardcoding the values which will not only help readability but also make it easy to change if it is being used at multiple places.

Now let’s start writing Clean Code 🚀 …

Valid Names

Naming is one of the hardest parts of programming and may take time to choose, but it is a strong way to convey your code’s intent to other developers who read them in the future which must show the purpose of that class, function, or variable and even tell what is going to do. Don’t append prefixes or type information. I always have a checklist in my mind as listed below which assures me that a name has been chosen well:

  • Are the variables named according to the convention (camelCase, PascalCase, etc)?.
  • Does the name have an appropriate length to ensure that no one will be confused by it?.
  • Are the name of variables clear about what they hold?.
  • Are the names meaningful, searchable, and easy to pronounce?.

Class names should not only be names (not verbs) but also should have the PascalCase convention. On the other hand, method names should be verbs or phrase verbs and follow the camelCase convention. The same rules apply to variable names.

Functions and Methods

Following S from SOLID principles, let functions and methods perform only one task which should be small. If the function arguments are too many, you should review your code and maybe pack them into an object or divide that task between some other functions. Prefer possible exceptions to return error codes and extract error handling try catch into their own function.

Comments

We should use comments only when it is necessary, not to explain bad code. Writing lengthy comments will not help us in changing our code into a clean one. If the code is bad, we should solve it by improving the code, not by adding instructions on how to use it, but it doesn’t mean that you shouldn’t use comments at all, sometimes it is important such as dealing with third party APIs where you need to explain some behavior.

Code Appearance

It might look unimportant at first glance, but it is of high importance to write your code with a well-organized format which makes it readable such as:

  • Do not write everything in a single line. Give proper whitespace, indentation, or line breaks in your code.

Indentation styles assist in identifying control flow and blocks of code. In some programming languages, indentation is used to delimit logical blocks of code; correct indentation in these cases is more than a matter of style. In other languages, indentation and white space do not affect function, although logical and consistent indentation makes code more readable. (Inspired by WIKIPEDIA)

  • Instance variables should be declared at the top of the class.
  • If functions are calling each other (dependent functions) they should be close while putting the caller at first.
  • It is often helpful to align similar elements vertically, to make typo-generated bugs more obvious.

Tests

The importance of Test code is equal to producing the code and of course, while writing tests you might find some bugs. However, it can sometimes be quite difficult to write a good test for a particular piece of code which is a result of a badly designed and untestable code. Testing is almost like coding for the second time so follow all the previous rules for writing a good test as well which should be as per below :

  • One assert and single concept per test
  • Easy to write and readable (simplicity)
  • Fast
  • Independent

To conclude, I hope this article could be helpful to write a better code although it is important to keep practicing and learning in this field 😃 …

Understanding Regular Expressions
18
Mar
2021

Understanding Regular Expressions

it’s time to get over your fears of regular expressions(regex)! If you’re like me, you’ve heard of regex but have always been confused by the cryptic syntax. Fear not, because in the next 5 minutes, you’ll have a basic understanding of what’s going on and how to use RegEx to make your life easier!

So what are Regular Expressions ?

Basically, regular expressions are patterns that you can use to find matching patterns in strings. This could be useful for password validation, or checking if the formatting of input fields is correct, or perhaps you want to parse a phone number, etc…

How do you use them?

There are a couple ways of creating a regex, you can either use the literal version( which I prefer) or you can use the constructor option. The literal version looks like so:

const regEx = /hello/;

When making a regular expression literal, you place the pattern between two forward slashes. Above, we would be searching for the word ‘hello’.

Using the constructor would look something like this:

 const regEx = new RegExp('hello');

I’m not the biggest fan of this, so moving forward I will only be using the literal version.

Testing Methods

How do you test your regex anyways? JavaScript provides us a couple of methods that are compatible with regular expressions:

  • test()
  • exec()
  • match()
  • matchAll()
  • replace()
  • search()
  • split()

For my examples I will primarily be using test() and match(). Test is a RegExp method used to search a string and return either true or false if your pattern is found, and match is a string method that can use regex and returns the instances found in an array.

How to Match Strings

As you saw in my example above, I created the regex /hello/. This pattern would be useful for finding the first case sensitive instance of ‘hello’. What if you want to find every instance of ‘hello’, case insensitive? This is where ‘flags’ come in.

Flags act as modifiers to your regular expression. They go after the closing slash and there are five native flags in JavaScript!

  • i : This makes your search case-insensitive!
  • g : This flag tells your search to look for all matches, not just the first one.
  • m : Multiline mode
  • s : This enables “dotall” mode. It allows ‘.’ to match newlines
  • u : Enables full unicode support
  • y : sticky; it matches only from the index indicated by the lastIndex property of the regex in the target string.

So if we wanted to find every instance of ‘hello’ case insensitive, in the string “Hello heLLO hellO HELLO!”, we would do something like this:

let regex = /hello/gi;
let string = “Hello heLLO hellO HELLO!”;
string.match(regex);//returns [ 'Hello', 'heLLO', 'hellO', 'HELLO' ]

See that’s not so bad! Let’s look at another tool that’s very useful: character classes.

Character Classes

Character classes let you match a group of characters by placing them inside square brackets! This lets you find multiple matches with different characters. For example: if you wanted to find the words ‘big’, ‘bag’, ‘bog’, ‘bug’ in the string “The big bug crawled out of my bag and went into the bog.” you could use a simple regular expression to do so!

let string = “The big bug crawled out of my bag and went into the bog.”;let regex = /b[aiou]g/gi;
string.match(regex);//returns [ 'big', 'bug', 'bag', 'bog' ]

That’s pretty cool. You can also search for a range of characters inside of a character set using a hyphen ‘-’! For example, if I wanted to find every number in a string for some reason, I could do something like this:

let string = "I want 6 chocolates, 5 pop tarts, and 3 pumpkin pies please."let regex = /[0-9]/g;
string.match(regex);//returns [ '6', '5', '3' ]

You can do the above and so, so much more using Regular Expressions. You’ll find it’s basically like another language! This is just the tip of the iceberg, and I hope at least I’ve made regex a little less scary. There is so much more to regular expressions than I can cover here, but there are plenty of great resources to continue learning.

Happy Coding!

Deploying a stateless Go app with Redis on Kubernetes
03
Mar
2021

Deploying a stateless Go app with Redis on Kubernetes

In this article, we’ll go a step further and deploy a stateless Go web app with Redis on Kubernetes. You’ll get to understand how the deployment of multiple distinct Pods work and how two Pods can communicate with each other in the cluster.

Building a sample Go app that uses Redis

We’ll create a simple web application in Go that contains an API to display the “Quote of the day”.

The app fetches the quote of the day from a public API hosted at http://quotes.rest/, then it caches the result in Redis until the end of the day. For subsequent API calls, the app will return the result from Redis cache instead of fetching it from the public API.

Open your terminal and type the following commands to create the project and initialize Go modules

$ mkdir go-redis-kubernetes
$ cd go-redis-kubernetes
$ go mod init github.com/username/go-redis-kubernetes # Change `username` to your Github username

Next, create a file called main.go with the following code:

package main

import (
	"context"
	"encoding/json"
	"errors"
	"log"
	"net/http"
	"os"
	"os/signal"
	"syscall"
	"time"

	"github.com/go-redis/redis"
	"github.com/gorilla/mux"
)

func indexHandler(w http.ResponseWriter, r *http.Request) {
	w.Write([]byte("Welcome! Please hit the `/qod` API to get the quote of the day."))
}

func quoteOfTheDayHandler(client *redis.Client) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		currentTime := time.Now()
		date := currentTime.Format("2006-01-02")

		val, err := client.Get(date).Result()
		if err == redis.Nil {
			log.Println("Cache miss for date ", date)
			quoteResp, err := getQuoteFromAPI()
			if err != nil {
				w.Write([]byte("Sorry! We could not get the Quote of the Day. Please try again."))
				return
			}
			quote := quoteResp.Contents.Quotes[0].Quote
			client.Set(date, quote, 24*time.Hour)
			w.Write([]byte(quote))
		} else {
			log.Println("Cache Hit for date ", date)
			w.Write([]byte(val))
		}
	}
}

func main() {
	// Create Redis Client
	var (
		host     = getEnv("REDIS_HOST", "localhost")
		port     = string(getEnv("REDIS_PORT", "6379"))
		password = getEnv("REDIS_PASSWORD", "")
	)

	client := redis.NewClient(&redis.Options{
		Addr:     host + ":" + port,
		Password: password,
		DB:       0,
	})

	_, err := client.Ping().Result()
	if err != nil {
		log.Fatal(err)
	}

	// Create Server and Route Handlers
	r := mux.NewRouter()

	r.HandleFunc("/", indexHandler)
	r.HandleFunc("/qod", quoteOfTheDayHandler(client))

	srv := &http.Server{
		Handler:      r,
		Addr:         ":8080",
		ReadTimeout:  10 * time.Second,
		WriteTimeout: 10 * time.Second,
	}

	// Start Server
	go func() {
		log.Println("Starting Server")
		if err := srv.ListenAndServe(); err != nil {
			log.Fatal(err)
		}
	}()

	// Graceful Shutdown
	waitForShutdown(srv)
}

func waitForShutdown(srv *http.Server) {
	interruptChan := make(chan os.Signal, 1)
	signal.Notify(interruptChan, os.Interrupt, syscall.SIGINT, syscall.SIGTERM)

	// Block until we receive our signal.
	<-interruptChan

	// Create a deadline to wait for.
	ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
	defer cancel()
	srv.Shutdown(ctx)

	log.Println("Shutting down")
	os.Exit(0)
}

func getQuoteFromAPI() (*QuoteResponse, error) {
	API_URL := "http://quotes.rest/qod.json"
	resp, err := http.Get(API_URL)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()
	log.Println("Quote API Returned: ", resp.StatusCode, http.StatusText(resp.StatusCode))

	if resp.StatusCode >= 200 && resp.StatusCode <= 299 {
		quoteResp := &QuoteResponse{}
		json.NewDecoder(resp.Body).Decode(quoteResp)
		return quoteResp, nil
	} else {
		return nil, errors.New("Could not get quote from API")
	}

}

func getEnv(key, defaultValue string) string {
	value := os.Getenv(key)
	if value == "" {
		return defaultValue
	}
	return value
}

Also, create the following structs in a file named quote.go to parse the JSON response returned from http://quotes.rest/ API.

package main

type QuoteData struct {
	Id         string   `json:"id"`
	Quote      string   `json:"quote"`
	Length     string   `json:"length"`
	Author     string   `json:"author"`
	Tags       []string `json:"tags"`
	Category   string   `json:"category"`
	Date       string   `json:"date"`
	Permalink  string   `json:"parmalink"`
	Title      string   `json:"title"`
	Background string   `json:"Background"`
}

type QuoteResponse struct {
	Success  APISuccess   `json:"success"`
	Contents QuoteContent `json:"contents"`
}

type QuoteContent struct {
	Quotes    []QuoteData `json:"quotes"`
	Copyright string      `json:"copyright"`
}

type APISuccess struct {
	Total string `json:"total"`
}

Let’s now build and run the app locally:

$ go build
$ ./go-redis-kubernetes
2019/07/28 13:32:05 Starting Server
$ curl localhost:8080
Welcome! Please hit the `/qod` API to get the quote of the day.

$ curl localhost:8080/qod
I’ve missed more than 9000 shots in my career. I’ve lost almost 300 games. 26 times, I’ve been trusted to take the game winning shot and missed. I’ve failed over and over and over again in my life. And that is why I succeed.

Containerizing the Go app

Let’s now containerize our Go app by creating a Dockerfile with the following configurations:

# Dockerfile References: https://docs.docker.com/engine/reference/builder/

# Start from the latest golang base image
FROM golang:latest as builder

# Add Maintainer Info
LABEL maintainer="Rajeev Singh <rajeevhub@gmail.com>"

# Set the Current Working Directory inside the container
WORKDIR /app

# Copy go mod and sum files
COPY go.mod go.sum ./

# Download all dependencies. Dependencies will be cached if the go.mod and go.sum files are not changed
RUN go mod download

# Copy the source from the current directory to the Working Directory inside the container
COPY . .

# Build the Go app
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .


######## Start a new stage from scratch #######
FROM alpine:latest  

RUN apk --no-cache add ca-certificates

WORKDIR /root/

# Copy the Pre-built binary file from the previous stage
COPY --from=builder /app/main .

# Expose port 8080 to the outside world
EXPOSE 8080

# Command to run the executable
CMD ["./main"] 

Check out the article Building Docker Containers for Go Applications to learn more about how to containerize a Go app.

I’ve already built and published the docker image for our app on docker hub. You can use the following commands to do so –

# Build the image
$ docker build -t go-redis-kubernetes .

# Tag the image
$ docker tag go-redis-kubernetes username/go-redis-app:1.0.0

# Login to docker with your docker Id
$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don\'t have a Docker ID, head over to https://hub.docker.com to create one.
Username (username): username
Password:
Login Succeeded

# Push the image to docker hub
$ docker push project/go-redis-app:1.0.0

Creating the Kubernetes deployment and service manifest for Redis

Let’s now create the configuration for deploying our Redis app on Kubernetes. We’ll need to create a deployment for managing the Redis instance and a Service to proxy traffic from our Go app to the Redis Pod.

Create a folder called deployments inside the project’s root directoy to store all the deployment manifests. And then, create a file called redis-master.yml with the following configurations:

---
apiVersion: apps/v1  # API version
kind: Deployment
metadata:
  name: redis-master # Unique name for the deployment
  labels:
    app: redis       # Labels to be applied to this deployment
spec:
  selector:
    matchLabels:     # This deployment applies to the Pods matching these labels
      app: redis
      role: master
      tier: backend
  replicas: 1        # Run a single pod in the deployment
  template:          # Template for the pods that will be created by this deployment
    metadata:
      labels:        # Labels to be applied to the Pods in this deployment
        app: redis
        role: master
        tier: backend
    spec:            # Spec for the container which will be run inside the Pod.
      containers:
      - name: master
        image: redis
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
        ports:
        - containerPort: 6379
---        
apiVersion: v1
kind: Service        # Type of Kubernetes resource
metadata:
  name: redis-master # Name of the Kubernetes resource
  labels:            # Labels that will be applied to this resource
    app: redis
    role: master
    tier: backend
spec:
  ports:
  - port: 6379       # Map incoming connections on port 6379 to the target port 6379 of the Pod
    targetPort: 6379
  selector:          # Map any Pod with the specified labels to this service
    app: redis
    role: master
    tier: backend

The redis-master Service is only accessible within the container cluster because the default type for a Service is ClusterIP. ClusterIP provides a single IP address for the set of Pods the Service is pointing to. This IP address is accessible only within the cluster.

Kubernetes deployment manifest for the Go app

Let’s now create a deployment and a service for our Go app. We’ll run 3 Pods for the Go app and the Pods will be exposed via a Service to the outside world:

---
apiVersion: apps/v1
kind: Deployment                 # Type of Kubernetes resource
metadata:
  name: go-redis-app             # Unique name of the Kubernetes resource
spec:
  replicas: 3                    # Number of pods to run at any given time
  selector:
    matchLabels:
      app: go-redis-app          # This deployment applies to any Pods matching the specified label
  template:                      # This deployment will create a set of pods using the configurations in this template
    metadata:
      labels:                    # The labels that will be applied to all of the pods in this deployment
        app: go-redis-app 
    spec:
      containers:
      - name: go-redis-app
        image: project/go-redis-app:1.0.0 
        imagePullPolicy: IfNotPresent
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
        ports:
          - containerPort: 8080  # Should match the port number that the Go application listens on    
        env:                     # Environment variables passed to the container
          - name: REDIS_HOST
            value: redis-master
          - name: REDIS_PORT
            value: "6379"    
---
apiVersion: v1
kind: Service                    # Type of kubernetes resource
metadata:
  name: go-redis-app-service     # Unique name of the resource
spec:
  type: NodePort                 # Expose the Pods by opening a port on each Node and proxying it to the service.
  ports:                         # Take incoming HTTP requests on port 9090 and forward them to the targetPort of 8080
  - name: http
    port: 9090
    targetPort: 8080
  selector:
    app: go-redis-app            # Map any pod with label `app=go-redis-app` to this service

The Golang app can communicate with Redis using the hostname redis-master. This is automatically resolved by Kubernetes to point to the IP address of the service redis-master.

Deploying the Go app and Redis on Kubernetes

We’ll deploy the Go web app and Redis on a local kubernetes cluster created using Minikube.

Please install Minikube and Kubectl if you haven’t installed them already. Check out the Kubernetes official documentation for instructions.

Start a Kubernetes cluster using minikube

$ minikube start

Deploy Redis

$ kubectl apply -f deployments/redis-master.yml
deployment.apps/redis-master created
service/redis-master created
$ kubectl get pods
NAME                            READY   STATUS    RESTARTS   AGE
redis-master-7b44998456-pl8h9   1/1     Running   0          34s

Deploy the Go app

$ kubectl apply -f deployments/go-redis-app.yml
deployment.apps/go-redis-app created
service/go-redis-app-service created
$ kubectl get pods
NAME                            READY   STATUS    RESTARTS   AGE
go-redis-app-57b7d4d4cd-fkddw   1/1     Running   0          27s
go-redis-app-57b7d4d4cd-l9wg9   1/1     Running   0          27s
go-redis-app-57b7d4d4cd-m9t8b   1/1     Running   0          27s
redis-master-7b44998456-pl8h9   1/1     Running   0          82s

Accessing the application

The Go app is exposed as NodePort via the service. You can get the service URL using minikube like this –

$ minikube service go-redis-app-service --url
http://192.168.99.100:30435

You can use the above endpoint to access the application:

$ curl http://192.168.99.100:30435
Welcome! Please hit the `/qod` API to get the quote of the day.

$  curl http://192.168.99.100:30435/qod
I’ve missed more than 9000 shots in my career. I’ve lost almost 300 games. 26 times, I’ve been trusted to take the game winning shot and missed. I’ve failed over and over and over again in my life. And that is why I succeed.

Conclusion

In this article, you learned how to deploy a stateless Go web app with Redis on a local Kubernetes cluster created using Minikube.

Reading and Writing Environment Variables in Go
08
Mar
2021

Reading and Writing Environment Variables in Go

An environment variable is a way to supply dynamic configuration information to programs at runtime. Environment variables are often used to make the same program work in different environments like Local, QA, or Production.

Get, Set, Unset, and Expand environment variables in Go

The following program demonstrates how to work with environment variables in Go. It makes use of the following functions provided by the os package:

  • os.Setenv(key, value): Set an environment variable.
  • os.Getenv(key): Get the value of an environment variable. If the environment variable is not present, it returns empty. To distinguish between an empty value and an unset value, use LookupEnv
  • os.Unsetenv(key): Unset an environment variable.
  • os.LookupEnv(key): Get the value of an environment variable and a boolean indicating whether the environment variable is present or not. It returns a string and a boolean – The boolean will be false if the environment variable is not present.
  • os.ExpandEnv(str): Expand a string by replacing ${var} or $var in the string according to the values of the current environment variables.
package main

import (
	"fmt"
	"os"
)

func main() {
	// Set an Environment Variable
	os.Setenv("DB_HOST", "localhost")
	os.Setenv("DB_PORT", "5432")
	os.Setenv("DB_USERNAME", "root")
	os.Setenv("DB_PASSWORD", "admin")
	os.Setenv("DB_NAME", "test")

	// Get the value of an Environment Variable
	host := os.Getenv("DB_HOST")
	port := os.Getenv("DB_PORT")
	fmt.Printf("Host: %s, Port: %s\n", host, port)

	// Unset an Environment Variable
	os.Unsetenv("DB_PORT")
	fmt.Printf("After unset, Port: %s\n", os.Getenv("DB_PORT"))

	/*
		Get the value of an environment variable and a boolean indicating whether the
		environment variable is present or not.
	*/
	driver, ok := os.LookupEnv("DB_DRIVER")
	if !ok {
		fmt.Println("DB_DRIVER is not present")
	} else {
		fmt.Printf("Driver: %s\n", driver)
	}

	// Expand a string containing environment variables in the form of $var or ${var}
	dbURL := os.ExpandEnv("postgres://$DB_USERNAME:$DB_PASSWORD@DB_HOST:$DB_PORT/$DB_NAME")
	fmt.Println("DB URL: ", dbURL)

}
# Output
Host: localhost, Port: 5432
After unset, Port:
DB_DRIVER is not present
DB URL:  postgres://root:admin@DB_HOST:/test

List and Clear all environment variables in Go

  • os.Environ(): This function returns a []string containing all the environment variables in the form of key=value.
  • os.Clearenv(): This function deletes all the environment variables. It may come in handy while writing tests to start with a clean environment.

The following example demonstrates how to use these two functions:

package main

import (
	"fmt"
	"os"
	"strings"
)

func main() {

	// Environ returns a copy of strings representing the environment,
	// in the form "key=value".
	for _, env := range os.Environ() {
		// env is
		envPair := strings.SplitN(env, "=", 2)
		key := envPair[0]
		value := envPair[1]

		fmt.Printf("%s : %s\n", key, value)
	}

	// Delete all environment variables
	os.Clearenv()

	fmt.Println("Number of environment variables: ", len(os.Environ()))
}
# Output
TERM_SESSION_ID : w0t0p1:70C49068-9C87-4032-9C9B-49FB6B86687B
PATH : /Users/fusebes/.nvm/versions/node/v10.0.0/bin:/usr/local/sbin:/usr/local/sbin:/Users/fusebes/protobuf/bin:/Users/fusebes/go/bin:/Users/fusebes/vaultproject:/Users/fusebes/google-cloud-sdk/bin:/Users/fusebes/.rbenv/bin:/Users/fusebes/.rbenv/shims:/Users/fusebes/anaconda3/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/fusebes/Library/Android/sdk/platform-tools:/opt/flutter/bin
.... # Output truncated for brevity

Number of environment variables:  0