API Gateway - Distributed Swiss Army Knife

Goal

• To review the purpose of an API gateway in a microservices architecture, and to understand some nuances in introducing an API gateway into a Kubernetes environment.

Discussion

API Gateway - What and Why?

An API gateway is a key component of a microservice architecture. It provides a single point of entry into your distributed system, and abstracts away all the internals, like the fact that we may have multiple microservices co-ordinating to provide a service. API gateways provide a plethora of goodness:

• Perform the role of a Reverse Proxy: we don’t want our actual microservice to be exposed as is to the outside world. A reverse proxy abstracts away the server implementation from clients.
• Perform most L7 load balancing functions, including TLS/SSL termination
• Provide a single URI and route to multiple microservices behind-the-scenes
• Aggregate responses from multiple microservices
• Support specialized protocols like websockets and gRPC in addition to HTTP and HTTPS
• Protect microservices by providing authentication and authorization (role-based access control)
• Provide IP block and allow-listing to manage which clients can have access to which services
• Provide ability to throttle access to services via rate limiting
• Help observability by providing logging, monitoring and analytics
• Help deployment by supporting multiple versions, canary and blue/green deployments

All this goodness. How do I get it?

Kubernetes Load Balancing

I happily did a bit of hand-waving in our earlier post where we kubernetized our helloworld service. I said that Kubernetes can manage a cluster to provide scalability, elasticity and reliability, and we demonstrated that with a simple deployment. While we added nodes, removed them etc., we didn’t quite explore how Kubernetes route requests to a node in the cluster. Intuitively, we will expect some form of load balancing, and we would be right, sort of.

The resource in Kubernetes that plays a key role here is a Service. Without this abstraction, each client will need to know the actual pod address to be able to access a pod. Moreover, since pods are expected to go up and down in a typical Kubernetes environment, there is no reliable way for clients to have access to the pods. The service abstraction solves this problem by providing a static IP for the service, and hiding the pods behind it. Yes, like a load balancer. But, that’s where the resemblance stops.

This service isn’t actually a real LB/proxy, but instead, it used a process called kube-proxy, which provides L4 random selection load balancing. Kube-proxy achieves this by default using iptables rules, which basically maintains the mapping between the virtual IP address of the service and the real IP addresses of the pods. This worked well as a poor man’s load balancer for our toy implementation, but that’s about it. Also, remember that we are still only referring to private IP addresses within the cluster. How do we make an external client actually reach this service?

Recall our service configuration in our Kubernetes post:

kubectl expose deployment helloworld --type=NodePort --port=80 --target-port=8000

By this command, we created a Kubernetes service. Notice the type option. That indicates the service type of a Kubernetes service. This is how we expose the service to the external world.

• If we don’t specify a type, the default is ClusterIP. This makes the service accessible only by the other nodes of the cluster.
• NodePort type exposes the service to the network on a static port ranging from 30000 to 32767, continuing to use kube-proxy for LB between the nodes of the cluster.
• LoadBalancer type: that’s what we want, right? Yes, but…

The LoadBalancer type allows Kubernetes to create and use a cloud-based network (L4) LB on cloud-deployed Kubernetes setups. This cannot be used for Kubernetes on our own ‘bare-metal’ infra. This made bare-metal implementations like ours a bit of a second-class citizen for a while. But, now, we have options to mitigate this problem. We will be using one of them, called MetalLB. The premise of MetalLB is to provide a network load balancer so that bare-metal Kubernetes implementations can use the LoadBalancer service type.

API Gateway in Kubernetes

This still doesn’t answer how we can make use of a sophisticated API gateway or an L7 LB in Kubernetes.

For this, we need a different resource called an Ingress, usually managed by an ingress controller. No points for guessing that this will be the API gateway we are looking for. An ingress controller is part of the Kubernetes cluster and sits in front of our service. It functions as the reverse proxy, has routing rules and all that fancy stuff. In a cloud-based Kubernetes implementation, this just exposes itself to the outside world via the cloud-based LB created by Kubernetes. In our bare-metal world, it is going to do the same with the help of MetalLB.

Now, we can get real fancy here, and there are a multitude of ingress options that play nice with Kubernetes: NGINX, HAProxy, Kong, Traefik, Ambassador, Gloo and other Envoy-based API gateways, service meshes like Istio etc. The option we are going to play with, is [Ambassador Edge Stack] (https://www.getambassador.io/docs/edge-stack/). Here is a sneak peek at the architecture we will set up in the next post:

API Gateways vs Service Meshes

A quick sidebar regarding service meshes like Istio. At the outset, a service mesh is capable of handling many of the functions of an API gateway, like routing, authentication, rate limiting and monitoring. The main difference comes in the intent of usage. An API gateway is meant for communication coming from external traffic, whereas a service mesh is meant for internal service-to-service communication. Explained in a fancy way, an API gateway manages north-south traffic whereas a service mesh manages east-west traffic. A typical service mesh will be used like a sidecar proxy which can decouple internal-facing features like timeouts, retries, service discovery etc.

There are many use cases where both an API gateway and a service mesh could be employed together, and increasingly the boundaries are blurring. I intend to demonstrate a service mesh use-case in a later post.

Summary

This post is a quick introduction to the concept of API gateways. In the next post, we will see how can introduce Ambassador API gateway into our architecture.