A basic random-per-request load balancer
Let's go through the pieces necessary to create our random-per-request load balancer.
A quick note about the code in this chapter: Because we won't be using any packages outside of the standard library, programs listed here won't include the package main line nor the imports.
Luckily, the goimports tool will allow you to automatically add those sections to your code.
The first thing we will need is someplace to store our state. We have two servers we will be pointing at, and these servers' structs will both be storing their reverse proxy information.
We'll also need the ratio of how many requests we want to go to one versus the other. And because we are choosing randomly, we can include our random seed in all of this state.
So create a new directory for our canary deployment server, and then we can start our load_balancer.go file like this.
func main() {
// We'll fill this in later.}
// Server defines settings for this specific server.type Server struct {
proxy *httputil.ReverseProxy
}
// NewServer creates a Server reverse proxy for the given URL.func NewServer(sURL string) (Server, error) {
serverURL, err := url.Parse(sURL)
if err != nil {
return Server{}, err
}
return Server{proxy: httputil.NewSingleHostReverseProxy(serverURL)}, nil
}
// LoadBalancer is a collection of Servers.type LoadBalancer struct {
servers []Server
rng *rand.Rand
ratio int // Positive int less than or equal to 10
}
NewServer is a helper method that'll make it easier for us to set up backends in the future.
This code notably uses httputil, which is in the "net/http/httputil" standard library package. We're using this so we don't have to write our own reverse proxy. Even though that might be fun, there are a number of RFCs (RFC 7230 and RFC 2616) that determine what headers need to be stripped from the request, among other things that we probably want to let someone else handle for us. If you want to see what you're missing, check out the httputil.ServeHTTP method from the Go standard library.
Now that we have our state handled, we need to determine how to choose the right server to send each request to. Since we'll assume we always have two servers available, we only have to choose a random number, see if it is greater than our ratio, and use that to determine what server to send it to. It ends up looking like this.
// NextServer generates a random integer to determine if the 1st or 2nd Server should be proxied to.// When the random integer is greater or equal to the ratio, proxy the request to the 2nd Server.//// We always assume there are two Servers.func (lb *LoadBalancer) NextServer() *Server {
index := 0
if lb.rng.Intn(10) >= lb.ratio {
index = 1
}
return lb.servers[index]
}
With that out of the way, actually handling the request becomes easy. We choose a server, and then we reverse proxy the request to it.
// ProxyRequest forwards the request to the Server.func (lb *LoadBalancer) ProxyRequest(w http.ResponseWriter, r *http.Request) {
server := lb.NextServer()
server.proxy.ServeHTTP(w, r)
}
Now that we have all the pieces, we can glue them together with a main function. It has flags for the URLs of our servers which are used as parameters to the NewServer function to create our reverse proxies. Then we can create our load balancer with the servers and start it up.
That gives us this:
func main() {
var (s1 = flag.String("s1", "http://localhost:8080/", "first server")
s2 = flag.String("s2", "http://localhost:8081/", "second server")
ratio = flag.Uint("ratio", 5, "How many requests out of 10 should go to server 1, default 5")
)
flag.Parse()
server1, err := NewServer(*s1)
if err != nil {
log.Fatalf("Couldn't setup server1: %s", err)
}
server2, err := NewServer(*s2)
if err != nil {
log.Fatalf("Couldn't setup server2: %s", err)
}
loadBalancer := LoadBalancer{
servers: []Server{server1, server2},
rng: rand.New(rand.NewSource(time.Now().UnixNano())),
ratio: int(*ratio),
}
server := &http.Server{
Addr: ":8888",
Handler: http.HandlerFunc(loadBalancer.ProxyRequest),
}
log.Fatal(server.ListenAndServe())
}
And that's it!
Making sure it works#
Now that we have the code written, we should make sure it works. To do that, we need to have two servers to test with, and we'll need to start our load balancer, and then send the requests.
Let's create a new file in a different directory for this test, since it will need its own main function to run it all. Call the directory test, and the file run_lb.go.
Start by creating the server functions we need to test with in our run_lb.go file.
func main() {
// TODO}
// SimpleServer sets up a simple server that always returns a message on the specified port.func SimpleServer(port, message string) {
serveMux := http.NewServeMux()
serveMux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/plain")
fmt.Fprintf(w, message+"\n")
})
log.Fatalln(http.ListenAndServe(":"+port, serveMux))
}
These servers are very simple. They accept any request to any path and return whatever message we set them up with.
For convenience, this method will also run the server. However, that means we will need to run this method in a Go routine, by prepending go to it, otherwise we would never get past the server starting up.
Now that the servers are made, we will need to run our load balancer. We can do that with the handy exec package, which will allow us to use shell commands, including go build. So we can build our load balancer binary, wait for that to finish, and then run it.
We will also need to run our load balancer asynchronously, because we still need to send requests to it. It will take some time for the load balancer to start up, so we'll throw a sleep in there too.
Setting up our servers and load balancer looks like this.
func main() {
buildCmd := exec.Command("go", "build", "-o", "load_balancer", "load_balancer.go")
if err := buildCmd.Start(); err != nil {
log.Fatal(err)
}
buildCmd.Wait()
This page is a preview of Reliable Webservers with Go