Go net/http
mail

net/http is a very popular standard library with Go. It starts simple and comes with a rich set of features so that you can build, test and scale without even thinking about it. This article explains how pieces fit together. What is the difference between HandleFunc and HandlerFunc? How to write a middleware? If you are starting or still do not fully understand everything... That is very easy; check for yourself.

TL;DR http.Handler is an interface and the building block to manage requests. httptest helps you all the way around with a Recorder and a test Server. HandlerFunc transforms a function into an Handler. A middleware is a decorator that takes a Handler and returns a Handler. Handle and HandleFunc helps you with the net/http Multiplexer that is part of the standard library...

The following sections explain some of the key concepts with net/http, they explain why, what for and provide some examples.

Handler, Server and Listener

http.Handler is a Go interface that is used as the building block by the net/http library to create an HTTP server. It is bare minimal and is composed of one-only method. This is the interface signature ServeHTTP(http.ResponseWriter, *http.Request).

As a result of this construct, building a web server with the standard library is very cheap: (1) implement the Handler interface and (2) start a with it. You'll find below the timeless Hello World! example that does exactly that:

// a simple hello world web server
package main

import (
	"fmt"
	"net/http"
)

type helloworld struct{}

func (h *helloworld) ServeHTTP(w http.ResponseWriter,r *http.Request) {
	fmt.Fprintln(w, "Hello World!")
}

func main() {
	http.ListenAndServe(":8080", &helloworld{})
}

As you can see, the helloworld type implements the http.Handler interface. The main function calls the ListenAndServe helper that creates a listener with an instance of the helloworld type. You are done, you can run the code with go run and use a tool like curl to perform an http request:

curl 0.0.0.0:8080
Hello World!

Note in the example above, the ListenAndServe helper creates a default http.Server behind the scene. You may want to create/access that Server by yourself to setup some advanced configuration, like a request TTL or to shut it down gracefully

Client and Request

Now that you have a server, you should be able to perform a request. The net/http library provides two types, named Request and Client, with a set of methods to help you coding an http request. To use it, you can start with the simple requestHome example below:

// Simple Get on / request
package main

import (
	"errors"
	"io"
	"fmt"
	"net/http"
)

func requestHome() error {
	request, _ := http.NewRequest("Get", "http://localhost:8080", nil)
	client := &http.Client{}
	response, err := client.Do(request)
	if err != nil {
		return err
	}
	return readClose(response.Body)
}

func main() {
	err := requestHome()
	if err != nil {
		fmt.Printf("**Error**: %v", err)
		return
	}
}

func readClose(r io.ReadCloser) error {
	if r == nil {
		return errors.New("empty")
	}
	defer r.Close()
	data := make([]byte, 1024)
	for {
		n, err := r.Read(data)
		if n > 0 {
			fmt.Printf("%s", string(data[:n]))
		}
		if err == io.EOF {
			return nil
		}
		if err != nil {
			return err
		}
	}
}

Test a Handler

The net/http/httptest library comes with a set of helpers to test your Handlers without actually having to start a listener. In the next example, you will find:

  • How to create and use an httptest.Recorder that can be used as a http.ResponseWriter to test a Handler. The Result() method that is available on the Recorder returns a *http.Response as if the call had been done to a server with a Client/Request.
  • How to create a vanilla http.Request with the httptest.NewRequest() function to test Handlers.

In the example below, you create the Request and Recorder, use the Handler and can test the Response as if that was a remote call.

package main

import (
	"net/http"
	"net/http/httptest"
	"testing"
)

func TestHelloWorldHandler(t *testing.T) {
	recorder := httptest.NewRecorder()
	request := httptest.NewRequest("Get", "/", nil)
	handler := &helloworld{}
	handler.ServeHTTP(recorder, request)
	result := recorder.Result()
	if result.StatusCode != http.StatusOK {
		t.Errorf(
			"Status Code should be 200, current: %d",
			result.StatusCode,
		)
	}
}

As a matter of fact, the net/http/httptest package also include a constructor for an http.Server that might be useful. Below is the same test that relies on NewServer(). In that case, you can simply run the request with client.Do() without having to use a httptest.Recorder. You could use the pattern that suits you the most:

package main

import (
	"net/http"
	"net/http/httptest"
	"testing"
)

func TestHelloWorldServer(t *testing.T) {
	handler := &helloworld{}
	server := httptest.NewServer(handler)
	client := server.Client()

	request, _ := http.NewRequest("Get", server.URL, nil)
	response, err := client.Do(request)
	if err != nil {
		t.Errorf("Should not return %v", err)
	}
	if response.StatusCode != http.StatusOK {
		t.Errorf(
			"Status Code should be 200, current: %d",
			response.StatusCode,
		)
	}
}

HandlerFunc and Closure

You've probably heard it already, in Go functions are first class. As a result, people like to use func instead of type. So the http.HandlerFunc() function does exactly that, it transform a function with the same signature as the ServeHTTP function of the http.Handler interface into that interface.

A simple example is better than a 1000 words, you'll find the rewrite of the first helloworld example with a function instead of a type:

// a simple hello world web server
package main

import (
	"fmt"
	"net/http"
)

func hello(w http.ResponseWriter,r *http.Request) {
	fmt.Fprintln(w, "Hello World!")
}

func main() {
	handler := http.HandlerFunc(hello) // transforms hello into an Handler
	http.ListenAndServe(":8080", handler)
}

So if you shrink the code with an anonymous fonction and by doing the transformation in the call, you can some shorten code like this one:

// a simple hello world web server
package main

import (
	"fmt"
	"net/http"
)

func main() {
	http.ListenAndServe(
		":8080",
		http.HandlerFunc(
			func(w http.ResponseWriter,r *http.Request) {
				fmt.Fprintln(w, "Hello World!")
			}),
	)
}

Note Remember the Go proverb "Clear is better than clever" and think about the readers when you write the code above

The huge benefit of using function is to be able to use some sort of closure. Say you want your webserver to run in different language, you can create a type that depends on the output of a function like below:

// a simple hello world web server
package main

import (
	"flag"
	"fmt"
	"net/http"
)

func translatedHello(lang string) func(http.ResponseWriter, *http.Request) {
	msg := ""
	switch lang {
		case "fr":
			msg = "Bonjour le Monde!"
		default:
			msg = "Hello World!"
	}
	return func(w http.ResponseWriter,r *http.Request) {
		fmt.Fprintln(w, msg)
	}
}

func main() {
	lang := flag.String("lang", "en", "Server language")
	flag.Parse()
	http.ListenAndServe(
		":8080",
		http.HandlerFunc(translatedHello(*lang)),
	)
}

You'll see other patterns like the use of a container struct to help with dependency injection and interact with backend services or ease testing.

Middleware

Now lets consider a function that have a Handler as a parameter and returns an Handler as an output. What you have is the basis to build middleware. The idea behind middleware is to be able to separate a set of features out from your code, say in function g. This way if f contains your core logic, then g(f) contains your core logic to which you have added the g aspect.

There are dozen of examples, you can use middleware to log calls and compute response time. You can use them to check for authentication and autorizations. You can maintain sessions, cache some data, check/response to specific headers like CORS. The main benefit is to be able to build once and use everywhere.

But enough about theory, below is a very basic middleware example:

// a simple hello world web server with a middleware
package main

import (
	"fmt"
	"log"
	"net/http"
	"time"
)

func hello(w http.ResponseWriter,r *http.Request) {
	fmt.Fprintln(w, "Hello World!")
}

func logMiddleware(next http.Handler) http.Handler {
	return http.HandlerFunc(func (w http.ResponseWriter, r *http.Request) {
		starttime := time.Now()
		next.ServeHTTP(w, r)
		log.Printf(
			"%dms %s %s",
			time.Since(starttime).Milliseconds(),
			r.Method,
			r.URL.Path,
		)
	})
}

func main() {
	handler := http.HandlerFunc(hello)
	http.ListenAndServe(":8080", logMiddleware(handler))
}

As you have already figured out, you can chain middleware one after the other to compose your set of required features. One step further, you can also turn your middleware with a closure and create a function that returns a middleware.

Let say for instance, that you want to customize log format, you can very well create a function that returns func(http.Handler) http.Handler like below:

package main

import (
	"fmt"
	"log"
	"net/http"
	"time"
)

func hello(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintln(w, "Hello World!")
}

func logFormatMiddleware(format string) func(http.Handler) http.Handler {
	return func(next http.Handler) http.Handler {
		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			starttime := time.Now()
			next.ServeHTTP(w, r)
			log.Printf(
				format,
				time.Since(starttime).Milliseconds(),
				r.Method,
				r.URL.Path,
			)
		})
	}
}

func main() {
	handler := http.HandlerFunc(hello)
	http.ListenAndServe(
		":8080",
		logFormatMiddleware("%d [%s] \"%s\"")(handler))
}

Multiplexer, Handle and HandleFunc

One thing is still missing to the puzzle and that is the ability to split a Handler into pieces. Lets say for example that, if the user call /healthz you want 1 Handler to be triggered and it the user call /compute, you want another handler. That what a Multiplexer or Mux. A multiplexer is actually an Handler itself that changes the behavior depending on the Handler you've registered with it.

That is where the Handle and HandleFunc functions come into places. They basically say:

  • if the caller is using route /compute, call the compute Handler
  • if the caller is using route /healthz, call the healthz Handler

Again, an example helps better understand:

package main

import (
	"fmt"
	"net/http"
)

type helloworld struct{}

func (h *helloworld) ServeHTTP(w http.ResponseWriter,r *http.Request) {
	fmt.Fprintln(w, "Hello World!")
}

func goodbye(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintln(w, "Goodbye World!")
}

func main() {
	mux := http.NewServeMux()
	mux.Handle("/hello", helloworld)
	mux.HandleFunc("/goodbye", goodbye)
	http.ListenAndServe(
		":8080",
		mux)
}

As you have guessed already, Handle() is used to register an Handler with the net/http multiplexer, where HandleFunc() is used to register an HandlerFunc. In addition to that:

  • You do not need to create a multiplexer as there is a default multiplexer. if you use http.Handle() and http.HandleFunc() instead of the Handle() and HandleFunc() method, they register the Handler with the default multiplexer. If you set the second argument of http.ListenAndServe() to nil, it will also use the default multiplexer.
  • if you end the first argument of Handle() and HandleFunc() with a / like in mux.Handle("/hello/", helloworld), then it will not simply match the route but also the child routes, like /hello/gregory. You can then use regexp or other string tools on the URL to code different behaviors.

Utilities

As already said, not only the library is nicely written but it scales to a number of scenarios, like working with TLS certificates, managing cookies, using keep-alive, tracing, profiling many more. It is an awesome library!

Among the many very useful tools that comes with it, you can check httputil.DumpRequest() that dumps the HTTP request and is very useful. Below is a simple example of how to use it:

package main

import (
	"fmt"
	"net/http"
	"net/http/httputil"
)

func dump(w http.ResponseWriter, r *http.Request) {
	b, err := httputil.DumpRequest(r, false)
	if err != nil {
		fmt.Fprintf(w, "Error dumping request: %v", err)
	}
	fmt.Fprint(w, string(b))
}

func main() {
	http.ListenAndServe(
		":8080",
		http.HandlerFunc(dump))
}

And what it returns when used:

curl 0.0.0.0:8080/goodbye
GET /goodbye HTTP/1.1
Host: 0.0.0.0:8080
Accept: */*
User-Agent: curl/7.64.1

To continue

Many applications written in Go rely on HTTP to provide services like some sort of access to an API, some integration with external services or the publishing of a web site. If the standard library does not provide everything, it does provide a lot and you should take some time to learn and use it.

Not that Gorilla, Chi, Gin, Echo, Go Kit, FastHTTP, HTTPRouter are not good but you do not always need them.

Take your time to learn more about net/http; read the doc and read the code. There are so many things to say about it! No doubt a few articles will continue digging with it.