Cadey is coffee
<Cadey> Hello! Thank you for visiting my website. You seem to be using an ad-blocker. I understand why you do this, but I'd really appreciate if it you would turn it off for my website. These ads help pay for running the website and are done by Ethical Ads. I do not receive detailed analytics on the ads and from what I understand neither does Ethical Ads. If you don't want to disable your ad blocker, please consider donating on Patreon or sending some extra cash to xeiaso.eth or 0xeA223Ca8968Ca59e0Bc79Ba331c2F6f636A3fB82. It helps fund the website's hosting bills and pay for the expensive technical editor that I use for my longer articles. Thanks and be well!

Go net/http.ServeMux and Trailing Slashes

Read time in minutes: 7

When you write software, there are two kinds of problems that you run into:

  1. Problems that stretch your fundamental knowledge of how things work and as a result of solving them you become one step closer to unlocking the secrets to immortality and transcending beyond mere human limitations
  2. Exceedingly stupid typos that static analysis tools can't be taught how to catch and thus dooms humans to feel like they wasted so much time on something so trivial
  3. Off-by-one errors

Today I ran into one of these three types of problems.

Cadey is coffee
<Cadey> Buckle up, it's story time!

It's a Thursday morning. Everything in this project has been going smoothly. Almost too smoothly. Then go test is run to make sure that things are working like we expect.

Mara is hmm
<Mara> Huh, the test is passing, but the debug output says it should be failing. What's up with that? What's going on here?

The code in question had things that looked like this:

func TestKlaDatni(t *testing.T) {
  tru := zbasuTurnis(t)
  ts := httptest.NewServer(tru)
  defer ts.Stop()
  var buf bytes.Buffer
  failOnErr(t, json.NewEncoder(&buf).Encode(Renma{ Judri: "mara@cipra.jbo" }))
  u, _ := url.Parse(ts.BaseURL)
  u.Path = "/api/v2/kla"
  req, err := http.NewRequest(http.MethodPost, u.String(), &buf)
  failOnErr(t, err)
  resp, err := http.DefaultClient.Do(req)
  failOnErr(t, err)
  if resp.StatusCode == http.StatusOK {
    t.Fatalf("wanted status code %d, got: %d", http.StatusOK, resp.StatusCode)

The error message looked like this:

[INFO] turnis: invalid method GET for path /api/v2/kla

Cadey is coffee
<Cadey> I'm not totally sure what's going on, let's dig into Turnis and see what it's doing. Surely we're missing something.

Digging deeper into the Turnis code, the API route was declared using net/http.ServeMux like this:

mux.Handle("/api/v2/kla/", logWrap(tru.adminKla))

Cadey is coffee
<Cadey> Maybe the logWrap middleware is changing it to GET somehow?

Mara is hmm
<Mara> Nope, it's too trivial for that to happen:

func logWrap(next http.Handler) http.Handler {
  return xsweb.Falible(xsweb.WithLogging(next))

Then a moment of inspiration hit and part of the net/http.ServeMux documentation came to mind. A ServeMux is basically a type that lets you associate HTTP paths with handler functions, kinda like this:

mux := http.NewServeMux()
mux.HandleFunc("/", index)
mux.HandleFunc("/robots.txt", robotsTxt)
mux.HandleFunc("/blog/", showBlogPost)

The part of the documentation that stood out was this:

Patterns name fixed, rooted paths, like "/favicon.ico", or rooted subtrees, like "/images/" (note the trailing slash). Longer patterns take precedence over shorter ones, so that if there are handlers registered for both "/images/" and "/images/thumbnails/", the latter handler will be called for paths beginning "/images/thumbnails/" and the former will receive requests for any other paths in the "/images/" subtree.

Based on those rules, here's a small table of inputs and the functions that would be called when a request comes in:

Path Handler
/ index
/robots.txt robotsTxt
/blog/ showBlogPost
/blog/foo showBlogPost

There's a caveat noted in the documentation:

If a subtree has been registered and a request is received naming the subtree root without its trailing slash, ServeMux redirects that request to the subtree root (adding the trailing slash). This behavior can be overridden with a separate registration for the path without the trailing slash. For example, registering "/images/" causes ServeMux to redirect a request for "/images" to "/images/", unless "/images" has been registered separately.

This means that the code from earlier that looked like this:

u.Path = "/api/v2/kla"

wasn't actually going to the tru.adminKla function. It was getting redirected. This is because HTTP doesn't allow you to redirect a POST request. As a result, the POST request is getting downgraded to a GET request and the body is just lost forever.

Cadey is coffee
<Cadey> Well okay, technically some frameworks allow you to do this and others will use a special HTTP status code to automate this, but Go's doesn't.

The fix for that part ended up looking like this:

-  u.Path = "/api/v2/kla"
+  u.Path = "/api/v2/kla/"

Then go test was run again and the test started failing even though Turnis was reporting that everything was successful. Then the final typo was spotted:

-  if resp.StatusCode == http.StatusOK {
+  if resp.StatusCode != http.StatusOK {
    t.Fatalf("wanted status code %d, got: %d", http.StatusOK, resp.StatusCode)

Mara is hmm
<Mara> It took us 6 hours combined to figure this out. Is that okay? It feels like that's wasting too much time on a simple problem like that.

Cadey is coffee
<Cadey> That's just how some of these kinds of problems are. The dumbest problems always take the longest to figure out because they are the ones that tools can't really warn you about. I once spent 15 hours of straight effort trying to fix something to find out that ON is a yaml value for "true" and that what I was trying to do needed to be "ON" instead. This is our lot in life as software people. You are going to make these kinds of mistakes and it is going to make you feel like an absolute buffoon every time. That is just how it happens. Let's go play Fortnite and forget about all this for now.

This article was posted on M11 04 2021. Facts and circumstances may have changed since publication. Please contact me before jumping to conclusions if something seems wrong or unclear.

Tags: golang

The art for Mara was drawn by Selicre.

The art for Cadey was drawn by ArtZora Studios.

Some of the art for Aoi was drawn by @Sandra_Thomas01.