Network Programming in Go

The Go runtime gives you a network stack that is unusual in a good way: every connection is a goroutine, every read and write is a blocking operation on that goroutine, and the scheduler multiplexes thousands of them onto a small number of threads. That means the code you write looks like ordinary sequential code, and the hard parts - timeouts, cancellation, backpressure - have to be handled explicitly, because the runtime will not save you.

1. TCP: The Shape of the Code

A TCP client is a dial, a deadline, and a loop.

func fetch(addr string, payload []byte) ([]byte, error) {
    conn, err := net.DialTimeout("tcp", addr, 5*time.Second)
    if err != nil {
        return nil, fmt.Errorf("dial %s: %w", addr, err)
    }
    defer conn.Close()

    if err := conn.SetDeadline(time.Now().Add(10 * time.Second)); err != nil {
        return nil, fmt.Errorf("set deadline: %w", err)
    }

    if _, err := conn.Write(payload); err != nil {
        return nil, fmt.Errorf("write: %w", err)
    }

    if err := conn.CloseWrite(); err != nil {
        return nil, fmt.Errorf("close write: %w", err)
    }

    var buf bytes.Buffer
    if _, err := io.Copy(&buf, conn); err != nil {
        return nil, fmt.Errorf("read: %w", err)
    }
    return buf.Bytes(), nil
}

CloseWrite matters for request/response protocols: it sends the FIN so the peer knows the request is complete, while the read side stays open for the response. Without it, the server waits for you to stop talking.

Handling many connections

Accepting connections in a loop and spawning a goroutine per connection is the standard server shape.

func serve(ctx context.Context, ln net.Listener, handle func(net.Conn)) error {
    go func() {
        <-ctx.Done()
        _ = ln.Close()
    }()

    for {
        conn, err := ln.Accept()
        if err != nil {
            if ctx.Err() != nil {
                return nil // shutting down
            }
            return fmt.Errorf("accept: %w", err)
        }

        go func() {
            defer conn.Close()
            handle(conn)
        }()
    }
}

Two production details are missing above and both matter: a limit on how many connections can be open at once, and a per-connection deadline. Without the first, a slow client can exhaust memory; without the second, a half-open connection holds a goroutine forever.

var sem = make(chan struct{}, 1000)

select {
case sem <- struct{}{}:
    go func() {
        defer func() { <-sem }()
        handle(conn)
    }()
default:
    conn.Close() // too many connections; shed load
}

2. UDP and DNS

UDP is connectionless, so the interesting part is handling datagrams that arrive out of order or not at all.

conn, err := net.ListenUDP("udp", &net.UDPAddr{Port: 9000})
if err != nil {
    return err
}
defer conn.Close()

if err := conn.SetReadBuffer(1 << 20); err != nil {
    return err
}

buf := make([]byte, 64*1024)
for {
    n, addr, err := conn.ReadFromUDP(buf)
    if err != nil {
        if errors.Is(err, net.ErrClosed) {
            return nil
        }
        log.Printf("read: %v", err)
        continue
    }
    go handleDatagram(buf[:n], addr)
}

The read buffer size is worth tuning explicitly: the default is often smaller than the datagram you expect, and the excess is silently discarded.

For DNS, use the net.Resolver rather than net.LookupIP so you can control the context and the resolver.

resolver := &net.Resolver{
    PreferGo: true,
    Dial: func(ctx context.Context, network, address string) (net.Conn, error) {
        d := net.Dialer{Timeout: 2 * time.Second}
        return d.DialContext(ctx, network, "10.0.0.53:53")
    },
}

ips, err := resolver.LookupIPAddr(ctx, "example.com")

3. HTTP Clients

The standard http.Client is the tool to reach for. Set a timeout on the client as a whole, and on the request as a deadline.

client := &http.Client{
    Timeout: 15 * time.Second,
    Transport: &http.Transport{
        MaxIdleConns:        100,
        MaxIdleConnsPerHost: 10,
        IdleConnTimeout:     90 * time.Second,
        TLSHandshakeTimeout: 5 * time.Second,
    },
}

req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, body)
if err != nil {
    return err
}
req.Header.Set("Content-Type", "application/json")

resp, err := client.Do(req)
if err != nil {
    return fmt.Errorf("post %s: %w", url, err)
}
defer resp.Body.Close()

if resp.StatusCode/100 != 2 {
    return fmt.Errorf("unexpected status %s", resp.Status)
}

var out Result
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
    return fmt.Errorf("decode: %w", err)
}

Two things to internalise. The Transport reuses connections, so a client built with http.DefaultClient under load will open a new TCP connection per request and exhaust the server’s accept queue. And the response body must always be closed and drained, or the connection cannot be returned to the pool.

4. Timeouts and Retries

Every network call needs a bound, and the bound belongs in three places: the dial, the response header, and the whole request. http.Client.Timeout covers the last; Transport.ResponseHeaderTimeout covers the second; Dialer.Timeout covers the first.

Retries need care, because a naive retry multiplies load exactly when the system is already struggling.

func withRetry(ctx context.Context, attempts int, fn func(context.Context) error) error {
    var err error
    backoff := 100 * time.Millisecond

    for attempt := 0; attempt < attempts; attempt++ {
        if err = fn(ctx); err == nil {
            return nil
        }
        if !isRetryable(err) {
            return err
        }
        select {
        case <-ctx.Done():
            return ctx.Err()
        case <-time.After(backoff):
        }
        backoff *= 2
        if backoff > 5*time.Second {
            backoff = 5 * time.Second
        }
    }
    return err
}

func isRetryable(err error) bool {
    var netErr net.Error
    if errors.As(err, &netErr) {
        return netErr.Timeout()
    }
    return false
}

Only retry errors that are plausibly transient - timeouts and connection resets - never a 400 or a 422. And cap the backoff, because exponential growth without a ceiling turns a brief blip into a long stall.

5. What to Take Away

  • One goroutine per connection, with a limit on how many may be open.
  • Always set deadlines: on the dial, on the read and write, and on the whole request.
  • Close the write side before reading a response, and always close and drain the response body.
  • Reuse a configured http.Transport instead of the default client.
  • Retry only transient failures, with exponential backoff and a ceiling.

Practice

GoTCPHTTP ClientTimeoutRetryBackoff

Approach: The question is about a client that hangs under load, so separate the causes: a hang can come from a connection that was never established, a connection that was established but never answered, or a server that answered slowly for only some requests. Each cause needs a different bound.

Steps:

  1. A call that never gets a connection is bounded by the dial timeout - without one, the default dialer waits for the operating system’s own timeout, which on Linux can be over two minutes.
  2. A connection that is established but where the server never sends a response header is bounded by Transport.ResponseHeaderTimeout; http.Client.Timeout covers the whole request including reading the body, and is the single value that most callers actually need.
  3. A request that is slow for only some calls points at the server or at a downstream dependency rather than at the client configuration, which is why the per-call context deadline matters: it lets a slow dependency be abandoned without taking the caller down with it.
  4. Retrying a timeout is reasonable; retrying a request that may have already been applied is not, unless the operation is idempotent. This is why the retry policy needs to know whether the server may have processed the request before the connection broke.
  5. Under load the symptom is usually connection exhaustion rather than slowness: with the default transport, each request may open a new connection, so the server’s accept queue fills. Configuring MaxIdleConnsPerHost and reusing the transport removes that failure mode.

Answer: Set a dial timeout, a response-header timeout, and an overall client timeout; give each call a context deadline; reuse a single configured transport so connections are pooled rather than created per request; and retry only transient errors, only for idempotent operations, with capped exponential backoff.

Found an issue with this content?

Tell us what could be improved.