SigV4 authentication is surprisingly complicated

SigV4 looks simple: sign a request, check the signature. Then you implement canonicalization, clock skew, and a cache that isn't allowed to hold your key.

Tigris is a drop-in replacement for AWS S3 (or GCS, anything S3API compatible). As such, we need to be fully compatible with both the mechanisms and semantics of S3 including the SigV4 authentication protocol. This is the lingua franca of authentication in the object storage landscape; even Google Cloud Storage has a way to enable SigV4 support so you can use existing applications against its object storage service.

At first I thought that SigV4 was fairly simple. Clients sign requests, servers do the same work and make sure the result matches. The main sticking point is that the cryptography involved is symmetric cryptography, the kind where both parties need to have the same secrets. This makes some scaling issues weird, but we'll get into that in the future.

SigV4 in a shellnut

At a high level when a client signs a request with SigV4 you get an access key ID and secret access key. The access key ID is functionally a username and the secret access key is functionally a password. Admins can identify keypairs by the access key ID (without special training or tools) and services use the owner of the access key or policies delegated to that access key to determine what actions that client may take.

SigV4 uses HMAC (hash-based Message Authentication Code) and SHA-256 (SHA-2 with a 256 bit hash width) to do authentication by creating salted hashes based on request metadata.

In order to send a SigV4 request, clients take the outgoing request, reduce it to a canonicalized form, and sign it with a symmetric key derived from the secret access key, the current date, region of the service, and service name, kinda like this Go code:

func HMAC(key, data []byte) []byte {
	h := hmac.New(sha256.New, key)
	h.Write(data)
	return h.Sum(nil)
}

var (
	kDate    = HMAC("AWS4"+secretAccessKey, nowDate)
	kRegion  = HMAC(kDate, region)
	kService = HMAC(kRegion, service)
	kSigning = HMAC(kService, "aws4_request")
)

As an example, let's see what a signed GET request to a HTTP debugging endpoint looks like on the wire with and without the signature:

$ curl http://localhost:3000 -v

GET /
User-Agent: curl/8.7.1
Accept: */*

And when you add the signature with --aws-sigv4:

$ curl \
  --user tid_YOISC719YLXSONFU:tsec_DiYqeH8t0IKjKUKfqhzTsqrCCUl9Wm0m+6MXNhhi1fU \
  --aws-sigv4 aws:amz:auto:s3 \
  -v \
  http://localhost:3000

GET /
User-Agent: curl/8.7.1
Accept: */*
Authorization:
  AWS4-HMAC-SHA256
  Credential=tid_YOISC719YLXSONFU/20260720/auto/s3/aws4_request,
  SignedHeaders=host;x-amz-date,
  Signature=879bcdd43749cfc9782b876d9ceb3ff153d79ab1482290cca7ab915bb7f8785d
X-Amz-Date: 20260720T153748Z

Breaking it down we have two extra headers in the request:

• Authorization: The fixed string AWS4-HMAC-SHA256 to signal to the server which authentication mechanism is in use. The rest of the string is information about the request signature so the server can properly canonicalize the request.
• X-Amz-Date: The date and time (UTC) of the request so the server knows when the request was signed. Servers will use this request date in order to reject old requests to prevent replay attacks.

Request canonicalization and signing

On the wire, HTTP/1.1 requests look kinda like this:

GET /api/list?page=0&count=30
User-Agent: curl/8.7.1
Accept: */*
Host: myawesomesite.example

However the headers could be sent in any order, and changing the order of request headers doesn't result in different requests. Additionally any query string parameters could be formatted in any way a client (or server) could imagine, including the use of semicolons to separate values. All attempts to canonicalise HTTP requests MUST deal with this ambiguity and define their own rules.

SigV4 canonical requests are made up of a few parts:

• The HTTP method (GET, PUT, POST, DELETE, etc.)
• The URI path of the request (/api/list, etc.)
• The sorted canonical query string (you must exactly match the server-side canonicalization logic)
• The signed headers terminated with two newlines
• The sorted list of signed headers joined by semicolons
• The SHA256 checksum of the request body

For that example /api/list request, the canonical form would look like this:

GET
/api/list
count=30&page=0
host:myawesomesite.example
x-amz-date:20260715T204745Z

host;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

As the request has no body, the empty sha256 checksum e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 is put as the body checksum.

I'm reading the blogpost at <link> and Xe mentioned AWS SigV4's
STREAMING-AWS4-HMAC-SHA256-PAYLOAD method. I would like to learn more about how
this works. Please research how this works and give me code and request body
samples.

To make the signature, you take the sha256 checksum of the canonical request and then HMAC it against that derived signing key:

finalRequestSignature := HMAC(kSigning, reqSig.Bytes())

And construct the Authorization header based on your access key ID, service region, and service name:

req.Header.Set("Authorization", fmt.Sprintf(
	"AWS4-HMAC-SHA256 Credential=%s/%s/%s/%s/aws4_request, SignedHeaders=%s, Signature=%x",
	accessKeyID, nowDate, region, service,
	strings.Join(signedHeaders, ";"),
	finalRequestSignature,
))

What about SigV4a?

AWS has made an extension to SigV4 that uses asymmetric cryptography called SigV4a (the "a" means asymmetric). Instead of using symmetric cryptography on both the client and server in ways that means the server needs to either know the client's secret access key (or a value derived from the secret access key), SigV4a uses key derivation functions to derive a cryptographic keypair. Servers authenticating requests fetch the public key from IAM. Only the client and IAM know what the private key is, and that private key is what signs outgoing requests.

I'd love to use SigV4a more because it makes adding additional services to the mix (such as a git service) a lot safer as you can have those additional services exist in different trust domains than the core product. This is the core of how microservices end up happening. However, it's not super widely used even within AWS. The only SigV4a use I can find in Amazon is S3 Express Zones, however they may end up using it in other services I'm just not aware of.

When I did my own experimentation with SigV4a (where I was implementing my own IAM server so that I really understood this all at a low level), I had to copy a lot of internal AWS SDK code into my repo in order to get it working.

I'll talk about SigV4a some more another time.

Replay attacks and you: a young coder's illustrated primer

One of the weaknesses of using signatures for API authentication like this is the problem of replay attacks. When you make a naïve signature of a value, there's no real way to tell when that signature was created. If you sign a request to create a compute instance at time instance t0, it's still technically valid at any other time instance tN. This is why the canonical form of SigV4 requests includes the current date and time:

Authorization: [...] SignedHeaders=host;x-amz-date, [...]
X-Amz-Date: 20260715T205432Z

This means that the request was signed on July 15, 2026 at 20:54:32 UTC. Time changes constantly (at least at the rate of one second per second!) and the client has to have a working clock in order for TLS to work. Servers can trivially read the contents of X-Amz-Date and reject old requests. This means that you don't need to add or store nonce (number used once) values with each request because that doesn't scale.

As a side effect of this, you need to set a temporal skew window for validating requests. This window needs to be generous enough to accommodate slow clients, sloppy timekeeping on the client side, highly latent clients, leap seconds, or other exceptional temporal phenomena. In general time synchronization is a surprisingly hard problem, so it's best to just be tolerant of clients in order to make things more robust in practice. AWS uses a temporal skew window of 15 minutes for validating requests. I'm going to use a window of 5 minutes for my API because 300 seconds is a nice round number and I don't have to deal with the same amount of legacy code that AWS does.

How TAG changes the game

So all of this SigV4 business had been working really well for Tigris. Then we worked with a few customers who needed a local cache to fully saturate their hungry GPUs. To be fair, Tigris is plenty fast, but the real thing that kills AI training is latency and something that runs locally will always be faster than the cloud.

In order to provide that sweet middle spot between making everything rely on the cloud and having everything local, we made TAG, the Tigris Acceleration Gateway. This effectively gives you most of a Tigris region in your own infrastructure.

When you connect to TAG, your code uses its existing access keypairs, buckets, and code. You point your code to TAG, you point TAG to Tigris, and then everything is cached for you. But how does TAG authenticate with your code? TAG doesn't have access to all your existing API keys (and to be honest it shouldn't), but it's still able to authenticate them with SigV4 authentication.

TAG and the IAM server both implement a signing key proxying feature that lets a client and TAG both prove their identity to Tigris. Once that proof is sent, then TAG gets the intermediate derived signing key and uses that for locally validating requests, kinda like this:

sequenceDiagram
   participant Client
   participant TAG
   participant Tigris

   Client->>TAG: ListBuckets<br/>(signed)
   TAG->>Tigris: ListBuckets<br/>(signed) + proxy hdrs
   Note right of Tigris: 2xx, keys returned
   Tigris-->>TAG: ListBucketsResponse<br/>+ keys (encrypted)
   Note right of TAG: decrypt, cache
   TAG-->>Client: ListBucketsResponse

   Client->>TAG: ListBuckets<br/>(signed)
   Note right of TAG: verify locally,<br/>cache hit
   TAG-->>Client: 200 OK

The actual implementation in TAG involves some derived AES logic so that the derived signing keys are very much limited to the client that requested it (namely: the AES key is the SHA256 encoded form of the proxy secret access key). One of the weird parts is that the canonical form of the proxied requests differ from the normal SigV4 canonicalization process, namely looking like this:

tag.default.svc.cluster.local # Host header from the client
1784577479                    # Unix timestamp of the request (X-Tigris-Proxy-Timestamp)
GET                           # HTTP method of the client
/                             # HTTP path of the client

This is signed using the same SigV4 signature process as before but added differently to the request:

• X-Tigris-Forwarded-Host: the HTTP Host of client requests (EG: tag.default.svc.cluster.local)
• X-Tigris-Proxy-Access-Key: the Tigris keypair used to authenticate TAG itself (must be in the same organization as the client)
• X-Tigris-Proxy-Timestamp: the time of the request in unix timestamp format
• X-Tigris-Proxy-Signature: the hex output of signing the canonical form of the request against TAG's secret access key

And then TAG reads the response from Tigris, caches those derived signing keys, and then uses those in the standard SigV4 process to authenticate clients: no round trip to the cloud required.

I was wrong about the simple part

The happy path is exactly what I thought it was. Reduce a request to a canonical form, run four HMACs, compare the result. That part fits in an afternoon.

Everything expensive lives in the questions around it. Which bytes count as the request? Whose clock decides that a signature is still good? Who gets to hold the key that proves any of it? Each question has an obvious answer, and each obvious answer is wrong in some specific way you only find by implementing it.

That last question is the one that surprised me. I read symmetric cryptography as a hard limit: if the verifier needs your secret, the verifier has to be Tigris. It isn't. SigV4 derives its signing key through a chain of four HMACs, each one scoped tighter than the last: date, then region, then service. Those intermediate values can travel without the secret behind them. TAG rides that. The key it holds stops working when the UTC date rolls over. It covers one region and one service. You can't walk it backwards into a secret access key.

We also didn't write any of this, which is its own kind of relief. SigV4 is old, widely deployed, and hammered on by every S3 client in existence. Any compatibility bugs here are ours. The protocol's bugs are everyone's.

The place a protocol bends is usually some intermediate value that somebody already designed to be thrown away.

If you want a Tigris region in your own datacentre, the Tigris Acceleration Gateway caches your buckets locally and authenticates your existing keypairs with the same SigV4 dance your SDK already speaks.

Facts and circumstances may have changed since publication. Please contact me before jumping to conclusions if something seems wrong or unclear.

Tags: