Case study

Root-causing a 504 on the OAuth callback

An opaque downstream timeout, localized by reading mesh and app logs side by side.

All case studies

Incident responseDistributed systemsAuth

2 correlated

Clusters

mesh vs. app

Signal

reusable triage

Outcome

Situation

Logging in through our enterprise identity provider's oauth-callback endpoint returned a 504 downstream duration timeout, but the application logs showed no obvious error.

Task

Determine whether the request was dying in the service mesh, the app pod, or a downstream dependency — across two clusters.

Action

Rather than guess, I pulled Istio proxy logs and application pod logs from both pods in both clusters and read them side by side, tracing the request path to find where it actually stalled.

Localize the layer first: did the request even reach the app? A 504 at the mesh with a clean app log points downstream.

I also checked for secondary 'logged-in' calls that might be involved, correlating timestamps across the mesh and the app to isolate the slow hop.

The triage question, in log termstext

# Mesh (Istio) access log — request arrived at the sidecar:
"GET /oauth-callback" 504 DC downstream_duration=... upstream_service_time=-

# App pod log — was there a matching handler entry?
(no matching request-received line) → request never reached the app
=> 504 is a mesh/downstream timeout, not an app exception

Result

Localized the failure to the correct layer, turning an opaque 504 into an actionable root cause.
Built a repeatable mesh-vs-app-vs-downstream triage approach I reused on later incidents.
Confirmed the fix should target the slow hop / timeout config, not the resolver or handler.

Next case study

Getting single-use auth-code semantics right