Skip to content

Responses WebSocket server_is_overloaded errors bypass model retry policies #3969

Closed

Description

@Filimoa

Please read this first

  • I read the Agents SDK model retry and Responses WebSocket documentation.
  • I searched open and closed issues/PRs for server_is_overloaded, service_unavailable_error, and ResponsesWebSocketError.

Describe the bug

When an established Responses WebSocket returns this error frame before any response.* event for the current request:

{
  "type": "error",
  "error": {
    "type": "service_unavailable_error",
    "code": "server_is_overloaded",
    "message": "Our servers are currently overloaded. Please try again later.",
    "param": null
  },
  "sequence_number": 2
}

Runner.run() terminates with ResponsesWebSocketError even when the run has ModelRetrySettings(max_retries=2) configured with retry_policies.provider_suggested() and transient HTTP status policies. No runner-managed retry or configured backoff occurs.

The equivalent transient failure over HTTP carries a 5xx status and is retryable. Over WebSocket transport, the overload is delivered as an error event without an HTTP status, so the signal currently falls through the retry classifier.

This is particularly costly for long-running agent workflows. Retrying the entire run at the application or worker level can repeat already-completed tool and sub-agent work, especially when running evals on the flex processing tier.

Debug information

  • Agents SDK version: 0.18.2
  • Python version: 3.14.3
  • OS: macOS
  • Transport: OpenAI Responses WebSocket
  • openai package version: 2.45.0
  • The relevant classifier is unchanged in 0.18.3 and on main at f663a06aea23c859be8e8555c005ac912bd58337

Repro steps

The service overload itself is intermittent, but the missing retry classification is deterministic:

from openai import AsyncOpenAI

from agents.models.openai_responses import (
    OpenAIResponsesWSModel,
    ResponsesWebSocketError,
)
from agents.retry import ModelRetryAdviceRequest

error = ResponsesWebSocketError(
    {
        "type": "error",
        "error": {
            "type": "service_unavailable_error",
            "code": "server_is_overloaded",
            "message": "Our servers are currently overloaded. Please try again later.",
            "param": None,
        },
        "sequence_number": 2,
    }
)

model = OpenAIResponsesWSModel(
    model="gpt-5.4",
    openai_client=AsyncOpenAI(api_key="test"),
)

for previous_response_id in (None, "resp_existing"):
    advice = model.get_retry_advice(
        ModelRetryAdviceRequest(
            error=error,
            attempt=1,
            stream=False,
            previous_response_id=previous_response_id,
        )
    )
    print(advice)

Actual output:

None
None

At runtime, this means retry_policies.provider_suggested() receives no positive provider advice. HTTP status policies cannot match either because the WebSocket error has no status code, so the transient overload terminates the run.

Expected behavior

A pre-response-event WebSocket rejection with code="server_is_overloaded" should be classified as a transient, provider-suggested retry.

Existing replay safeguards should remain unchanged once any response event has begun.

For stateful requests, replay safety may need to depend on the state owner and storage mode rather than solely on the absence of output:

  • A conversation_id or store-backed previous_response_id may be replayable after an explicit pre-response overload rejection.
  • A connection-local previous_response_id with store=false/ZDR should not receive unconditional replay approval, because a failed WebSocket continuation evicts that ID from the connection-local cache and there is no persisted fallback.

Additional context

ResponsesWebSocketError already retains code="server_is_overloaded" and error_type="service_unavailable_error". However:

  1. OpenAIResponsesWSModel.get_retry_advice() eventually delegates this error to the common OpenAI classifier.
  2. The common classifier recognizes retry headers, network/timeouts, and HTTP statuses such as 408/409/429/5xx.
  3. It normalizes error_code but does not currently use it to classify this transient overload.
  4. WebSocket error frames do not necessarily provide the equivalent HTTP 503 status, so this case returns no advice.

Would it make sense to classify this explicit pre-response server_is_overloaded rejection as provider-suggested retryable while preserving the current hard veto after response processing has begun?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions