Repository navigation
Potential misleading documentation on OpenAiClientAsync #644
Description
OpenAIClientAsync's documentation states:
This client performs best when you create a single instance and reuse it for all interactions
with the REST API. This is because each client holds its own connection pool and thread pools.
Reusing connections and threads reduces latency and saves memory. The client also handles rate
limiting per client. This means that creating and using multiple instances at the same time will
not respect rate limits. (emphasis mine)
However I find no evidence on rate limits actually being used proactively as the text suggests. I can only find evidence of reactive retry, when encountering 429 errors it automatically retries with an exponential backoff. However, that is request specific, not global to the client.
Is my understanding incorrect here or is this a documentation error? Thanks in advance.
Activity
dpiet-oai commentedon Sep 30, 2026
Fixed in #1118, now merged into main. Both synchronous and asynchronous client documentation now clarify client reuse, per-request retries, and application-managed request rates. Thank you for reporting the misleading wording.