Skip to content

StreamingTranscriber cannot reconnect after a dropped connection: connect() throws "Already connected", and close() removes all listeners #228

Open

Description

@Pulkit7070

Summary

When the streaming socket closes unexpectedly after Begin (for example a network drop, close code 1006), StreamingTranscriber.connect() rejects with Already connected, also when called from the close listener, because the dead socket is never cleared. The only way to open a new session on the same instance is close() then connect(), but close() clears every listener, so the new session's turn, error and close events are silently dropped. TextToSpeechSession in the same SDK supports calling connect() again from its close listener, so the streaming transcriber behaves differently from what users will expect.

Steps to reproduce

  1. In an empty directory: npm init -y && npm i assemblyai@4.43.4 (installs ws too), then create a self-signed certificate, since the SDK only accepts wss: URLs:
openssl req -x509 -newkey rsa:2048 -nodes -keyout key.pem -out cert.pem -days 1 -subj /CN=127.0.0.1
  1. Save as reconnect.mjs. A local server sends Begin and one Turn, and drops the first session after 150 ms:
import fs from "node:fs";
import https from "node:https";
import { WebSocketServer } from "ws";
import { StreamingTranscriber } from "assemblyai";

process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";
const server = https.createServer({ key: fs.readFileSync("key.pem"), cert: fs.readFileSync("cert.pem") });
const wss = new WebSocketServer({ server });
let sessions = 0;
wss.on("connection", (sock) => {
  const n = ++sessions;
  sock.send(JSON.stringify({ type: "Begin", id: "session-" + n, expires_at: 0 }));
  setTimeout(() => sock.send(JSON.stringify({ type: "Turn", turn_order: 0, transcript: "hello from session-" + n, end_of_turn: true, words: [] })), 50);
  if (n === 1) setTimeout(() => sock.terminate(), 150);
});
await new Promise((r) => server.listen(0, "127.0.0.1", r));

const rt = new StreamingTranscriber({
  apiKey: "test-key",
  sampleRate: 16000,
  websocketBaseUrl: `wss://127.0.0.1:${server.address().port}/v3/ws`,
});
rt.on("turn", (t) => console.log("turn:", t.transcript));
rt.on("close", async (code) => {
  console.log("close listener, code", code);
  try {
    await rt.connect();
    console.log("reconnected from close listener");
  } catch (e) {
    console.log("connect() from close listener threw:", e.message);
  }
});

await rt.connect();
await new Promise((r) => setTimeout(r, 400));
await rt.close();
const begin = await rt.connect();
console.log("connect() after close() opened", begin.id);
await new Promise((r) => setTimeout(r, 300));
console.log("sessions opened on server:", sessions);
process.exit(0);
  1. Run node reconnect.mjs.

Expected

After the drop, connect() from the close listener opens a new session, and the existing turn listener prints turn: hello from session-2.

Actual

turn: hello from session-1
close listener, code 1006
connect() from close listener threw: Already connected
connect() after close() opened session-2
sessions opened on server: 2

The server sent a Turn on session-2, but nothing is printed for it: close() removed the turn listener.

How I found and tested this

I found this while reading how StreamingTranscriber handles a socket that closes after the session has started, to see what a long-running voice app has to do to recover. The connect() guard checks this.socket, the close handler for an established session never clears it, and close() resets this.listeners.

To test it I installed assemblyai@4.43.4 from npm into an empty project and ran the script above against a local wss server that sends Begin and a Turn and then drops the first session at the TCP level. No request went to the AssemblyAI API. I did not test in a browser; the code path is shared.

In practice a voice app that loses its connection once (Wi-Fi change, proxy idle timeout, server restart) either cannot reconnect, or reconnects and then receives no transcripts, with no error to explain why.

Environment

  • assemblyai 4.43.4 from npm (same as main at commit 92903c1)
  • Node.js 22.23.1, macOS
  • connect() guard:
    if (this.socket) {
    throw new Error("Already connected");
    }
  • post-Begin onclose keeps this.socket:
    this.socket.onclose = ({ code, reason }: CloseEvent) => {
    if (!reason) {
    if (code in StreamingErrorMessages) {
    reason = StreamingErrorMessages[code as StreamingErrorTypeCodes];
    }
    }
    // A close before `Begin` is a failed connection attempt — reject so
    // connect() can retry (or surface a permanent failure).
    if (!settled) {
    const err = new StreamingError(
    reason || `Streaming connection closed (code=${code})`,
    ) as ConnectionAttemptError;
    err.code = code;
    err.retryable = isRetryableCloseCode(code);
    failAttempt(err);
    return;
    }
    // Stop the flush timer when the socket is gone (server-initiated close,
    // network drop, etc.) — otherwise subsequent ticks call send() on a
    // closed socket and spam the error listener.
    if (this.flushTimer) {
    clearInterval(this.flushTimer);
    this.flushTimer = undefined;
    }
    // The socket is gone, so no `Termination` frame is coming. Release a
    // `close()` that is waiting for one.
    this.resolveSessionTermination();
    this.emitClose(code, reason);
    };
  • close() clears listeners:

Suggested fix

In the post-Begin onclose branch, set this.socket = undefined so connect() can open a new session, and keep this.listeners across close() (or document that listeners must be registered again after close()).

Related

I checked the existing issues and pull requests and did not find this reported. #192, #173 and #155 deal with connect() and close() while the first connection is still being set up, not with reconnecting after an established session drops.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions