Ollama tool-calling loop does not complete for query/chat on v0.4.5 and v0.5.0-rc1 #205

Summary

openkb add works with local Ollama models, but openkb query does not reliably complete the tool-calling loop. This remains reproducible on OpenKB v0.5.0-rc1, even with the litellm.drop_params configuration introduced for Ollama compatibility.

The models can emit tool calls, but the CLI either:

• prints raw tool-call JSON instead of executing the tool and continuing;
• returns {};
• returns an incomplete/raw response;
• or times out without a final answer.

Environment

• Linux x86_64
• OpenKB v0.5.0-rc1
• openai-agents==0.17.3
• Ollama running as a separate local service
• Ollama accessed through its OpenAI-compatible/LiteLLM route
• Knowledge base containing indexed Markdown summaries
• Query executed through the normal CLI:

openkb --kb-dir <kb-dir> query "What topics are discussed in these messages?"

No cloud API, private endpoint, API key, token, or other credential is required to reproduce the local-backend behavior.

Configuration

The KB configuration was equivalent to:

model: ollama/llama3.1:8b-instruct-q8_0
api_base: http://ollama-host:11434
language: en
pageindex_threshold: 20
parallel_tool_calls: false
litellm:
  drop_params: true

The hostname above is intentionally a placeholder. The real endpoint is private and is not part of this report.

Reproduction

1. Create or use a KB with several indexed Markdown documents.
2. Configure the KB for a local Ollama model as shown above.
3. Run:

openkb --kb-dir <kb-dir> query "What topics are discussed in these messages?"

4. Observe that the query does not produce a normal final answer grounded in the KB.

Observed result with llama3.1:8b-instruct-q8_0

The CLI returned raw JSON similar to:

{
  "reqId": "<redacted>",
  "message": "<a requested wiki path was not found>",
  "toolCalls": [
    {
      "id": "<redacted>",
      "type": "function",
      "name": "read_file",
      "arguments": {"path": "summaries/index.md"}
    }
  ]
}

The tool call was not followed by a normal final answer.

The same model was also tested through the non-streaming Runner.run path. In that path it attempted to call a tool named search_strategy, which was not one of the tools registered by OpenKB:

ModelBehaviorError: Tool search_strategy not found in agent wiki-query

Additional local-model results

Using the same KB, question, and OpenKB v0.5.0-rc1 CLI:

openkb add had previously completed successfully on the same type of KB, including document summaries. The failure is specific to the query/chat agent path and tool-loop completion, not basic LiteLLM connectivity.

Expected behavior

For an Ollama-compatible model that returns a valid structured tool call, OpenKB should:

1. parse the tool call;
2. execute the registered OpenKB tool;
3. append the tool result to the agent conversation using the correct Chat Completions/LiteLLM format;
4. continue the agent loop;
5. return a final natural-language answer.

If a model returns an unsupported or malformed tool call, OpenKB should report a clear actionable error rather than emitting raw JSON or returning {}.

Investigation notes

• Issue [Feature] Plans to support litellm config in .openkb/config.yaml? (Encountered UnsupportedParamsError with Ollama) #137 and PR feat(config): pass through global LiteLLM settings via a litellm: config block #138 address the parallel_tool_calls parameter rejected by Ollama. Setting parallel_tool_calls: false and litellm.drop_params: true avoids that specific parameter error, but does not solve this issue.
• OpenKB query code still uses Runner.run_streamed(...) for the streaming path.
• The non-streaming path also fails with some local models, so this may involve the Agents SDK/LiteLLM message-adapter format, model-specific tool-call behavior, or both.
• Direct low-level LiteLLM tests can return structured tool calls for some Ollama models, but the full OpenKB query agent does not reliably consume and continue from them.

Request

Could the maintainers please confirm:

1. which Ollama/LiteLLM model and version combinations are officially supported for query/chat;
2. whether OpenKB expects OpenAI Responses API semantics or Chat Completions semantics for the LiteLLM provider;
3. whether the streaming path is expected to support Ollama tool calls;
4. whether a non-streaming fallback should be used for LiteLLM/Ollama providers;
5. whether malformed model-emitted tool names should be handled with a clearer validation error.

A small regression test covering one structured Ollama tool call, tool execution, and a final answer would help prevent this from recurring.