Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,8 @@ Native code shared by both wrappers lives in `package/shared-native/{ios,android
- the main entry points must never reach `src/ai-components` or its dependencies (`react-syntax-highlighter`, `@khanacademy/simple-markdown`, `@stream-io/state-store`, `zod`, `victory-native`, Skia);
- the AI entry points may only reach `src/ai-components` plus an explicit allowlist (`src/native.ts`, the native shimmer). The wrappers' `ai-components/index.js` registers the native pieces the AI code needs itself and must never import the wrappers' `src/index.js`.

`AIConversation` and its list sit behind a third entry point, `ai-components/conversation` (`package/ai-components/conversation/`, `package/{native,expo}-package/ai-components/conversation/`), because `@legendapp/list/keyboard` requires `react-native-keyboard-controller` unguarded. The isolation check keeps the `ai-components` and `a2ui` entry points from reaching `src/ai-components/conversation` or `@legendapp/list/keyboard`, and lets them load keyboard-controller only through the guarded `src/ai-components/internal/keyboard.tsx`. The conversation's state layer (`AIConversationController`, its types, `useAIConversationState`) lives in `src/ai-components/conversation-state` and stays in the plain entry.

Do not add AI slots to `DEFAULT_COMPONENTS`; core `require`s the defaults, which would break isolation. Skia and `victory-native` are optional peers (native libraries must be owned by the app); the JS-only AI dependencies are regular dependencies.

Native pieces live in `package/shared-native/`: the Android-only `PerfText` Fabric view (Kotlin plus a C++ shadow node in `shared-native/android/jni`, built through each wrapper's `android/src/main/jni/CMakeLists.txt`, registered with `cmakeListsPath`/`componentDescriptors` in `react-native.config.js`) and the `AIDictation` TurboModule. **The iOS dictation sources are excluded from the pod unless the app opts in** (`$StreamChatEnableDictation = true` in the Podfile, or the `stream-chat-expo` config plugin in `package/expo-package/plugin`), because referencing the Speech framework makes App Store Connect demand a speech recognition usage description from every app. Never declare `RECORD_AUDIO` in a library manifest. The dictation codegen spec is duplicated into each wrapper's `src/native/` (codegen only reads a package's own `jsSrcsDir`); the isolation check keeps the copies identical.
Expand Down
2 changes: 1 addition & 1 deletion examples/ExpoChatGPTSample/screens/ChatContent.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ import { useStateStore } from 'stream-chat-expo';

import { createA2UIRenderer } from 'stream-chat-expo/a2ui';
import {
AIConversation,
type AIConversationController,
type AIConversationControllerState,
AISuggestions,
Expand All @@ -26,6 +25,7 @@ import {
isAIConversationAnswering,
useAIConversationState,
} from 'stream-chat-expo/ai-components';
import { AIConversation } from 'stream-chat-expo/ai-components/conversation';

import type { AgentChat, AgentChatState } from '../agent/agentChat';
import type { AgentRequestCustom } from '../agent/agentTransport';
Expand Down
6 changes: 6 additions & 0 deletions package/ai-components/conversation/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"main": "../../lib/commonjs/ai-components/conversation/index.js",
"module": "../../lib/module/ai-components/conversation/index.js",
"react-native": "../../src/ai-components/conversation/index.ts",
"types": "../../lib/typescript/ai-components/conversation/index.d.ts"
}
82 changes: 79 additions & 3 deletions package/bin/validate-entry-isolation.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,11 @@
* `ai-components/index.js`) may only reach the AI sources plus an explicit allowlist of other
* files. Apps that only use the AI components must not bundle the chat SDK. Nor the A2UI
* renderer: the `a2ui` entry points (core `src/a2ui/index.ts` and the wrappers'
* `a2ui/index.js`) reach it and `@a2ui/web_core`, besides what the AI entry points reach.
* `a2ui/index.js`) reach it and `@a2ui/web_core`, besides what the AI entry points reach. Nor
* the conversation, which needs react-native-keyboard-controller: the `ai-components/conversation`
* entry points (core `src/ai-components/conversation/index.ts` and the wrappers'
* `ai-components/conversation/index.js`) reach it, and the other AI entry points load
* react-native-keyboard-controller only through a guarded `require`.
* 3. The codegen spec copies the wrappers need (codegen only reads a package's own `jsSrcsDir`) stay
* byte-identical to the core source of truth.
*
Expand All @@ -26,6 +30,7 @@ const PACKAGE_DIR = path.resolve(__dirname, '..');
const CORE_SRC = path.join(PACKAGE_DIR, 'src');
const AI_SRC = path.join(CORE_SRC, 'ai-components');
const A2UI_SRC = path.join(CORE_SRC, 'a2ui');
const CONVERSATION_SRC = path.join(AI_SRC, 'conversation');
const WRAPPERS = ['native-package', 'expo-package'].map((name) => path.join(PACKAGE_DIR, name));

// Third-party packages only the AI components may use. (`lodash` is not one of them: the main
Expand All @@ -45,6 +50,16 @@ const AI_ONLY_PACKAGES = [
// that use the AI components without it.
const A2UI_ONLY_PACKAGES = ['@a2ui/web_core'];

// Optional peers the AI components load with a guarded `require`, each from one file only, so apps
// that don't install them still bundle.
const GUARDED_PACKAGES = {
'react-native-keyboard-controller': path.join(AI_SRC, 'internal/keyboard.tsx'),
};

// Dependencies only the conversation entry points may reach: they require
// react-native-keyboard-controller unguarded.
const CONVERSATION_ONLY_SPECIFIERS = ['@legendapp/list/keyboard'];

// Files outside `src/ai-components` that the AI entry points are allowed to reach.
const AI_ALLOWED_FILES = [
path.join(CORE_SRC, 'native.ts'),
Expand Down Expand Up @@ -98,6 +113,9 @@ const resolveSpecifier = (specifier, fromFile) => {
if (specifier === 'stream-chat-react-native-core/ai-components') {
return { file: path.join(AI_SRC, 'index.ts') };
}
if (specifier === 'stream-chat-react-native-core/ai-components/conversation') {
return { file: path.join(CONVERSATION_SRC, 'index.ts') };
}
if (specifier === 'stream-chat-react-native-core/a2ui') {
return { file: path.join(A2UI_SRC, 'index.ts') };
}
Expand Down Expand Up @@ -177,6 +195,32 @@ const errors = [];
const report = (message, chain) =>
errors.push(`${message}\n via: ${chain.map(relative).join('\n -> ')}`);

/** Checks that an entry point outside the conversation reaches neither it nor what it needs. */
const checkWithoutConversation = (entry, { files, packages }) => {
for (const [file, chain] of files) {
if (file.startsWith(CONVERSATION_SRC + path.sep)) {
report(`${relative(entry)} reaches the AI conversation (${relative(file)})`, chain);
}
for (const specifier of runtimeSpecifiers(file)) {
const guardedFile = GUARDED_PACKAGES[packageNameOf(specifier)];
if (guardedFile && file !== guardedFile) {
report(
`${relative(file)} imports the optional "${specifier}"; load it through ${relative(guardedFile)}`,
chain,
);
}
}
}
for (const [specifier, chain] of packages) {
if (CONVERSATION_ONLY_SPECIFIERS.includes(specifier)) {
report(
`${relative(entry)} reaches "${specifier}", which only the AI conversation may`,
chain,
);
}
}
};

// 1. Main entry points must not reach the AI components.
for (const entry of [
path.join(CORE_SRC, 'index.ts'),
Expand Down Expand Up @@ -205,7 +249,9 @@ for (const entry of [
path.join(AI_SRC, 'index.ts'),
...WRAPPERS.map((w) => path.join(w, 'ai-components/index.js')),
]) {
const { files, packages } = walk(entry);
const reached = walk(entry);
const { files, packages } = reached;
checkWithoutConversation(entry, reached);
for (const [file, chain] of files) {
if (!file.startsWith(AI_SRC + path.sep) && !AI_ALLOWED_FILES.includes(file)) {
report(
Expand All @@ -227,7 +273,9 @@ for (const entry of [
// 2b. The A2UI entry points reach the AI components and the A2UI renderer only.
const A2UI_WRAPPER_ENTRIES = WRAPPERS.map((w) => path.join(w, 'a2ui/index.js'));
for (const entry of [path.join(A2UI_SRC, 'index.ts'), ...A2UI_WRAPPER_ENTRIES]) {
const { files, packages } = walk(entry);
const reached = walk(entry);
const { files, packages } = reached;
checkWithoutConversation(entry, reached);
for (const [file, chain] of files) {
if (
!file.startsWith(AI_SRC + path.sep) &&
Expand All @@ -248,6 +296,34 @@ for (const entry of [path.join(A2UI_SRC, 'index.ts'), ...A2UI_WRAPPER_ENTRIES])
}
}

// 2c. The conversation entry points reach the AI components only.
const CONVERSATION_WRAPPER_ENTRIES = WRAPPERS.map((w) =>
path.join(w, 'ai-components/conversation/index.js'),
);
for (const entry of [path.join(CONVERSATION_SRC, 'index.ts'), ...CONVERSATION_WRAPPER_ENTRIES]) {
const { files, packages } = walk(entry);
for (const [file, chain] of files) {
if (
!file.startsWith(AI_SRC + path.sep) &&
!AI_ALLOWED_FILES.includes(file) &&
!CONVERSATION_WRAPPER_ENTRIES.includes(file)
) {
report(
`${relative(entry)} reaches ${relative(file)}, which is not in the AI allowlist`,
chain,
);
}
}
for (const [specifier, chain] of packages) {
if (AI_FORBIDDEN_PACKAGES.includes(specifier)) {
report(`${relative(entry)} reaches "${specifier}"`, chain);
}
if (A2UI_ONLY_PACKAGES.includes(packageNameOf(specifier))) {
report(`${relative(entry)} reaches the A2UI-only dependency "${specifier}"`, chain);
}
}
}

// 3. Codegen spec copies must not drift.
const copies = [
[
Expand Down
1 change: 1 addition & 0 deletions package/expo-package/ai-components/conversation/index.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from 'stream-chat-react-native-core/ai-components/conversation';
6 changes: 6 additions & 0 deletions package/expo-package/ai-components/conversation/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// The AI conversation, an entry point of its own because it needs react-native-keyboard-controller,
// which the rest of the AI components don't. It draws with the AI components, whose entry point
// registers the native pieces they need.
import '..';

export * from 'stream-chat-react-native-core/ai-components/conversation';
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from 'stream-chat-react-native-core/ai-components/conversation';
6 changes: 6 additions & 0 deletions package/native-package/ai-components/conversation/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// The AI conversation, an entry point of its own because it needs react-native-keyboard-controller,
// which the rest of the AI components don't. It draws with the AI components, whose entry point
// registers the native pieces they need.
import '..';

export * from 'stream-chat-react-native-core/ai-components/conversation';
11 changes: 10 additions & 1 deletion package/src/ai-components/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,14 @@ yarn add victory-native @shopify/react-native-skia

`react-native-reanimated`, `react-native-gesture-handler` and `react-native-svg` are already required by the chat SDK.

`AIConversation` comes from its own entry point, `stream-chat-react-native/ai-components/conversation` (or `stream-chat-expo/ai-components/conversation`), and needs [`react-native-keyboard-controller`](https://kirillzyusko.github.io/react-native-keyboard-controller/) with its `KeyboardProvider` above it:

```bash
yarn add react-native-keyboard-controller
```

The rest of the AI components work without it. With it installed and a `KeyboardProvider` above them, `ComposerView` rides the keyboard up; without it, the composer doesn't move with the keyboard.

The `zod` dependency ships `export * as` syntax, which the React Native Babel preset does not transform, so add the `export-namespace-from` plugin to your `babel.config.js`:

```bash
Expand Down Expand Up @@ -299,7 +307,8 @@ An agent can answer with UI instead of text: a form, a list of options, a card w

```tsx
import { createA2UIRenderer } from 'stream-chat-react-native/a2ui'; // or 'stream-chat-expo/a2ui'
import { AIConversation, AIUIHost, aiUIActionJSON } from 'stream-chat-react-native/ai-components';
import { AIUIHost, aiUIActionJSON } from 'stream-chat-react-native/ai-components';
import { AIConversation } from 'stream-chat-react-native/ai-components/conversation';

const Conversation = ({ messages, session }) => {
// One host per conversation, kept for as long as it shows, so what a reply's UI holds (a
Expand Down
6 changes: 4 additions & 2 deletions package/src/ai-components/__tests__/conversation.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,17 @@ import { Text } from 'react-native';

import { act, fireEvent, render, screen } from '@testing-library/react-native';

import { AIPendingMessageTracker, turnsOf } from '../conversation/entries';
import {
AIConversation,
type AIConversationAssistantMessageProps,
type AIConversationUserMessageProps,
} from '../conversation';
import { AIPendingMessageTracker, turnsOf } from '../conversation/entries';
import {
AIConversationController,
type AIConversationControllerOptions,
type AIConversationFeed,
type AIConversationTransport,
type AIConversationUserMessageProps,
type AIMessage,
type AIToolApprover,
createFromStreamMessage,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import type { AIConversationRequest, AIMessage } from '..';
import { AIConversationExchange } from '../conversation/controller/AIConversationExchange';
import { AIConversationExchange } from '../conversation-state/controller/AIConversationExchange';

const user = (id: string, text = 'Plan a weekend in Lisbon'): AIMessage => ({
id,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
import { renderHook } from '@testing-library/react-native';

import type { AIMessage, AIPendingMessage } from '../conversation/types';
import {
type ConversationRow,
useConversationRows,
type UseConversationRowsOptions,
} from '../conversation/useConversationRows';
import type { AIMessage, AIPendingMessage } from '../conversation-state/types';
import type { AIMessagePartData } from '../message-parts/types';

const user = (id: string, text: string): AIMessage => ({ id, role: 'user', text });
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ import { Pressable, StyleSheet, Text, View } from 'react-native';

import { act, fireEvent, render, screen } from '@testing-library/react-native';

import { AIConversation } from '../conversation';
import {
AIConversation,
AIConversationController,
type AIMessage,
AIUIComponentView,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
import React from 'react';
import { Keyboard, Text } from 'react-native';

import { act, fireEvent, render, screen } from '@testing-library/react-native';

import { ComposerView, sheetStore, sheetStoreApi, StreamTheme } from '../index';
import { dismissKeyboard, KeyboardStickyView, keyboardHeight } from '../internal/keyboard';

jest.mock('react-native-keyboard-controller', () => {
throw new Error("Cannot find module 'react-native-keyboard-controller'");
});

const flushPromises = () => new Promise<void>((resolve) => setTimeout(resolve, 0));

describe('without react-native-keyboard-controller', () => {
afterEach(() => {
act(() => {
sheetStoreApi.closeSheet();
});
jest.restoreAllMocks();
});

it('lays the sticky view out as a plain view', () => {
render(
<KeyboardStickyView offset={{ closed: 0, opened: 34 }} testID='sticky'>
<Text>inside</Text>
</KeyboardStickyView>,
);

const sticky = screen.getByTestId('sticky');
expect(sticky.type).toBe('View');
expect(sticky.props.offset).toBeUndefined();
expect(screen.getByText('inside')).toBeTruthy();
expect(keyboardHeight()).toBe(0);
});

it('closes the keyboard with React Native', async () => {
const dismiss = jest.spyOn(Keyboard, 'dismiss');

await expect(dismissKeyboard()).resolves.toBeUndefined();
expect(dismiss).toHaveBeenCalled();
});

it('renders the composer and opens its sheet', async () => {
const dismiss = jest.spyOn(Keyboard, 'dismiss');
render(
<StreamTheme>
<ComposerView onSendMessage={() => Promise.resolve()} />
</StreamTheme>,
);

await act(async () => {
fireEvent.press(screen.getByText('+'));
await flushPromises();
});

expect(dismiss).toHaveBeenCalled();
expect(sheetStore.getLatestValue().open).toBe(true);
});
});
3 changes: 3 additions & 0 deletions package/src/ai-components/conversation-state/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
export * from './controller';
export * from './fromStreamMessage';
export * from './types';
21 changes: 12 additions & 9 deletions package/src/ai-components/conversation/AIConversation.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,18 @@ import Animated, { Easing, FadeIn } from 'react-native-reanimated';

import type { LegendListRenderItemProps } from '@legendapp/list/react-native';

import type {
AIConversationController,
AIConversationControllerState,
} from './controller/AIConversationController';
import { editActionOf, replyActionsOf } from './controller/offers';
import { useAIConversationState } from './controller/useAIConversationState';
import type { AIConversationEntry } from './entries';
import { AIChatList } from './list/AIChatList';
import type { AIMessage, AIPendingMessage } from './types';
import { type ConversationRow, type RowPosition, useConversationRows } from './useConversationRows';

import { useTheme } from '../contexts';
import type {
AIConversationController,
AIConversationControllerState,
} from '../conversation-state/controller/AIConversationController';
import { editActionOf, replyActionsOf } from '../conversation-state/controller/offers';
import { useAIConversationState } from '../conversation-state/controller/useAIConversationState';
import type { AIMessage, AIPendingMessage } from '../conversation-state/types';
import type { AIUIHost } from '../generative-ui/AIUIHost';
import { AIUIGenerated, useAIUIHostRefresh } from '../generative-ui/AIUIView';
import { primitives } from '../internal/tokens';
Expand Down Expand Up @@ -103,8 +103,7 @@ export type AIConversationProps = {
* How far above the bottom of the screen the composer sits, such as above a tab bar. The
* keyboard comes up over the conversation, which makes room for it below the latest turn (the
* room the pinned question leaves takes it first), and covers it only as far as it rises past
* this. Defaults to the bottom safe area. Needs a `KeyboardProvider` from
* react-native-keyboard-controller above it.
* this. Defaults to the bottom safe area.
*/
keyboardOffset?: number;
/**
Expand Down Expand Up @@ -423,6 +422,10 @@ const DefaultConversationReplyActions = ({
* - A button scrolls back to the latest message when the end is out of view.
* - A message the person sent shows at once (`pendingMessage`) and the agent thinks under it
* (`AIThinking`) until its reply shows something.
*
* Import it from `stream-chat-react-native/ai-components/conversation` (or
* `stream-chat-expo/ai-components/conversation`). It needs react-native-keyboard-controller, with
* its `KeyboardProvider` above it.
*/
export const AIConversation = (props: AIConversationProps) =>
props.conversation ? (
Expand Down
2 changes: 1 addition & 1 deletion package/src/ai-components/conversation/entries.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import type { AIMessage, AIPendingMessage } from './types';
import type { AIMessage, AIPendingMessage } from '../conversation-state/types';

import type { AIUIDocument } from '../generative-ui/AIUIDocument';
import type { AIMessageAttachmentData } from '../message/AIMessageAttachment';
Expand Down
3 changes: 0 additions & 3 deletions package/src/ai-components/conversation/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,2 @@
export * from './AIConversation';
export * from './controller';
export * from './fromStreamMessage';
export * from './types';
export type { AIConversationCanShowUI, AIConversationEntry, AIConversationTurn } from './entries';
Loading
Loading