Stream tokens in a React app

By Ryze AI Team

This tutorial streams LLM tokens into a React component using the Fetch API and a readable stream.

1. Component skeleton

import { useState } from 'react';
export default function Chat() {
  const [text, setText] = useState('');
  const [loading, setLoading] = useState(false);
  // ...
}

2. Fetch and read the stream

async function send(prompt) {
  setLoading(true); setText('');
  const res = await fetch('/api/chat', { method: 'POST', headers: {'Content-Type':'application/json'}, body: JSON.stringify({prompt}) });
  const reader = res.body.getReader();
  const dec = new TextDecoder();
  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    setText(t => t + dec.decode(value, { stream: true }));
  }
  setLoading(false);
}

3. Stop button

Store the reader in a ref and call reader.cancel() on stop. An AbortController also works and cancels the fetch itself.

4. UX details

Show a typing indicator while loading. Auto-scroll to the bottom but pause if the user scrolls up. Disable the input while streaming. Render markdown after completion, or use a streaming markdown renderer.

5. Error handling

Wrap the loop in try/catch. On network error, show the partial text with a retry option. Handle non-200 responses before reading the body.