At a recent event in London, I was excited to learn how RedwoodSDK makes it easy to run React Server Components (RSC) on Cloudflare.
I decided to learn more about this by building multi-user AI chat. Code on GitHub. Deployed at https://agents-chat.jldec.me/.
RSC Chat - syncs via RedwoodSDK realtime websockets - stores messages in a separate durable object.
Agent Chat - syncs via Cloudflare Agents websockets - stores messages in a separate durable object.
Agent SDK Chat - uses AIChatAgent with the useAgentChat hook - stores messages in the same (per agent intance) durable object.
TinyBase Chat - syncs via TinyBase websockets and persists in TinyBase durable object.
Agent Agent Chat - More advanced Cloudflare agent with subagents and MCP tool calling (sse only, no auth). Syncs via agent websocket.
Here are just a few of the challenges:
RedwoodSDK (RSCs on Cloudflare workers) is very interesting. The addition of client-side navigation (SPA mode) together with Cloudflare cache integration, will make this stack hard to beat.
I was a bit surprised to learn that RSC pre-renders use client
components on the server, on initial load. This can produce errors e.g. running the useChatAgent()
hook from the agents sdk. The rwsdk team was super-responsive helping to debug the issue, and provided a solution to disable ssr. More details in this PR.
All implementations rely on Cloudflare durable objects with websockets. This is great for runtime performance and makes deployment easy. There are no containers to build or servers to manage.
React is great for a use case like this where updates are coming from both the server and the client. All implementations use the same MessageList component.
Server components are a succinct way to pre-populate JSX with data and then keep clients up to date.
It's nice to be able to use async data loading inline on the server. Rendering with data from remote storage during streaming can be slower unless data is memoized.
The scope of the RSC update payload sent to clients may become a problem during streaming, e.g. for pages with a lot of data. Discussion about this in the rwsdk discord.
Server functions are convenient, but should be used with care since they generate HTTP APIs which is where auth/authz is commonly required. See this take from Jack Herrington for more.
Using Cloudflare Agents raw websockets gives us full control over the payloads. This allows for nice optimizations e.g. to send partial data during streaming.
Rendering chat history on the client via fetch or via websocket makes the initial UX a little janky. (TODO: investigate pre-rendering)
AIChatAgent handles multi-user real-time message sync over websockets. This simplifies the implementation.
The SDK abstracts tool calling and supports different LLMs with Vercel's AI SDK.
useAgentChat which calls AI SDK's useChat, manages chat UI interactions with react, however only a single client sees the AI streaming reponse (see #23).
MCP tools can be added, removed or listed.
A built-in tool calls AIChatAgent.saveMessages() on a named subagent, passing in a new message, as if it were coming from a user.
This makes it possible for the main agent to prompt the subagent.
There are also built-in tools for clearing and listing messages.
Subagent responses currently don't stream (TODO)
Synchronization is happening between memory and persistance on every node, and between nodes.
This improves the UX once data is persisted on the client and makes it easy to use React hooks listening for database updates.
The APIs for persistence and synchronization feel like they could be consolidated.
Since store operations run on the client we have to be extra careful with validation e.g. to deal with clients being compromized.
I'd ❤️ to see if there are ways to combine the best of all the approaches above.
And I believe multi-user AI chat could be the beginning of something much bigger.
- Agents and humans organize into specialized groups.
- Universal agents with tools do much of the work.
- Persisted conversations become the new basis for organizational memory.
Bright futures ahead 🚀