At Air-Coach we try develop a chat app that can help skydivers providing an AI agent built on our specific documentation. In our codebase in app, at some point we need to parse a chunk string from LLM and extract a list of messages. The received string from our LLM is:

const  = `data: {"type": "agent_message", "data": "Ciao Diego!\n\n"} data:{"type": "agent_message", "data": " come stai??"}\n\n
data: {"type": "agent_message", "data": " my data: example"}\n\n
data: {"type": "agent_message", "data": "aggio, 06.01.24 \u2013 Collisioni a vela aperta</hide>"}`

Here there are 4 messages. Each message is defined by object inside "data".
We need to handle:

• explicit break lines with \n\n

• hidden break lines since message is a template string

• presence of "my data: example" which contains "data:" but this is content of a message, not a new message

We use matchAll() like that:

const regex = /data:\s*(\{[^}]+\})/g
[...chunk.matchAll(regex)]
  .map((match) => match[1])
  .forEach((matchedString) => {
    const obj = JSON.parse(matchedString)
    const { type, data } = obj
  })

And here inside forEach we can read correctly "type" and "data" keys.