I have added a new Livebook demonstrating the details of streaming chat completion. It is functionally the same as Deeplearning.AI Order Bot — openai_ex v0.9.21, but using the streaming version of the API.
The sample is at Streaming Orderbot — openai_ex v0.9.21 and can also be deployed as a Livebook app.
I was pleasantly surprised by how easy it was to integrate the streaming api into Livebook, using Kino to create the UI.
Going by my own user experience, I think the streaming API is pretty much a requirement in any production scenario, and I now understand why it’s even there. I’m happy to provide this sample to help others get started with it.
