There is an enormous interest from the ecosystem to build LLM applications. But it is not yet clear the best practices to build those applications in cloud native environments. We will be presenting lessons learnt from real world use cases.
LLMs like ChatGPT are analogous to brains without memory, senses, or hands. APIs and plugins allow developers to add these key functionalities to LLMs and enable LLM applications. The popularity of OpenAI plugins and function templates has shown that there is a great demand for such services.
However, current LLM plugins are often built upon complex cloud native infrastructure designed for fully fledged services. The infrastructure is too heavyweight for plugins/serverless applications that mostly act as the lightweight links between LLMs and external services. Wasm has emerged as a compelling lightweight, portable, and secure runtime for this type of functions.