Is Using an LLM as the Reasoning Engine a Good Idea? What is a multi-agent concept? How does Waylay utilize subflows to manage what an LLM can and cannot do from a control perspective?

In this video, we explore these questions in depth.

‍

‍

Navigating the World of Multi-Agent Systems with LLM

Until now, we have tackled digitization challenges by converting everything into APIs, enabling every capability to be utilized by RPA tools to replicate human tasks, integrate information, and determine necessary actions. An important question arises: will large language models be sophisticated enough to handle API orchestration independently, serving as the reasoning engine? This is a significant consideration - to say the least. For this to be feasible, we must not only transform every API into an agent (providing LLM with capabilities) but also evaluate whether LLMs alone can effectively manage actions and decisions. This is a crucial question.

‍

In order to illustrate the challenge and how we in Waylay address shortcomings of LLM as a reasoning engine, we prepared a simple video, which illustrates each of these items separately.

‍

Turning API's to LLM agents

In our framework, every API can be transformed into an agent in a matter of minutes. For instance, consider APIs for rolling dice or creating alarms. By defining these APIs as agents, we enable the LLM to manage and interact with them seamlessly. The process involves wrapping the API functionalities into agents, providing the LLM with descriptions of their functions and required inputs.

‍

We kick off with a simple yet illustrative example: a multi-agent game involving dice rolls. The goal is straightforward—roll the dice a few times and, if a guess is correct, trigger an alarm. This example serves as a foundation for understanding how LLMs can be employed to interact with various agents and APIs, effectively turning them into functional components of a broader system.

‍

Testing the Setup

To demonstrate this, we ran a practical test where the LLM was tasked with rolling the dice three times and creating an alarm based on the outcome. Do we trust LLM that will do the right thing? Probably yes, that doesn't sound as too hard a problem to crack for LLM, but we still need to watch out for LLM assumptions in relation to providing inputs to different agents, which if not clearly specified can lead to LLM filling the gaps with defaults that might or might not be desired.

‍

Managing Complex Scenarios

We then tackled a more complex use case: finding a car rental service when a vehicle breaks down. This scenario involved coordinating multiple agents—contract agent, rental agent, and location agent. The LLM’s role was to mediate between these agents to provide a solution. Since this scenario requires not only a simple API search but also contract verification and day rate allowance checks to find the appropriate car, can we fully trust the LLM, knowing it sometimes struggles with basic math problems? Probably not. However, can we mitigate these issues and still leverage the LLM for its strengths? Specifically, using it to call the right agents at the right time and, alongside its reasoning engine, provide a simple interface (bot/text or voice) to guide a user to the correct solution?

‍

Implementing Subflows for Better Control

This example highlighted the necessity of having complex parts of the problem encapsulated as sub-flows, where we can with absolute certainty "control the reasoning process", in order  to avoid potential issues such as incorrect calculations or “hallucinations” by the LLM.

‍

For instance, a location service was used to find the nearest rental service based on a user’s phone number and the exact car location, while a sub-flow efficiently managed complex data interactions. Once the location and the name of the person in need were known, the remaining decisions were handled by a sub-flow presented to the LLM as another agent. Unlike the initial location agent, this sub-flow acted as a "super agent," a composable API flow with logic that the LLM recognized as an additional capability.

‍

Transform Your Decision-Making with Advanced AI: Integrate LLMs for Enhanced Automation Today!

Are you ready to revolutionize your decision-making process with cutting-edge AI?

Contact us today to integrate LLMs into your workflow, enabling effective use of generative AI in automation.

Let us help you streamline operations, enhance accuracy, and unlock new capabilities with advanced AI solutions. Reach out now to explore how we can transform your business!

‍