Updated on 17th November 2023 to reflect the latest changes in the Semantic Kernel APIs:

• All the Azure OpenAI extension methods have been renamed from Azure to AzureOpenAI. For example, WithAzureChatCompletionService() is now WithAzureOpenAIChatCompletionService().
• The method to import an OpenAI plugin into the kernel has been renamed from ImportPluginFunctionsAsync() to ImportOpenAIPluginFunctionsAsync(). Please be aware that there’s also a new method to import directly an OpenAPI definition called ImportOpenApiPluginFunctionsAsync(). As you can see, the two names are very similar, so make sure to use the right one.

After reading the previous posts, especially the ones about native plugins and OpenAI plugins, you probably have started to grasp the value of plugins, but in the end we could have achieved the same goal by directly calling a REST API. So why bother with plugins? In this post, we’re going to explore the concept of orchestration and how Semantic Kernel can help you to automatically manage plugins through a tool called planner. Thanks to the planner, you’ll be able to load multiple plugins, define a task and let Semantic Kernel figure out which are the right plugins to call and in which order to satisfy the ask.

Manual orchestration

First, let’s see a simpler example to understand the Semantic Kernel capabilities to manage orchestration. In all the examples we have seen so far, we have used the kernel to execute a single function, using the RunAsync() method. However, the same method can be used to invoke multiple functions and automatically chain the output together.

Let’s say we want to generate a mail to share the number of the US population given a specific year. We can combine the semantic function and the native function (or the OpenAI plugin) we have built to achieve this goal. Assuming that you have already created them, following the guidance of the previous posts, the first step is to load both of them in the kernel:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

string apiKey = "api-key"
string deploymentName = "deployment-name";
string endpoint = "endpoint";

var kernelBuilder = new KernelBuilder();
kernelBuilder.
    WithAzureOpenAIChatCompletionService(deploymentName, endpoint, apiKey);

var kernel = kernelBuilder.Build();

const string pluginManifestUrl = "https://localhost:7071/api/.well-known/ai-plugin.json";
await kernel.ImportOpenApiPluginFunctionsAsync("UnitedStatesPlugin", new Uri(pluginManifestUrl));

var pluginsDirectory = Path.Combine(System.IO.Directory.GetCurrentDirectory(), "Plugins");
kernel.ImportSemanticFunctionsFromDirectory(pluginsDirectory, "MailPlugin");

The previous code is a mix of the code we have seen in different posts. The first part is the code to load the OpenAI plugin using the ImportOpenApiPluginFunctionsAsync() method, while the second part loads the MailPlugin using the ImportSemanticFunctionsFromDirectory() method. Now we can get a reference to the two functions we need to use:

1
2

var mailFunction = kernel.Functions.GetFunction("MailPlugin", "WriteBusinessMail");
var populationFunction = kernel.Functions.GetFunction("UnitedStatesPlugin", "GetPopulation");

The first one is WriteBusinessMail, which is the semantic function included in the MailPlugin. The second one is the GetPopulation function, which is the native function included in the UnitedStatesPlugin. Now we can use the RunAsync() method to invoke both functions and chain the output together:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

ContextVariables variables = new ContextVariables
{
    { "year", "2018" }
};

var result = await kernel.RunAsync(
    variables,
    populationFunction,
    mailFunction
);

Console.WriteLine(result.GetValue<string>());
Console.ReadLine();

We need to supply only the input for the first function. Automatically, the kernel will take the output of the first function and use it as input for the second function. The result of the second function will be the result of the whole chain. In our scenario, we execute first the GetPopulation function to retrieve the number of the US population in a given year, so we use the ContextVariables collection to provide the year. The information about the US population in 2018 will be then passed to the WriteBusinessMail function to generate a mail. In this case, the output will be similar to the following mail:

1
2
3
4
5
6
7
8
9

Dear [Recipient],

I am writing to inform you that the population number in the United States in 2018 was 322,903,030. I hope this information is useful to you.

If you have any further questions or concerns, please do not hesitate to contact me.

Best regards,

AI Assistant

Pretty cool, right? However, if the outcome is quite impressive, the way we got there is less exciting. First of all, we need to know in advance which functions we need to call and in which order. This is not a big deal in our example, but it can be a problem in more complex scenarios. Moreover, we need to know the input and output of each function, so we can chain them together. If, for example, we would have passed the functions in a reverse order to the RunAsync() method, Semantic Kernel wouldn’t have been able to produce a meaningful result, since the output of the WriteBusinessMail function (a mail text) is not a valid input for the GetPopulation function (which expects, instead, a year).

Let’s see now how we can improve this scenario using the planner.

Automatic orchestration with the planner

Thanks to the planner component of Semantic Kernel, we can start from the same premise (we want to generate a mail to share the number of the US population given a specific year) but, this time, we’ll let the kernel figure out which functions to call and in which order. The first step is the same as before: we need to load inside the kernel all our plugins, so you can reference the code we have already used.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

string apiKey = "api-key"
string deploymentName = "deployment-name";
string endpoint = "endpoint";

var kernelBuilder = new KernelBuilder();
kernelBuilder.
    WithAzureOpenAIChatCompletionService(deploymentName, endpoint, apiKey);

var kernel = kernelBuilder.Build();

const string pluginManifestUrl = "https://localhost:7071/api/.well-known/ai-plugin.json";
await kernel.ImportOpenApiPluginFunctionsAsync("UnitedStatesPlugin", new Uri(pluginManifestUrl));

var pluginsDirectory = Path.Combine(System.IO.Directory.GetCurrentDirectory(), "Plugins");
kernel.ImportSemanticFunctionsFromDirectory(pluginsDirectory, "MailPlugin");

What changes is that, this time, we don’t need to get a reference to the functions we want to call, but we just need to define the ask we want to satisfy and run it through the planner:

1
2
3
4
5
6
7
8

var planner = new StepwisePlanner(kernel);

var ask = "Write a mail to share the number of the United States population in 2015 for a research program.";
var originalPlan = planner.CreatePlan(ask);
var originalPlanResult = await kernel.RunAsync(originalPlan);

Console.WriteLine(originalPlanResult.GetValue<string>());
Console.ReadLine();

There are different type of planners available in Semantic Kernel. For our scenario, we’re going to use the StepwisePlanner, which evaluates the sequence to perform step by step. After we have defined the ask (which is a simple prompt that explains the goal to achieve), we create a plan using the CreatePlan() method. The plan is a sequence of functions that the planner has identified as the best way to satisfy the ask. Once we have a plan, we can execute it by passing it to the usual RunAsync() method of the kernel.

This is the result we’re going to get:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

The email text to share the number of the United States population in 2015 for a research program is:

Subject: Sharing Latest Data on United States Population

Dear Research Program,

I hope this email finds you well. I am writing to share with you the latest data on the United States population in 2015. As per the recent statistics, the population was 316,515,021.

I believe this information will be useful for your ongoing research program. If you require any further assistance or information, please do not hesitate to contact me.

Thank you for your time and consideration.

Best regards,

AI Assistant
    

As you can see, without having to specify which plugins and functions we wanted to use, Semantic Kernel has been able to satisfy our ask: writing a mail to share the number of the US population in 2015. From the ask, the planner has been able to automatically figure out:

• Which plugins and functions to use (GetPopulation and WriteBusinessMail)
• The order to follow (GetPopulation first, then WriteBusinessMail)
• Which is the input to provide to each function (it automatically extracted from our ask the year 2015 and used it as input for the GetPopulation function)

The interesting part is that, by analyzing the plan, we can see what happened behind the scenes. If you put a breakpoint after the result has been generated, we can explore the content of the originalPlanResult variable and see, inside the Metadata collection, the thought process implemented by Semantic Kernel:

We can see that the plan is made by three steps and it’s going to use two different functions: GetPopulation and WriteBusinessMail. The most interesting information, however, can be found in the stepsTaken property:

As you can see, the property includes a detailed description of the reasoning made by the planner to achieve our ask:

1

To write a mail, we can use the MailPlugin.WriteBusinessMail function to generate the text of the email. To get the population of the United States in 2015, we can use the UnitedStatesPlugin.GetPopulation function with the year parameter set to 2015.

This is the same reasoning we have done manually in the previous section, but this time it has been done automatically by Semantic Kernel with the help of the reasoning capabilities offered by LLMs.

Testing a plan with Visual Studio Code

The Visual Studio Code extensions we have learned about in the post dedicated to semantic functions can be used also to test a plan. This way, we can evaluate if the plugins we have built are enough to satisfy our ask before starting to write any code. Before we see how to do that, it’s important to highlight that this feature has an important limitation: it works only with semantic functions. As such, we wouldn’t have been able to test the plan we have created in this post, since it includes a native plugin.

As such, for the purpose of this demo, I’ve created a second plugin, called AskPlugin, which includes a semantic function called Ask, which defines a prompt to simply ask to the LLM to answer a question for us:

1
2
3

Answer the question below in a concise way.

{{$input}}

Now we have two semantic functions we can use to test a plan. Once you open the Semantic Kernel extension in Visual Studio Code, expand the Plan section and click on the button to create a new plan:

As first thing, you will be asked for a folder where to save the plan, which is stored with a series of JSON and text files. As next step, the extension will identify all the semantic functions available in the plugins you have included in the folder, so that you can pick up the ones you want to include in the plan:

For my scenario, I have picked them both.

The next step is to give a name to the plan. Finally, you will be asked to provide a goal for the plan, which is the equivalent of the ask we have seen in the previous section. For my test, I provided the following goal:

1

Answer the question "Which is the distance between Earth and Sun?" and share it via mail

Once the plan is created, you can run it by pressing the Execute plan button:

During the execution, you will see Visual Studio Code picking up the needed plugins and resolving the plan step by step. At the end, you will see the result of the plan:

You can see how the planner has used, first, the Ask function to answer the question “Which is the distance between Earth and Sun?” and then it used it as input for the WriteBusinessMail function to generate the mail text. We can see that our plan and our plugins are working as expected, without writing a single line of code.

Wrapping up

In this post, we have unlocked the full value of Semantic Kernel. Previously, we have started to hint at the value of Semantic Kernel and using plugins, but it’s only with the planner that we can realize its full potential. Even in scenarios in which we have complex workflows to manage, we don’t have to worry about picking the right functions and choosing the right order, or chaining the operations in the correct way. Semantic Kernel will do all the heavy lifting for us. This post included a simple scenario, in which we loaded two plugins and we used both of them to satisfy the ask. However, in a real application, we could have loaded dozens of plugins, and the planner would have picked up only the ones which are really needed to complete the requested task.

You can find the demo used in this post in the GitHub repository. Specifically,

• The SemanticKernel.Orchestration project contains the manual orchestration example.
• The SemanticKernel.Planner project contains the planner example.

Happy coding!