In the previous post we have learned that Semantic Kernel 1.0 has added support for a feature that OpenAI has introduced in their most recent models, called function calling. This feature has made the Semantic Kernel planner outdated for many scenarios. Function calling, in fact, serves the same purpose, which is enabling the LLM to figure out automatically which functions are needed to perform a task, but it does it in a more efficient way. The approach used by the planner is called ReAct, which means that AI is going to call a function, evaluate the response and then call another function if needed. With the planner, all the steps required back and forth communication between the LLM and the code, using lot of tokens (which means worse performance and a more expensive bill). Function calling, instead, is baked into the model, which means that you can skip the back and forth communication completely, since the model is able to perform this type of reasoning on its own.

For this reason, for many scenarios, you don’t need a planner anymore: the function calling capabilities we have highlighted in the previous post are capable to managing them. However, there might be scenarios in which these capabilities aren’t good enough for the task you’re trying to perform, since you need to apply a more complex reasoning. If you observe a scenario in which function calling isn’t leading to the outcome you’re expecting, there’s are two new tools in Semantic Kernel 1.0 that you can use: the Function Calling Stepwise planner and the Handlebars planner.

Introducing the Function Calling Stepwise planner

The Function Calling Stepwise planner is built on top of calling functions, so it uses the same approach under the hood. However, compared to pure function calling, it’s able to reach the LLM to perform additional reasoning when it comes to generating the plan, so that it can improve the reliability of identifying the right functions to call. The first step to use this new planner is to install the dedicated NuGet package, which is Microsoft.SemanticKernel.Planners.OpenAI.

Let’s setup now the project in a similar way we did in the previous post. The goal is to get the body of a mail to share the information about the population number of the United States in 2015, split by the number of people who identify themselves as male or female. As you can see, this is a task that requires using some of the plugins we have built in the previous posts: the WriteBusinessMail prompt function and the UnitedStatesPlugin native class. Here is the initialization code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

string apiKey = "configuration";
string deploymentName = "AzureOpenAI:DeploymentName";
string endpoint = "AzureOpenAI:Endpoint";

var kernel = Kernel.CreateBuilder()
    .AddAzureOpenAIChatCompletion(deploymentName, endpoint, apiKey)
    .Build();

kernel.ImportPluginFromType<UnitedStatesPlugin>();

var pluginsDirectory = Path.Combine(Directory.GetCurrentDirectory(), "Plugins", "MailPlugin");
kernel.ImportPluginFromPromptDirectory(pluginsDirectory, "MailPlugin");

Now we need to add, like we have seen in the post about using OpenAI plugins, a #pragma directive: this new planner is marked as experimental, so we have to suppress the warning, otherwise we won’t be able to compile our code:

1
2
3
4
5

#pragma warning disable SKEXP0061 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.

//code goes here

#pragma warning restore SKEXP0061 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.

Using this new planner is quite straightforward, as you can see in the following example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

#pragma warning disable SKEXP0061 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
var planner = new FunctionCallingStepwisePlanner();

var ask = @"Write a business mail to share the population of the United States in 2015. 
Make sure to specify how many people, among the population, identify themselves as male and female.
Don't share approximations, please share the exact numbers.";

var result = await planner.ExecuteAsync(kernel, ask);

Console.WriteLine(result.FinalAnswer);
Console.ReadLine();

#pragma warning restore SKEXP0061 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.

We create a new FunctionCallingStepwisePlanner object, then we call the ExecuteAsync() method passing as parameter our kernel object and the prompt which describes the task we want to perform. We directly get back the outcome of the task once the LLM has completed the orchestration process, inside the FinalAnswer property.

If we run the code, we will get a result similar to the following one:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

Subject: 2015 United States Population Data Request

Dear [Recipient Name],

I trust this email finds you well.

I am reaching out to deliver the requested population statistics for the United States for the year 2015. Below are the exact details:

- Total population: 316,515,021
- Male population: 155,728,568
- Female population: 160,786,456

These numbers reflect the self-identified gender count as of that year.

Should you require any additional information or have further inquiries, please do not hesitate to contact me. I am at your service to assist.

Warm regards,

AI Assistant

If we want to understand in more details what happened, the result object includes a property called ChatHistory, which includes the whole conversation between the LLM and the user (in this case, the application):

As you can see from the image, the history contains the whole chain of functions that was called by the planner: the two native ones (GetPopulation and GetPopulationByGender, which was called two times) and the prompt one, WriteBusinessMail, to generate the text of the business mail.

The Handlebars planner

Function callings are very powerful for most scenarios, but there are still some advantages in using a planner:

• You can generate the plan ahead of the execution, giving you the chance to evaluate it.
• If you get a good plan, you can save it and reuse it, without having to regenerate it every time.
• The plan is generated with a single LLM call, helping to save tokens.

These were the main features that led the Semantic Kernel team to build tools like the Sequential planner. However, there was a catch. The plans were generated using a custom XML syntax, which is challenging for the LLM to understand sometimes, leading to the generation of wrong plans. As the team has shared in a blog post, however, researches have demonstrated that LLMs performs much better when they are asked to code in a language they are trained on. As such, the team has decided to switch from XML to Handlebars, which is a template language originally built for JavaScript, but which has been ported to many other languages, including C#. Thanks to this language, you can easily define a template and then, at runtime, replace the various placeholders with real values. As a template language, additionally, it supports also features that, otherwise, would require a full programming language, like conditions and iterators. For example, let’s say that you need to render a list of items in HTML. With a Handlebars template, you can write something like this:

1
2
3
4
5

<ul class="people_list">
  {{#each people}}
    <li>{{this}}</li>
  {{/each}}
</ul>

The {{#each}} and {{/each}} are the iterators, which are used to iterate over the list of people and render the list items. The {{this}} is the placeholder, which is replaced with the value of the current item in the list. By providing in input a people collection like the following one:

1
2
3
4
5
6
7

{
  "people": [
    "John",
    "Mary",
    "Peter"
  ]
}

The output will be the following one:

1
2
3
4
5

<ul class="people_list">
    <li>John</li>
    <li>Mary</li>
    <li>Peter</li>
</ul>

In the context of Semantic Kernel, a plan written with Handlebars gives the ability to the LLM to use this powerful features in the generated plan, making more easily to manage conditions, loops and other complex scenarios.

Now that we have understood what is Handlebars, let’s see how we can use the Handlebars planner in our Semantic Kernel applications. First, like we did for the Function Calling Stepwise planner, we need to install the dedicated NuGet package, called Microsoft.SemanticKernel.Planners.Handlebars. Also in this case, we need to add a specific #pragma directive to suppress the warning about the experimental nature of the planner:

1
2
3

#pragma warning disable SKEXP0060 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
// code
#pragma warning restore SKEXP0060 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.

Now we can generate and execute the plan in the following way:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

#pragma warning disable SKEXP0060 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
var planner = new HandlebarsPlanner();

var ask = "Write a mail to share the number of the United States population in 2015 for a research program.";

HandlebarsPlan plan = await planner.CreatePlanAsync(kernel, ask);

var originalPlanResult = await plan.InvokeAsync(kernel);

Console.WriteLine(originalPlanResult);

#pragma warning restore SKEXP0060 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.

First, we create a new HandlebarsPlanner() object, then we use the CreatePlanAsync() method to create the plan, providing as inputs the kernel and the prompt with the task we want to achieve. Once the plan is generated, we can execute it by calling the InvokeAsync() method, passing as input again the kernel. We get directly back the result of the plan, like the following one:

1
2
3
4
5
6
7
8
9

Dear Sir/Madam,

I would like to bring to your attention that the population of the United States in the year 2015 was recorded to be approximately 316,515,021 individuals. Please take this figure into account for your records.

Should you need any further clarification or additional information, please do not hesitate to contact me.

Best regards,

AI Assistant

If we print on the terminal console the plan, we can see the Handlebars language in action which highlights the usage of the GetPopulation function from the UnitedStatesPlugin and the WriteBusinessMail function from the MailPlugin:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

{{!-- Step 1: Set the year for which the data is needed --}}
{{set "year" "2015"}}

{{!-- Step 2: Use the custom helper to get the population data for the year --}}
{{set "populationData" (UnitedStatesPlugin-GetPopulation year=year)}}

{{!-- Step 3: Format the data and message for the body of the mail --}}
{{set "mailContent" (concat "The United States population in " year " was " (json populationData.TotalNumber) " people.")}}

{{!-- Step 4: Use the custom helper to generate the business mail with the formatted content --}}
{{set "businessMail" (MailPlugin-WriteBusinessMail input=mailContent)}}

{{!-- Step 5: Output the business mail --}}
{{json businessMail}}

As you can see, the plan is just text content, so we can easily store it in a text file and reload it for later usages. The following code shows an improved version of the previous one:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

#pragma warning disable SKEXP0060 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
var planner = new HandlebarsPlanner();

var ask = "Write a mail to share the number of the United States population in 2015 for a research program.";

HandlebarsPlan plan;

if (!File.Exists("plan.txt"))
{
    // Create the plan
    plan = await planner.CreatePlanAsync(kernel, ask);

    var serializedPlan = plan.ToString();
    await File.WriteAllTextAsync("plan.txt", serializedPlan);
}
else
{
    string serializedPlan = await File.ReadAllTextAsync("plan.txt");
    plan = new HandlebarsPlan(serializedPlan);
}

// Execute the plan
var originalPlanResult = await plan.InvokeAsync(kernel, []);
Console.WriteLine(originalPlanResult);
#pragma warning restore SKEXP0060 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.

We store the plan in a file called plan.txt. If the file doesn’t exist, it means we have to generate one in the same way we have seen before, by calling the CreatePlanAsync(). Then, once the plan has been generated, we call the ToString() extension method to get the serialized version of the plan and we store it in the file by calling the static method File.WriteAllTextAsync(). If the file exists, instead, we read the content of the file and we use it to create a new HandlebarsPlan object, passing it to the initializer. Then, independently by the way we acquired the plan, we execute it by calling the InvokeAsync() method.

If you run this code twice, you will notice that the second time you will get the result back much faster. This because Semantic Kernel had to call the LLM just to process the plan and get the outcome of the task we have asked, but not to create it.

Wrapping up

In this post, we have learned that, even if the function calling feature provided by OpenAI is able to take care of most of the scenarios, there are still some cases in which you might need to use a planner. Semantic Kernel 1.0 has introduced two new planners, which are the Function Calling Stepwise planner and the Handlebars planner. The first one is built on top of the function calling feature, but it’s able to reach the LLM to perform additional reasoning when it comes to generating the plan, so that it can improve the reliability of identifying the right functions to call. The second one, instead, is a new version of the Handlebars planner, which is now built on top of the Handlebars language, which is a template language that makes it easier to generate plans that are easier to understand by the LLM and to store for later usages.

You can find all the samples for these new scenarios in the usual repository on GitHub.

Happy coding!