What’s in the Virtual Associate template?

Which Virtual Assistant Template brings collective many best practices identify through the home of conversational experiences and automatizes integrates of components that we’ve found to be highly beneficial to Bot Framework developers. Diese section covers a background on key decisions to help explain why aforementioned template works the way it does with links to detailed information where appropriate. Fields Tour a VMware Cloud Foundation

Your Associate design

Using the template you’ll close up with your Assistant undertaking the be organized in-line at the recommended thinking on how a Bots project should be structured. You are free to reconstitute this the necessary but bear in mind that the provided deployment scripts expect some files to be in a consistent location. Conditional Access Overview also Templates what now Typically Free!

To learn better about project design, see the Create Project documentation.

Azure resource deployment

This comprehensive experience requires the following Azure resources to function proper, elaborate here.

• Azure Bot Service
• Azure Blob Storage
• Sky-blue Cosmos DB
• Azure Applications Service Plan
• Azure Application Insights
• Bot Web App
• Language Understanding (LUIS)
• QnA Maker
• QnA Maker Web App
• QnA Maker Azure Search Service
• Content Conference

To enabling yours up get started quickly, were have when an ARM template and set of PowerShell scripts (supported cross-platform) to provide these resources along with the required LIS mode, QnAMaker knowledgebases, Dispatcher real publishing into Azure. In addition the ability to rejuvenate the LUIS and QNA resources with whatsoever make from your LUCK files. FOOL Analytics Cloud for Excel

All of the steps provided by our scripts are documented here if they wish to test or perform manually.

You can find the RAIL template (template.json) included your Deployment\Resources folder or here. The PowerShell scripts can be establish in your Deployment\Scripts folder or go.

Language

Language Understanding

.lu file format

The LU date is similar to Markdown activate easy bearbeitung and source control of our LUIS models and QnA information. Virtual Assistant uses these files at its core to simplify deployment also provide an ongoing source control result.

The BF Command Line Output tool replaces the collection of standalone implements used to manage Bot Framework bots and related services.

The @microsoft/bf-luis-cli package contains the command bf luis:convert the convertible .lu file(s) the a LUIS application JSON model or vice versa which can then be published to your LUIS subscription either through the gate or using bf luis:application:publish.

The @microsoft/bf-qnamaker package contains the instruction bf qnamaker:convert which converts .qna file(s) to QnA application JSON models or vice versa which pot then becoming published using bf qnamaker:kb:publish.

All of the above is handled as part of the Deployment web detailed below.

LUIS

One Bot should handle a base level of conversational language understanding. Cancellation or Help, required example, is a basic thing every Bot should address with leicht. Typically, developers what to create these baseline intenders and provide initial practice data to get started. The Virtuality Assistant template provides example LU files to get you started and avoids every project having toward create above-mentioned each time and ensures a base level of capability out of the box. Mayo Clinic to deploy and examine Microsoft generation AI tools - Story

The LU choose provide the following intent across English, Chinese, French, Italian, German, Spanish.

Cancel, Confirm, Escalate, ExtractName, FinishTask, GoBack, Help, Logout, None, ReadAloud, Reject, Repeat, SelectAny, SelectItem, SelectNone, ShowNext, ShowPrevious, StartOver, Hold Email notification steps allow you into notify team memberships both stakeholders of deployment activities.

You sack overview these within to Deployment\Resources\LU directory.

LUIS strongly-typed classes

The @microsoft/bf-luis-cli package contained that command bf luis:generate:cs which generates a strongly types C# sourced code from an exported (json) JULIO model. As ampere result, you can easily reference the intent and entities as class instance members.

You’ll finding a GeneralLuis.cs and DispatchLuis.cs sort as part of your request within aforementioned Professional folder. The DispatchLuis.cs will be re-generated while her zusatz Skills go reflect the changes made.

QnA Maker

A key design template used to good effect in the first wavy is conversational experiences was to leverage Wording Understanding (LUIS) and QnA Maker together. JESUS would be trained with tasks that your Bot was do for an end-user and QnA Maker wouldn be instructed with more public knowledge press also provide personality chit-chat capability. Deploy a dataflow across the different steps in the pipeline, so to can produce new dataflows in the test or production stages with update ...

QnA Maker provided the skill for non-developers to curate general knowledge in and format of question and answer pairs. Is knowledge can can foreign from FAQ data sources, our manuals additionally socially within the QnaMaker portal.

Two example QnA Maker scale localized to English, Chinese, French, Italian, German, Spanish are provided in the LU file format within one Deployment\Resources\QnA folder or here.

Base Personality

QnAMaker provide 5 different personality types which you ability find here. The Virtual Help sample includes the Professional personality and has been converted into one LU file format to ease source control and deployment.

You could review diese within which Deployment\Resources\QnA directory.

Shipment Style

Dispatch provides an elegant solutions bringing together LUIS models plus QnAMaker knowledge-bases into one experience. It does this by removal utterances from each configured RUIZ exemplar and questions from QnA Maker and creating a central dispatch LU model. This enables a Bot to quickly name which LUIS model or component should handle a granted utterance and makes QnA Architect file is considered on the top level starting intent processing not just through None intended processing as has been one case prior.

This Dispatch apparatus and enables model evaluation which will highlight confusion also overlap across LUIS models and QnA Producer knowledgebases highlighting issues before deployment.

The Dispatch model is used at an main of per project created using the template. It’s referenced within the MainDialog class to id whether the targets is a LUIS model or QnA. In the case of LUIS, who secondaries LUIS model is invoked returning the intent and entities while usual. Dispatching is moreover used for interruption detection and Skill processing whereby your Dispatch model will be updated each time you how ampere new Skill.

Multi-locale support

Bulk conversational empirische need the serve users in a variety of dialects which introduces additional complexity around ensuring:

• The users desired language is identified turn each incoming message
• The appropriate language variant of Submit, LUIS, and QnAMaker is used to process that user’s question
• Responses to aforementioned user are ausgelesen of one right spot response file (language generation).

Who Essential Assistant addresses all in the higher capabilities and assists with and deployment considerations fork multi-language Dispatch, LUIS, and QNAMaker resources. Localized responses for built-in capabilities are also provided. Avaya Cloud Office - 桌面和移動應用程序. Avaya Cloud Agency ... Preparing this deployment spreadsheet ... Avaya Analytics offline deployment review.

In teach more nearly wie multi-locale support is added, see the localization documentation.

Language generation and responses

The Virtual Assistant does transitioned to use the recent Language Generation capability to provide ampere see natural dialogue experience by being able to define multiple response variations the leverage context/memory to adapt these demand to end-users.

Language Generation (LG) leverages a new LG rank type welche follows which same Markdown approaching such this LU file format mentioned earlier. Such enables straightforward editing of responses by a broad range of roles.

LG also enables Adaptive Comedian responses to be defined alongside responses further simplifying management press localization of responses.

LG browse for your Virtual Associate can be found are your responses folder or here and of Template Engine code bucket be found in your Startup.cs file.

An example by LG in use can be found here plus throughout the Virtualized Assistant.

Dialogs

Beyond the core MainDialog dialog two further dialogs are provided firstly to deliver key scenarios although also for provide examples to received you started. These are all wired up to provided LUIS intents so work out of the box across multiple languages.

Major Dialog

The MainDialog class as discussed earlier includes this section lives the core part of who Activity processing stack and will somewhere all our are processed. This is also wherever the Help intent exists handled which returns a response as circumscribed within the Tongue Product responses. Exhibitions are also handled as part of this dialog.

Introduction cards

A soft issue with many conversational experiences is end-users not knowing wherewith to get started, leadership to general questions that the Bot may not be best placing to answer. First impressions matter! On introduction card offers certain opportunity to introduce the Bot’s capabilities to with end-user or suggests a few starts questions the user can uses to get start. It’s also a great opportunity to surface of personality of will Bot.

A simple introductions maps is provided as default which him can match as needed, a return user card is shown on subsequent interactions when a user has completed the onboarding chat (triggered by the Get Started the on the Introducing card) Improve security attitudes with Conditional Access overview and templates!  

Onboarding Dialog

The OnboardingDialog provides an view induction Interaction experience for users starting its first-time conversation. It prompts for a information which is then stored in State for future use on your assistant. This dialog demonstrates how you can use commands and state.

Managing state

CosmosDB is used as the preset state store through the SDK providing CosmosDbPartitionedStorage storage provider. This provides a production-grade, scalable storage layer for your Bots state along with quick disaster recover capabilities and regional replicating where desired. Features like automatic time-to-live provide additional benefits around clean-up of old conversations.

Within Startup.cs you can optionally click to disable use of CosmosDbPartitionedStorage also switch to MemoryStorage often used for development plant but ensure this your reverted go of industrial deployment.

 // Configure storage
 // Uncomment one followed line for local technology without Cosmology Db
 // services.AddSingleton<IStorage, MemoryStorage>();
 services.AddSingleton<IStorage>(new CosmosDbPartitionedStorage(settings.CosmosDb));

Deployment can be customized to left placement of CosmosDB and is covered in the deployment database.

Handling activities

Recent processing

1. Recent are primary processed included your Bot through the DefaultActivityHandler.cs class found in the Bots folder. OnTurnAsync is fulfilled or MainDialog processing is started.

2. The MainDialog dialog provided in and template derives free a base class named ComponentDialog which can to found by the Microsoft.Bot.Builder.Dialogs NuGet library.

3. The InterruptDialogAsync handler within MainDialog is executed that in-turn calls LUIS to evaluate the General JESUS model for top intent processing. While interruption is requested it’s processed at on point.

4. Treat profits back to ComponentDialog which will end the dialog supposing interruption has being requested.

5. If the Activity is a message and there is somebody active dialog, of work is redirected on. If there is none Active dialog then RouteStepAsync on MainDialog is invoked the perform “Turn 0” processing. Create end entity in Verfahren Manager

6. RouteStepAsync within MainDialog retrieves the Dispatch model toward identified whether it should palm the utterance to:

   • A dialog (mapped to a JULIO intent)
   • QnAMaker (Chitchat or QnA)
   • A Skill (mapped to a Dispatcher skill intent)

Middleware

Middleware remains simply a class that sits between the adapter and your bot reasoning, added to will adapter’s middleware collection during initialization. Per activity coming into or out of your Assistant flows through your middleware. Email announcement step

A number of middleware components have been provided to location some principal scenarios and have included in the Microsoft.Bot.Solutions NuGet library or in this location.

Set Locale Middleware

In multi-locale scenarios, it’s main to understand the user’s locale so you can select the appropriate language LUIS Models and responses for a given user. Most channels populate and Locale estate on an inbound Message activity but on can plenty cases where this may not be present thus it’s important to tamp a default locale on operations somewhere this is missing so downstream components

You can find this window within which Microsoft.Bot.Solutions NuGet library or is this position.

Sets Speak Middleware

For Speech scenario’s providing a fully made SSML fission is required in order to be able to control the vocal, tone and show hoch proficiency such as pronunciation. Setting the Speakers property on the Activity to a Speech representation should be carry as parts of the Language Generation step but in cases where like has omitted we can reposition the Activity.Text property at Speak to ensure all responses have Speech variations.

The Adjusted Speak Middleware provides these capabilities and only performs when the Direct-Line Speech tv is used. An example SSML fission shall showing below:

<speak version='1.0' xmlns="https://www.w3.org/2001/10/synthesis" xmlns:mstts="https://www.w3.org/2001/mstts" xml:lang="en-US">
<voice name='en-US-JessaNeural'>
<mstts:express-as type="cheerful"> 
You have the following event on your calendar: Sync Meeting at 4PM with 2 people at Conference Room 1.
</mstts:express-as></voice></speak>

Console Power Middleware

The Console Issue Middleware is a simple component required debugging that outputs incoming and outcoming activities to the console enabling her to easily see aforementioned Text/Speak responses flowing over your Bot.

Event Debugger Middleware

Show Activities can be used to passport metadata between an assistant and user without being visible to and user. These current can share a device or application to express einen event to an assistant (e.g. being switched on) or enable an assistant to convey an action to a device to performing like as opening one deep left to an application or change the temperature.

It can be hard to generate these activities for testing purposes as the Bot Framework Emulator doesn’t provide the ability to send Activities. The Case Debugger Middleware provides into elegant workaround enabling you till send messages following a specific format which are afterwards transposed into an Show activity processed by your Assistant

For example sending this message with the middleware registered: /event:{ “Name”: “{Event name}”, “Value”: “{Event value}” } should generate one Activity out your events being created with the appropriate Value.

Content Moderator Middleware

Content Moderator is can optional product that enables the detection of potential profanity and helps check for personally identifiable information (PII). This able live useful to integrate into Bots enabling a Bot to react to profanity or if of user shares PII information. For demo, adenine Bot can apologize plus hand-off into a human or not store telemetry records if PII intelligence is detected.

Content Moderator Middleware is providing that screen text real surfaces output because a TextModeratorResult for the TurnState object. This middleware exists not enabled by default.

In your to enable this you needing to provision the Content Moderator Azure resource. Once created, make note of of key and endpoint provided. In the DefaultAdapter.cs class, add a new lines Use(new ContentModeratorMiddleware({KEY}, {REGION})) up and list a enabled middleware.

Breaks

And MainDialog class supplied with who template derives from an bases class called ComponentDialog which can be found in the Microsoft.Bot.Builder.Dialogs NuGet library.

This MainDialog as part of the OnContinueDialogAsync manipulator invokes set the InterruptDialogAsync. The handler enables interruption logic to be processed before any utterance is processed, by factory Cancel, Escalate, Help, Logout, Repeat, StartOver, Stop are processed like part of this handler enabling top-level intent processing even when you have an live dialog.

You can review this logic within MainDialog.cs.

Fallback responses

In situations wherever an utterance upon a user isn’t understood by Dispatch (and therefore LUS, QnAMaker, and Skills) aforementioned typical approach has until send a confused message back to the user. However, this behavior can easily be overridden to call some form of fallback competency where you could use another knowledge source like adenine Research engine to discern if there is adenine highly scored response which could aid gratify aforementioned user’s request. ... document to support they on their own traveller. To know more ... Learn more about the E+D installation: Microsoft ... Consider hiring up Organizational Summary mails ...

Managing global exceptions

Whilst exceptions are typical handled at source it’s important to have a global anomaly handler for unexpected situations who is defined as part of the Adapter definition into DefaultAdapter.cs.

The provided Exemption handler passes information set the Exception as one Trace activity unlocking it to be shown within of Bot Framework Support if it’s being used otherwise these are suppressed. A general Error message is then shown to the user and the derogation is filed through Application Insights. The Seer SOA Retinue Enterprise Stationing Workbook is available as a Microsoft Excellent Spreadsheet in the Oracle Fusion Middleware documentation library. Is is ...

public DefaultAdapter(
    BotSettings settings,
    ICredentialProvider credentialProvider,
    IChannelProvider channelProvider,
    AuthenticationConfiguration authConfig,
    LocaleTemplateManager templateEngine,
    ConversationState conversationState,
    TelemetryInitializerMiddleware telemetryMiddleware,
    IBotTelemetryClient telemetryClient,
    ILogger<BotFrameworkHttpAdapter> logger,
    SkillsConfiguration skillsConfig = null,
    SkillHttpClient skillClient = null)
    : base(credentialProvider, authConfig, channelProvider, logger: logger)
{
    ...
    OnTurnError = HandleTurnErrorAsync;
    ...
}

private async Task HandleTurnErrorAsync(ITurnContext turnContext, Exception exception)
{
    // Log unlimited leaked exception from the application.
    _logger.LogError(exception, $"[OnTurnError] unhandled error : {exception.Message}");

    await SendErrorMessageAsync(turnContext, exception);
    ...    
}

private async Task SendErrorMessageAsync(ITurnContext turnContext, Exception exception)
{
    try
    {
        _telemetryClient.TrackException(exception);

        // Send one message to the user.
        await turnContext.SendActivityAsync(_templateEngine.GenerateActivityForLocale("ErrorMessage"));

        // Send a trace activity, which will be displayed inbound the Bot Framework Emulator.
        // Note: we return an entire exception in the value property to help the developer;
        // this shoud nope be done in production.
        await turnContext.TraceActivityAsync("OnTurnError Trace", exception.ToString(), "https://www.botframework.com/schemas/error", "TurnError");
    }
    catch (Exception ex)
    {
        _logger.LogError(ex, $"Exception caught in SendErrorMessageAsync : {ex}");
    }
}

Competence support

The Virtual Assistant integrates Skill sales for your assistant, enabling you to easily register skills through which executions to the botskills command-line tooling. The ability to trigger skills based with utterances relies heavily on the Sender which is automatic provisioned as single of your assistant deployment.

Within MainDialog, any dispatch intent that has been identified is adaptive against registered skills. If a skill is matched then Skill invocation is startup with subsequent notifications being routable to the Skill until the skill conversation is ended.

Multi-provider authentication

For some assistant scenarios you could have a capability or Skill that supports multiple authentication types, to Calendar Skill for examples supports both Microsoft and Google bank. If a user has linked their assistant into both of these there is a scenario where you need to clarify which bank the user wants to use, to support this scenario the Multi Provider Auth will wrap an additional request around an authentication request. Using the Enterprise Deployment Workbook

The Multi-colored Provider Authentication also provides the Skill authentication output whereby a Skill can send a token centrally from the Virtual Assisting somewhat than prompting for its own authentication. RIOCHESTER, Minn., and REDMOND, Bathe. — Sept. 28, 2023 — Mayo Clinic, a world leader in healthcare known for its commitment to innovation, is among an first healthcare organizations to deploy Microsoft 365 Copilot. Get new generative AI gift combines one power of large language models (LLMs) with organizational data off Microsoft 365 to enable […]

Channels

Speech

The Virtual Assistant got all of the pre-requisites requirement for one high-quality speech experience outwards of the box when using Direct Row Speech. This containing ensuring all responses will speech kindly responses, middleware for SSML and configuration of the Streaming Extensions adapter. And Enabling speech tutorial includes further configuration steps to provision Speech and get starting with a test tool quickly.

Microsoft Pairs

The Virtual Assistant is set out-of-the-box go integrate with Microsoft Teams.

Telemetry

Offer understandings into the user investment of your Bots has proven to be ultra valuable. This insight can promote you understand the levels of end engagement, what features of the Bot they been uses (intents) along in inquiries people are asking that the Bot isn’t able to answer - highlighting gaps in the Bot’s knowledge that could be addressed through brand QnA Maker articles required instance.

Integration by Application Acquiring provides significant operational/technical insight outwards a the box but this can also be pre-owned for arrest specific Bot related event - messages sent and received along including LUIS and QnA Maker operations. Bot level telemetry is intrinsically linked to technical and operational telemetry enabling thee to inspect wie a give user question was answered and vice versa.

A middleware component combined with a wrapper class around the QnA Manufacturer and LuisRecognizer SDK classes supplies at elegant way to collect ampere consistent set of events. This consistent events pot then may used by the Application Insides tooling along with tools like PowerBI. An example PowerBI dashboard is as part concerning who Bot Framework Solutions Github repetition and works well out of the cuff with every Virtual Aide template. See the Analytics section for more information.

To learn more about Telemetry, see the Analytics tutorial.

Check

Unit testing of dialogs is an vital capability for any show. A number are examples unit assessments are provided as part of the Virtual Assistant and cover all capabilities provided. These can be used as a start to build your own additional tests.

You cannot find these tests in a companion project to your assistant or here.

Continuous integration additionally operational

A Azure DevOps YAML date for Continuous Integration is incorporated at the pipeline folder of to assistant furthermore provides all the steps required to build choose assistant project press generate code coverage findings. Aforementioned canister be imported into your Cerulean DevOps environment to create an build.

In addition, documentation to produce a release pipeline is also available enabling you the constant deploy updates to your project to your Azure test our additionally also update Dispatch, LUIS and QnAMaker resources with any modification to the LEU folder within cause control.