UI context

The uiContext and context parameters in the act() function allow the Action Transformer to retrieve and understand the context of the current page the user is interacting with. This helps the AI agent consider the text, visual and structural elements displayed on the page and provide more accurate and relevant responses.

Dialog script

async function uiContext(p, c) {
    console.log(c);
    let md = await api.html2md_v2(c);
    console.log('Page in markdown ' + md);
    return md;
}

async function getContext(p) {
    console.log('p.uiContext ' + p.uiContext);
    return p.uiContext || `There's nothing on the UI`;
}

act({
    uiContext: uiContext,
    context: getContext
});

uiContext and context

The uiContext and context parameters work together to provide the Action Transformer with real-time page content. These parameters take functions responsible for retrieving the current UI state and passing this information to the Action Transformer.

How uiContext() works

The uiContext parameter takes a function that asynchronously retrieves the current page context. In the example above, the uiContext() function is used, which takes two parameters:

• p: A predefined Alan AI object containing data about the current dialog session and context information.

• c: The current context of the page, such as the HTML structure or other UI elements.

The uiContext() function processes the page content provided by the c parameter into markdown format using the api.html2md_v2() method. The resulting markdown is then returned and saved to the p.uiContext variable.

How getContext() works

The context parameter takes a function that asynchronously retrieves the value previously stored in p.uiContext. In the example above, the getContext() function is used. This function passes the value of p.uiContext to the Action Transformer, allowing the AI agent to reference the current UI context when generating a response.

If p.uiContext has no data, the function returns a default message such as: There's nothing on the UI.

Example of use

Assume you are building an AI agent that provides contextually aware responses about hypertension. The AI agent needs to use the context of the page a user is currently viewing to adjust its responses. To achieve this:

1. To the dialog script, add a data corpus that provides information about hypertension:

   Dialog script

   corpus({
       title: 'Cleveland Clinic',
       urls: "https://my.clevelandclinic.org/health/diseases/4314-hypertension-high-blood-pressure",
       depth: 1,
       maxPages: 5
   });
   

2. Embed the Alan AI Plugin to the website page:

   1. At the top of the code editor, click Integrations.

   2. To the right of Browser plugin, click Configure.

   3. If Alan AI Plugin is not installed yet, follow the instructions to set it up.

   4. To the field in the Web/Alan Button Extension view, enter the website URL: clevelandclinic.org.

   5. In the top right corner of the view, click Copy Application Code.

   6. In a web browser, go to the Alan AI Browser Plugin page, in the Application Code field, paste the copied code and click Apply.

   7. Open the Hypertension page and make sure the Alan AI Chat is displayed on the page.

   8. In the Alan AI Chat, ask a broad question that requires context, for example: What should I know about it?

      

3. In Alan AI Studio, open Alan AI logs, make sure the Output filter is on and click the chart icon to the right of the AI agent response. In the ACT Script block of the reasoning graph, check the Action Transformer reasoning. To open the reasoning block, double-click it.

   

4. To the dialog script, add the act() function, uiContext() and getContext() functions:

   Dialog script

   async function uiContext(p, c) {
       console.log(c);
       let md = await api.html2md_v2(c);
       console.log('Page in markdown ' + md);
       return md;
   }
   
   async function getContext(p) {
       console.log('p.uiContext ' + p.uiContext);
       return p.uiContext || `There's nothing on the UI`;
   }
   
   act({
       uiContext: uiContext,
       context: getContext
   });
   

5. In the Alan AI Chat, ask the same question: What should I know about it?

   

6. In Alan AI Studio, open Alan AI logs, make sure the Output filter is on and click the chart icon to the right of the AI agent response. In the reasoning graph, check the UI context leveraged for the response and the Action Transformer reasoning using the UI Context and ACT Script blocks.