Visual state

To inform Alan AI about the state of the client app, the app can send the visual state object to the dialog script at any time. The visual state lets you pass the information about the current visual context, for example:

  • What page or screen the user is viewing now

  • What data is currently available to the user

  • What element is selected and so on

The visual state can help you give relevant responses to the user and synchronize the visuals and conversational experience in your app.

To send the visual state object to the dialog script, you must call the setVisualState() method of the Alan AI SDK on the app side. In the method, you pass a JSON object representing the visual state, for example:

Client app
alanBtnInstance.setVisualState({data:"your data"});
Client app
- (void)setVisualState {
  /// Providing any params
  [self.button setVisualState:@{@"data":@"your data"}];
}
Client app
func setVisualState() {
  /// Providing any params
  self.button.setVisualState(["data":"your data"])
}
Client app
fun setVisualState() {
  /// Providing any params
  val params = JSONObject()
  try {
    params.put("data", "your data")
  } catch (e: JSONException) {
    Log.e("AlanButton", e.message)
  }
  alanButton?.setVisualState(params.toString())
}
Client app
void setVisualState() {
  /// Providing any params
  JSONObject params = new JSONObject();
  try {
    params.put("data","your data");
  } catch (JSONException e) {
    Log.e("AlanButton", e.getMessage());
  }
  alanButton.setVisualState(params.toString());
}
Client app
_MyHomePageState() {
  void _setVisualState() {
    /// Providing any params with json
    var visualState = jsonEncode({"data":"your data"});
    AlanVoice.setVisualState(visualState);
  }
}

Ionic Angular

Client app
var myAlanBtn = document.getElementById('myAlanBtn');

myAlanBtn.componentOnReady().then(function () {
  myAlanBtn.setVisualState({data: 'your data'});
});

Ionic React

Client app
const alanBtnComponent = useRef<any>(null);

alanBtnComponent.current.setVisualState({data: 'your data'});

Ionic Vue

Client app
<ion-content :fullscreen="true">
  <alan-button ref="alanBtn" alan-key="YOUR_KEY_FROM_ALAN_STUDIO_HERE" />
</ion-content>

<script lang="ts">
  import {Components} from '@alan-ai/alan-button';

  export default defineComponent({
    mounted : function() {
      const alanBtnInst = this.$refs["alanBtn"] as Components.AlanButton;
      alanBtnInst.setVisualState({data: 'your data'});
    }
  });
</script>
Client app
setVisualState() {
  /// Providing any params with json
  AlanManager.setVisualState({"data":"your data"});
}

On the script side, the sent JSON object is accessible through the p.visual runtime variable. You can use it to:

Note

Mind the following:

  • The visual state object is session-specific: it contains data that apply to a particular user conducting the dialog with the AI agent. The data stored in the visual state object does not persist between dialog sessions.

  • When you save changes to the dialog script or push a newer script version to the Testing or Production environment, the dialog model is rebuilt and the current visual state is reset. To maintain the visual state on script saving, use the scripts event. For details, see onEvent handler.

Differentiating the command logic

You can use the information sent in the visual state object to differentiate the voice and text command logic and play relevant responses to the user.

Let’s assume your app sends the {"screen": "Products"} object when the user opens the products screen and the {"screen": "Checkout"} object when the checkout screen is active. On the dialog script side, this data becomes available in the p.visual.screen runtime variable.

Now we can let Alan AI play different responses depending on the visual state received from the app:

Dialog script
intent('What is it?', 'What screen am I viewing?', p => {
    let screen = p.visual.screen;
    switch (screen) {
        case "Products":
            p.play('This is the Products screen');
            break;
        case "Checkout":
            p.play('This is the Checkout screen');
            break;
        default:
            p.play('(Sorry,|) I have no data about this screen');
    }
});

Note

To test these examples in Alan AI Studio, set the visual state in the Debugging Chat.

Filtering commands

Visual states can be used as conditional filters for voice and text commands. To follow this scenario, add the visual() function to the script. In the function, you need to define either a JSON object or filter that must be matched and pass this filter as a parameter to the necessary command. If the data sent in the visual state matches the filter, the command will be invoked.

Let’s assume your app sends the same JSON object: {"screen": "Products"} or {"screen": "Checkout"}.

Dialog script
const vProductsScreen = visual(state => state.screen === "Products");
const vCheckoutScreen = visual({"screen": "Checkout"});

intent(vProductsScreen, 'What is it?', p => {
    p.play('This is the Products screen');
});

intent(vCheckoutScreen, 'What is it?', p => {
    p.play('This is the Checkout screen');
});

Now, depending on the visual state received from the app, only one of these intents will be invoked and matched at a time, even though they have the same patterns.

As you write your dialog script, you may have several commands using the same visual state. These commands may be scattered across the script.

For your convenience, you can group commands under the same visual filter. This will greatly increase the readability of your script should you need to change anything in it.

Dialog script
const vProductsScreen = visual(state => state.screen === "Products");
const vCheckoutScreen = visual({"screen": "Checkout"});

vProductsScreen(() => {
    intent('What is it?', p => {
        p.play('This is the Products screen');
    });

    intent('What can I do here', p => {
        p.play('On this screen you can choose the products that you want to order');
    });
});

vCheckoutScreen(() => {
    intent('What is it?', p => {
        p.play('This is the Checkout screen');
    });

    intent('What can I do here', p => {
        p.play('On this screen you can complete your order or remove some products');
    });
});