Instructions: To get started, click on a "Load Scenario" button, then click on "Run update" to show the belief state after each dialog turn. You can also modify any of the fields to change the user model parameters, system actions taken, ASR results, and priors -- after making a change, click on "Run update" to show the belief state after each dialog turn.

Sunny-day path: This scenario shows two low-confidence correct recognitions, and illustrates how belief mass accumulates to form a whole-dialog confidence measure. This example considers only 1 N-Best entry, and uses a uniform prior over callees.

Synthesizing N-Best lists: This scenrio shows two low-confidence recognitions where the correct answer appears lower-down on the N-Best list. After the second recognition, the item which appears on both lists has the most belief mass. This illustrates how belief monitoring finds commonality across N-Best lists.

Effects of a strong prior: This scenario illustrates how a prior affects belief monitoring. Here "Jason Wilson" has a strong prior (of 0.5), but the name "Jason Williams" is recognized twice with low confidence. After the first recognition, the two names have about the same belief mass; after the second, "Jason Williams" has much more. This illustrates how priors initially bias the belief state, but how recognition results can eventually swamp this bias.

Effects of a user model: This scenario illustrates how a user model affects the belief monitoring process. Here the system asks for the first name, then the last name. In the first turn, the top ASR result is "Jason Williams" (0.5), and the first name "Jay" (0.1) is second. When asked for the first name only, the user model indicates that the probability of the user saying the first and last names is 0.15, and for saying just the first name is 0.8. The second turn asks for the last name. Again, the ASR indicates that a first and last (this time Jason Wilson) name is most likely (0.5), with a last name (Wilpon) alone as less likely (0.1). The user model indicates that the user is more likely to say just a last name (0.8) than to say both a first and last name (0.15). This example illustrates how the user model in effect re-scales the results from the ASR by indicating the agreement between the user goal (the callee) and each ASR N-Best entry.

Synthesizing across slots: In this scenario, the system asks for the first and last names separately, recognizing "Jason" then "Williams". After the first turn, the belief in all callees with first name "Jason" increase. After the second turn, "Jason Williams" has the most mass, but names containing either "Jason" or "Williams" have more mass than those with neither. This illustrates how information across different slots is combined.

Jason Williams    AT&T Labs - Research    jdw@research.att.com    www.research.att.com/info/jdw