[...]... number of barge-in attempts from the user in a dialogue average number of cancel attempts from the user in a dialogue average number of help requests from the user in a dialogue average number of system error messages in a dialogue average number of system questions in a dialogue average number of system turns in a dialogue average number of system words uttered in a dialogue average number of time-out... prediction These systems are called spoken dialogue systems (SDSs), in some literature also spoken language dialogue systems (SLDSs) They have to be differentiated from systems with more restricted capabilities, e.g command systems or systems accepting only dialling tones as an input A categorization of interactive speech systems will be given in Section 2 .1. 3 Most of the currently available systems enable... prompts in a dialogue average number of turns in a dialogue average number of user questions in a dialogue average number of user turns in a dialogue average number of user words uttered in a dialogue average number of in-vocabulary user words in a dialogue average number of words uttered in a dialogue false speaker rejection rate scaling factor for expectation factor coefficients for number of correct... taxonomy of quality aspects developed in Chapter 2 proved to form a solid basis for quality modelling approaches which aim at being generic, and applicable to a variety of other systems The interaction scenario which is addressed here is of course limited However, it is expected that the structured approach to quality of telephone-based spoken dialogue systems can be transferred to other types of systems. .. successfully been used for the definition of design guidelines for spoken dialogue systems The guidelines encompass general principles of cooperative behavior in HMI, and form one aspect of interaction quality A more general picture of interaction quality and of the quality of services offered via SDSs is presented in Section 2.3 A new taxonomy is developed which allows quality aspects to be classified, and... for carrying out subjective1 interaction experiments with human users in order to determine the quality of the developed systems, and the resulting satisfaction of their users As a wide range of novice users is the target group of current state -of- theart systems and services, the need for subjective assessment and evaluation of quality is increasing (Hone and Graham, 20 01, p 2083): “In the past speech... designers of spoken dialogue systems In this chapter, an attempt is made to close the gap The starting point is a description of communication scenarios in which a human user interacts with a spoken dialogue system over some type of speech transmission network, see Section 2 .1 It takes into account the source and the sink of information, as well as the transmission channel Different types of networks... a study of 13 0 information offices 2 in six countries (Billi and Lamel, 19 97), over 10 0 million calls were handled per year, with at least another 10 million calls remaining unanswered About 91% of the callers solely asked for information, and only 9% performed a reservation task It was estimated that over 90% of the calls could be handled by an automatic system with a recognition capability of 400... individual elements of a spoken dialogue system will be addressed first individually, and commonly used methods and developments will be pointed out However, the quality of the whole system and of the service visible (audible) to the user will not be just a sum of the individual components It is therefore necessary to quantify the contribution of the individual components to the quality perception of the whole... the developers of spoken dialogue systems, for transmission network operators, and of course for the end users Interactive speech systems are “computer systems with which humans interact on a turn-by-turn basis” (Fraser, 19 97, p 564) They enable and support the communication of information between two parties (agents), mostly between a human user and a machine agent Here, only those systems will be . Experiment Summary 93 97 97 10 1 10 2 10 3 10 3 10 3 10 6 10 8 11 4 11 9 12 1 12 7 12 8 13 1 13 3 13 4 14 0 14 7 15 7 15 9 16 2 16 5 16 9 17 1 17 5 17 9 18 1 18 3 18 4 18 6 18 8 19 0 19 5 19 8 Contents 5. QUALITY OF SYNTHESIZED SPEECH. Evaluation 2.5 Summary ix xi xiii 1 9 11 13 17 18 38 41 43 47 51 59 63 63 66 68 77 79 86 90 vi 3. ASSESSMENT AND EVALUATION METHODS 3 .1 Characterization 3 .1. 1 3 .1. 2 3 .1. 3 3 .1. 4 3 .1. 5 Agent Factors Task. Experiment 5.5 Summary 6. QUALITY OF SPOKEN DIALOGUE SYSTEMS 6 .1 Experimental Set-Up 6 .1. 1 6 .1. 2 6 .1. 3 6 .1. 4 6 .1. 5 The BoRIS Restaurant Information System Speech Recognition Simulation Measurement of Interaction