Speech and Speaker Recognition Evaluation -- Evaluation of Speech Synthesis -- Modelling and Evaluating Verbal and Non-Verbal Communication in Talking Animated Interface Agents -- Evaluating Part-of-Speech Tagging and Parsing Patrick Paroubek -- General Principles of User-Oriented Evaluation Margaret King -- An Overview of Evaluation Methods in TREC Ad Hoc Information Retrieval and TREC Question Answering -- Spoken Dialogue Systems Evaluation -- Linguistic Resources, Development, and Evaluation of Text and Speech Systems -- Towards International Standards for Language Resources Nancy Ide and Laurent Romary.

In its nine chapters, this book provides an overview of the state-of-the-art and best practice in several sub-fields of evaluation of text and speech systems and components. The evaluation aspects covered include speech and speaker recognition, speech synthesis, animated talking agents, part-of-speech tagging, parsing, and natural language software like machine translation, information retrieval, question answering, spoken dialogue systems, data resources, and annotation schemes. With its broad coverage and original contributions this book is unique in the field of evaluation of speech and language technology. This book is of particular relevance to advanced undergraduate students, PhD students, academic and industrial researchers, and practitioners.