DEALING WITH PHRASE LEVEL CO-ARTICULATION (PLC) IN SPEECH RECOGNITION A FIRST APPROACH

DEALING WITH PHRASE LEVEL CO-ARTICULATION (PLC)IN SPEECH RECOGNITION: A FIRST APPROACH

Roeland J. F. Ordelman#, Arjan J. van Hessen#, David A. van Leeuwen*# University of Twente, Enschede, The Netherlands* TNO - Human Factors Research Institute, Soesterberg, The Netherlands

ABSTRACTWhereas nowadays within-word co-articulationeffects are usually sufficiently dealt with inautomatic speech recognition, this is not always thecase with phrase level co-articulation effects (PLC).This paper describes a first approach in dealingwith phrase level co-articulation by applying theserules on the reference transcripts used for trainingour recogniser and by adding a set of temporaryPLC phones that later on will be mapped on theoriginal phones. In fact we temporarily break downacoustic context into a general and a PLC context.With this method, more robust models could betrained because phones that are confused due toPLC effects like for example /v/-/f/ and /z/-/s/,receive their own models. A first attempt to applythis method is described.

1. INTRODUCTIONThe DRUID1 project (Document Retrieval UsingIntelligent Disclosure), a collaboration ofCTIT2/University of Twente, TNO3 and CWI4, aimsat the development of tools for the indexing ofmultimedia content. For the Spoken DocumentRetrieval (SDR) part of this project, we useABBOT, the hybrid connectionist-hidden Markovmodel large vocabulary speech recognition system[1,2] developed for English by CambridgeUniversity, Sheffield University and SoftSound.TNO already participates in the annual EnglishTREC SDR tracks with this system [3], but sincethe DRUID project focuses on Dutch SDR, we arecurrently developing a Dutch version of ABBOT.ABBOT uses a recurrent neural net (RNN) foracoustic modelling and a Markov process forlanguage modelling. Since the RNN is able tocapture temporal acoustic context, very goodrecognition results can be achieved using context-independent phone models. Although languagemodelling often makes it possible to transform sets 1http://www.seti.cs.utwente.nl/Parlevink/Projects/druid.html2 Centre for Telematics and Information Technology3 Institute of Applied Physics, departements MultimediaTechnology (Soesterberg) and Human Factors (Delft)4 Centre for Mathematics and Computer Science,Amsterdamof erroneously recognised phones into well-recognised words, better phone recognitionundoubtedly leads to better word recognition.Our first target was training the phone modelsin a baseline training, which eventually performed a33.3% Phone Error Rate (PER) on the test data.Next steps should involve improving acousticmodelling and starting language model training inorder to be able to do word recognition. Followingon a more detailed description of our methods toimprove acoustic modelling in the next sections,this paper reflects our first attempt of improvingacoustic modelling by applying phrase level co-articulation rules on the reference transcripts usedfor training the phone models.2. ACOUSTIC MODEL2.1. Acoustic Training DataThe baseline training material consisted of about 7hours of speech material of 52 (26 male - 26female) speakers reading 66 sentences from anewspaper text database, recorded in a noise freeroom (TNO-NRC-0 database). PLP feature vectors(12th order cepstral coefficients derived usingperceptual linear prediction and log energy) werepresented at the input of the RNN that contained256 state units. Our phoneset consisted of 44context-independent phones plus silence.Obviously, we need far more and also differenttypes of training data to build robust phone modelsfor speaker independent continuous speechrecognition in typical SDR tasks, but it is quite aneffort to collect large annotated speech corpora forDutch. Currently we are collecting and annotatingspeech material from Dutch radio shows andrecordings of sessions of parliament.2.2. AnnotationsFrom some of the speech material we arecollecting, text auto cues (text to read fornewsreader) or annotations (recording andannotation is in special cases a statutoryrequirement) are available that could reduce at leastsome of the hard labour. More important, it canprovide additional context specific training data forlanguage modelling. Also, CEEFAX documents ofthe recorded news broadcastings are collected in