Searching in spoken words. Disclosure of recorded content in MediaMosa.

1. Arjan van Hessen Speech & Language TechnologyA.J.vanHessen@ewi.utwente.nl Searching in spoken wordsDisclosure of recorded content in MediaMosa SURFnetRelatiedagen 2010Noordwijkerhout December 9, 2010

2. Content Introduction Why speech is so important What is HLT? Working applications: Self-service (Internet & Telephony) Searching in recorded audiovisual recordings Demonstrations

3. Humans as speaking creatures The start of the human speech started some 100.000 years ago. Before, the shape of the vocal track was not “ready” for the modern speech. The larynx was situated too high, something you can see with chimps.

4. Humans as writing creatures Sumer (3300 AD, Mesopotamia) is probably the oldest written language. NU -3300text -10.000farming -100.000speech

5. What is HLT? Human Language Technology is the technology that mimics the human language capacity. Language UNDERSTANDING speech text sign

6. Redundancy Vlgoneseenoznrdeeok op eenEglneseuvinretsietmkaat het neituitinwlkeevloogdre de ltteers in eenwroodsaatn, het eingewatblegnaijrk is is dat de eretse en de ltaatseltteer op de jiutsepatalssaatn. De rset van de ltteersmgoenwllikueirggpletaastwdoren en je knutvrelvogensgwoeonlzeenwatersaatt. Ditkmotodmat we neitekleltteer op zcihlzeen maar het wroodalsgheeel.

7. Pensez a cequevsavezfnit et demandezvs i<<est coque 3 ai

8. Working applications Dialogue systems (telephony, real time, limited complexity) Disclosure systems (high quality audio, offline, complex)

9. ContactCenter Voice HLT Natural Language Search Web Mobile

10. Companies using speech technology

11. How may I help you Why are they calling? Classification based on the recognition of the question: “how may I help you” Who is calling? Identification via ZIP-code and house number

12. Organisations using speech technology

13. Disclosure of audiovisual archives The number of AV-archives on the Internet increases rapidly Archiving is not enough: disclosure and reusing is required! The use of HLT is needed (humans cost too much).

14. Digitalized (historic) collections Digital recorded collections WFH H.M. Koningin Wilhelmina Second feministic wave Buchenwald Memories of Indonesia LVSR

15. Searching in historic radio recordings:Radio Oranje

16. Oral History: Buchenwald

17. Oral History: Brandgrens, Rotterdam 10 getuigen van het bombardement van Rotterdam (mei ‘40) vertellen hun verhaal. TST wordt gebruikt om in de getuigenissen te zoeken.

18. Searching in the radio interviews of WFH

19. Searching in 46 interview collections:getuigenverhalen (600 hour)

20. Searching in 500 interviews in Croatia

21. CroMe - Audio Search Searching for: commandant Phrase boundaries 5 fragments found Found word (5x commandant)

22. CroMe - Audio Search Search word traumas Language found

23. Politicalmetings

24. Parliament transcriptions Gisteren was er een bespreking ivm de betrekkingen tussen Nederland en Vlaanderen

25. Recognition of lectures Record the speech Record the PPT Recognise the speech Use the display time of each slide as THE time unit Use the recognised speech as keywords for each slide

26. Searching in news broadcasts

27. Metadata -> Language model Text in the slide(s) Lectures handouts Language model Environmental texts

28. Questions?