Inter-frame contextual modelling for visual speech recognition

Adrian Pass; Ji Ming; Philip Hanna; Jianguo Zhang; Darryl Stewart

doi:10.1109/ICIP.2010.5652630

Inter-frame contextual modelling for visual speech recognition

Adrian Pass, Ji Ming, Philip Hanna, Jianguo Zhang, Darryl Stewart

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

Abstract

In this paper, we present a new approach to visual speech recognition which improves contextual modelling by combining Inter- Frame Dependent and Hidden Markov Models. This approach captures contextual information in visual speech that may be lost using a Hidden Markov Model alone. We apply contextual modelling to a large speaker independent isolated digit recognition task, and compare our approach to two commonly adopted feature based techniques for incorporating speech dynamics. Results are presented from baseline feature based systems and the combined modelling technique. We illustrate that both of these techniques achieve similar levels of performance when used independently. However significant improvements in performance can be achieved through a combination of the two. In particular we report an improvement in excess of 17% relative Word Error Rate in comparison to our best baseline system. © 2010 IEEE.

Original language	English
Title of host publication	2010 17th IEEE International Conference on Image Processing, ICIP 2010
Subtitle of host publication	Proceedings
Place of Publication	Piscataway, NJ
Publisher	IEEE
Pages	93-96
Number of pages	4
ISBN (Print)	9781424479948
DOIs	https://doi.org/10.1109/ICIP.2010.5652630
Publication status	Published - 2010
Event	17th IEEE International Conference on Image Processing - Hong Kong Convention and Exhibition Centre, Hong Kong, Hong Kong Duration: 26 Sept 2010 → 29 Sept 2010 http://www.icip2010.org/

Conference

Conference	17th IEEE International Conference on Image Processing
Abbreviated title	ICIP 2010
Country/Territory	Hong Kong
City	Hong Kong
Period	26/09/10 → 29/09/10
Internet address	http://www.icip2010.org/

Access to Document

10.1109/ICIP.2010.5652630

http://www.scopus.com/record/display.url?eid=2-s2.0-78651061157&origin=resultslist&sort=plf-f&src=s&st1=An+Investigation+into+Features+for+Multi-view+Lipreading&sid=519CA88E287E53F3338E63F77B2DF5B5.fM4vPBipdL1BpirDq5Cw%3a360&sot=b&sdt=b&sl=71&s=TITLE-ABS-KEY%28An+Investigation+into+Features+for+Multi-view+Lipreading%29&relpos=0&relpos=0&citeCnt=0&searchTerm=TITLE-ABS-KEY%28An+Investigation+into+Features+for+Multi-view+Lipreading%29

2 Citations
1 Paper

Inter-frame contextual modelling for visual speech recognition
Pass, A., Ming, J., Hanna, P., Zhang, J. & Stewart, D., 2010.
Research output: Contribution to conference › Paper

1 Participation in conference

17th IEEE International Conference on Image Processing
Zhang, J. (Participant)
26 Sept 2010 → 29 Sept 2010
Activity: Participating in or organising an event types › Participation in conference

Cite this

@inproceedings{e14c429712f04e169fabb01f3461bb95,

title = "Inter-frame contextual modelling for visual speech recognition",

abstract = "In this paper, we present a new approach to visual speech recognition which improves contextual modelling by combining Inter- Frame Dependent and Hidden Markov Models. This approach captures contextual information in visual speech that may be lost using a Hidden Markov Model alone. We apply contextual modelling to a large speaker independent isolated digit recognition task, and compare our approach to two commonly adopted feature based techniques for incorporating speech dynamics. Results are presented from baseline feature based systems and the combined modelling technique. We illustrate that both of these techniques achieve similar levels of performance when used independently. However significant improvements in performance can be achieved through a combination of the two. In particular we report an improvement in excess of 17% relative Word Error Rate in comparison to our best baseline system. {\textcopyright} 2010 IEEE.",

author = "Adrian Pass and Ji Ming and Philip Hanna and Jianguo Zhang and Darryl Stewart",

year = "2010",

doi = "10.1109/ICIP.2010.5652630",

language = "English",

isbn = "9781424479948",

pages = "93--96",

booktitle = "2010 17th IEEE International Conference on Image Processing, ICIP 2010",

publisher = "IEEE",

note = "17th IEEE International Conference on Image Processing, ICIP 2010 ; Conference date: 26-09-2010 Through 29-09-2010",

url = "http://www.icip2010.org/",

}

Pass, A, Ming, J, Hanna, P, Zhang, J & Stewart, D 2010, Inter-frame contextual modelling for visual speech recognition. in 2010 17th IEEE International Conference on Image Processing, ICIP 2010: Proceedings. IEEE, Piscataway, NJ, pp. 93-96, 17th IEEE International Conference on Image Processing, Hong Kong, Hong Kong, 26/09/10. https://doi.org/10.1109/ICIP.2010.5652630

TY - GEN

T1 - Inter-frame contextual modelling for visual speech recognition

AU - Pass, Adrian

AU - Ming, Ji

AU - Hanna, Philip

AU - Zhang, Jianguo

AU - Stewart, Darryl

PY - 2010

Y1 - 2010

N2 - In this paper, we present a new approach to visual speech recognition which improves contextual modelling by combining Inter- Frame Dependent and Hidden Markov Models. This approach captures contextual information in visual speech that may be lost using a Hidden Markov Model alone. We apply contextual modelling to a large speaker independent isolated digit recognition task, and compare our approach to two commonly adopted feature based techniques for incorporating speech dynamics. Results are presented from baseline feature based systems and the combined modelling technique. We illustrate that both of these techniques achieve similar levels of performance when used independently. However significant improvements in performance can be achieved through a combination of the two. In particular we report an improvement in excess of 17% relative Word Error Rate in comparison to our best baseline system. © 2010 IEEE.

AB - In this paper, we present a new approach to visual speech recognition which improves contextual modelling by combining Inter- Frame Dependent and Hidden Markov Models. This approach captures contextual information in visual speech that may be lost using a Hidden Markov Model alone. We apply contextual modelling to a large speaker independent isolated digit recognition task, and compare our approach to two commonly adopted feature based techniques for incorporating speech dynamics. Results are presented from baseline feature based systems and the combined modelling technique. We illustrate that both of these techniques achieve similar levels of performance when used independently. However significant improvements in performance can be achieved through a combination of the two. In particular we report an improvement in excess of 17% relative Word Error Rate in comparison to our best baseline system. © 2010 IEEE.

U2 - 10.1109/ICIP.2010.5652630

DO - 10.1109/ICIP.2010.5652630

M3 - Conference contribution

SN - 9781424479948

SP - 93

EP - 96

BT - 2010 17th IEEE International Conference on Image Processing, ICIP 2010

PB - IEEE

CY - Piscataway, NJ

T2 - 17th IEEE International Conference on Image Processing

Y2 - 26 September 2010 through 29 September 2010

ER -

Inter-frame contextual modelling for visual speech recognition

Abstract

Conference

Access to Document

Fingerprint

Research output

Inter-frame contextual modelling for visual speech recognition

Activities

17th IEEE International Conference on Image Processing

Cite this