A Transcription Task for Crowdsourcing with Automatic Quality Control

  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Declaration of views_handler_filter::options_validate() should be compatible with views_handler::options_validate($form, &$form_state) in /home/speechmap/stcspeechmap.org/Drupal/modules/views/handlers/views_handler_filter.inc on line 589.
  • strict warning: Declaration of views_handler_filter::options_submit() should be compatible with views_handler::options_submit($form, &$form_state) in /home/speechmap/stcspeechmap.org/Drupal/modules/views/handlers/views_handler_filter.inc on line 589.
  • strict warning: Declaration of views_plugin_style_default::options() should be compatible with views_object::options() in /home/speechmap/stcspeechmap.org/Drupal/modules/views/plugins/views_plugin_style_default.inc on line 25.
  • strict warning: Declaration of views_plugin_row::options_validate() should be compatible with views_plugin::options_validate(&$form, &$form_state) in /home/speechmap/stcspeechmap.org/Drupal/modules/views/plugins/views_plugin_row.inc on line 135.
  • strict warning: Declaration of views_plugin_row::options_submit() should be compatible with views_plugin::options_submit(&$form, &$form_state) in /home/speechmap/stcspeechmap.org/Drupal/modules/views/plugins/views_plugin_row.inc on line 135.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Declaration of views_handler_argument::init() should be compatible with views_handler::init(&$view, $options) in /home/speechmap/stcspeechmap.org/Drupal/modules/views/handlers/views_handler_argument.inc on line 745.
  • strict warning: Declaration of views_handler_filter_node_status::operator_form() should be compatible with views_handler_filter::operator_form(&$form, &$form_state) in /home/speechmap/stcspeechmap.org/Drupal/modules/views/modules/node/views_handler_filter_node_status.inc on line 14.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
TitleA Transcription Task for Crowdsourcing with Automatic Quality Control
Publication TypeJournal Article
Year of Publication2011
AuthorsLee, C, Glass J
JournalProc. Interspeech2011, Florence
Keywordscrowdsourcing, quality control, Transcription
Abstract

In this paper, we propose a two-stage transcription task de- sign for crowdsourcing with an automatic quality control mech- anism embedded in each stage. For the first stage, a support vector machine (SVM) classifier is utilized to quickly filter poor quality transcripts based on acoustic cues and language patterns in the transcript. In the second stage, word level confidence scores are used to estimate a transcription quality and provide instantaneous feedback to the transcriber. The proposed de- sign was evaluated using Amazon Mechanical Turk (MTurk) and tested on seven hours of academic lecture speech, which is typically conversational in nature and contains technical mate- rial. Compared to baseline transcripts which were also collected from MTurk using a ROVER-based method, we observed that the new method resulted in higher quality transcripts while re- quiring less transcriber effort.

URLhttp://groups.csail.mit.edu/sls/publications/2011/Lee1_Interspeech11.pdf