A stochastic model of human-machine interaction for learning dialog strategies

  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Declaration of views_handler_filter::options_validate() should be compatible with views_handler::options_validate($form, &$form_state) in /home/speechmap/stcspeechmap.org/Drupal/modules/views/handlers/views_handler_filter.inc on line 589.
  • strict warning: Declaration of views_handler_filter::options_submit() should be compatible with views_handler::options_submit($form, &$form_state) in /home/speechmap/stcspeechmap.org/Drupal/modules/views/handlers/views_handler_filter.inc on line 589.
  • strict warning: Declaration of views_plugin_style_default::options() should be compatible with views_object::options() in /home/speechmap/stcspeechmap.org/Drupal/modules/views/plugins/views_plugin_style_default.inc on line 25.
  • strict warning: Declaration of views_plugin_row::options_validate() should be compatible with views_plugin::options_validate(&$form, &$form_state) in /home/speechmap/stcspeechmap.org/Drupal/modules/views/plugins/views_plugin_row.inc on line 135.
  • strict warning: Declaration of views_plugin_row::options_submit() should be compatible with views_plugin::options_submit(&$form, &$form_state) in /home/speechmap/stcspeechmap.org/Drupal/modules/views/plugins/views_plugin_row.inc on line 135.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Declaration of views_handler_argument::init() should be compatible with views_handler::init(&$view, $options) in /home/speechmap/stcspeechmap.org/Drupal/modules/views/handlers/views_handler_argument.inc on line 745.
  • strict warning: Declaration of views_handler_filter_node_status::operator_form() should be compatible with views_handler_filter::operator_form(&$form, &$form_state) in /home/speechmap/stcspeechmap.org/Drupal/modules/views/modules/node/views_handler_filter_node_status.inc on line 14.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
  • strict warning: Non-static method view::load() should not be called statically in /home/speechmap/stcspeechmap.org/Drupal/modules/views/views.module on line 879.
TitleA stochastic model of human-machine interaction for learning dialog strategies
Publication TypeJournal Article
Year of Publication2002
AuthorsLevin, E, Pieraccini R, Eckert W
JournalSpeech and Audio Processing, IEEE Transactions on
Volume8
Pagination11–23
ISSN1063-6676
KeywordsDialog systems, Markov decision process, rein- forcement learning, sequential decision process, speech, spoken language systems.
Abstract

In this paper, we propose a quantitative model for dialog systems that can be used for learning the dialog strategy. We claim that the problem of dialog design can be formalized as an optimization problem with an objective function reflecting different dialog dimensions relevant for a given application. We also show that any dialog system can be formally described as a sequential decision process in terms of its state space, action set, and strategy. With additional assumptions about the state transition probabilities and cost assignment, a dialog system can be mapped to a stochastic model known as Markov decision process (MDP). A variety of data driven algorithms for finding the optimal strategy (i.e., the one that optimizes the criterion) is available within the MDP framework, based on reinforcement learning. For an effective use of the available training data we propose a combination of supervised and reinforcement learning: the supervised learning is used to estimate a model of the user, i.e., the MDP parameters that quantify the user’s behavior. Then a reinforcement learning algorithm is used to estimate the optimal strategy while the system interacts with the simulated user. This approach is tested for learning the strategy in an air travel information system (ATIS) task. The experimental results we present in this paper show that it is indeed possible to find a simple criterion, a state space representation, and a simulated user parameterization in order to automatically learn a relatively complex dialog behavior, similar to one that was heuristically designed by several research groups.

URLhttp://www.robertopieraccini.com/publications/IEEE_SAP_01.pdf