DSTC11
The Eleventh Dialog System Technology Challenge
News
Note about the upcoming DSTC11 Workshop: It will likely take place in August or September 2023, and its location is TBD. We will announce the location as soon as it is known. If the event is in-person, the organizers will make arrangements to facilitate hybrid attendance (either remote presentation or recorded talks). There will be only one DSCT11 workshop, combining both summer and winter tracks.
Mar 06, 2023 - Workshop will take place in Aug or Sept 2023 and location is TBD. We will work towards facilitating hybrid attendance (in-person and remote)
Jan 28, 2023 - Updated timeline for papers of the summer track (notification of acceptance pushed back to Feb 24)
Dec 19, 2022 - Websites for Winter tracks available here
Nov 11, 2022 - Call for DSTC11 workshop papers added (paper deadline: Dec 2, 2022, 11:59 pm UTC-12 "anywhere on earth").
Nov 02, 2022 - 'Speech-Aware Dialog Systems Technology Challenge'. Finished! For details and the final results, see here.
August 1, 2022 - 'Robust and Multilingual Automatic Evaluation Metrics for Open-Domain Dialogue Systems' track website available here
June 29, 2022 - 'Ambiguous Candidate Identification and Coreference Resolution for Immersive Multimodal Conversations' track website available here
June 20, 2022 - 'Speech-Aware Dialog Systems Technology Challenge' track website available
June 15, 2022 - 'Intent Induction from Conversations for Task-Oriented Dialogue' track resources available here
May 17, 2022 - DSTC11 registration is open! Please sign up here.
May 13, 2022 - DSTC11 tracks announced
December 6, 2021 - DSTC11 call for proposals announced
Background and Motivation
Starting as an initiative to provide a common testbed for the task of Dialog State Tracking, the first Dialog State Tracking Challenge (DSTC) was organized in 2013, followed by DSTC2&3 in 2014, DSTC4 in 2015, and DSTC5 in 2016. Given the remarkable success of the first five editions, and understanding both, the complexity of the dialog phenomenon and the interest of the research community in a wider variety of dialog related problems, the DSTC rebranded itself as "Dialog System Technology Challenges" for its sixth edition. Then, DSTC6, DSTC7, DSTC8, DSTC9, and DSTC10 have been completed in in 2017-2021.
In this eleventh challenge, the call for task proposals has resulted into the following 5 different tracks split across two sessions: (1) Ambiguous Candidate Identification and Coreference Resolution for Immersive Multimodal Conversations, (2) Intent Induction from Conversations for Task-Oriented Dialogue, (3) Speech-Aware Dialog Systems Technology Challenge, (4) Robust and Multilingual Automatic Evaluation Metrics for Open-Domain Dialogue Systems and (5) Task-oriented Conversational Modeling with Subjective Knowledge. The objective of these tracks is to invite interested organizations conduct dialog related challenges in specific areas of research and under the umbrella of the DSTC.