What this is about?
Unlike many other areas within natural
language processing, Natural Language Generation (NLG) does not have a
tradition of sharing data resources, and no tradition of comparative
evaluation of alternative techniques. We believe the introduction of a
shared task for NLG would be of benefit to the community, both internally
and in terms of making the field more visible to those outside the
community. We would like to move discussion on this topic forward by means
of a special session at INLG'06. The focus of the session will be to move
us closer to an answer to the question: what would a shared task for NLG
look like?
Background
Over the past year, in particular at
ENLG'05 and UCNLG'05, NLG researchers have started discussing ways in
which the field might move towards a situation where sharing data and
other resources becomes possible and comparative evaluation results can be
produced. It is clear that the wide range of tasks and issues of interest
to researchers in NLG makes it difficult to identify a single set of data
that can be used by many, or a single task that could serve as the basis
for comparative evaluation of techniques. For this reason we believe it is
important to see if we can establish a consensus view amongst NLG
researchers as to the content and focus of a shared task.
We invite submissions of two kinds of
contributions to the INLG'06 Special Session on Sharing Data and
Comparative Evaluation:
(1) Short papers for inclusion in the
main INLG'06 proceedings:
(a)
Position papers that explore or make proposals regarding shared data and
shared tasks, covering topics including (but not necessarily limited to)
the following:
* standardisation of NLG tasks
* what kind of data people are interested in sharing
* mechanisms
for sharing
* evaluation
metrics
* human
evaluation methods
* evaluation
campaigns
* lessons
from evaluation campaigns in MT and document summarisation
(b) Reports on the results of experimenting with sharable data sets.
Contributors may use any data set at all, but for anyone looking for a
data set we recommend the SumTime Meteo data set (created by the SumTime
project group at Aberdeen), available from the SIGGEN webpage.
(2) Abstracts for oral presentations
during an Open Mic session on the topics of the special session. We hope
to involve a good number of NLG researchers, and particularly encourage
contributions likely to stimulate discussion.
Submission Information
Paper submissions should follow the two-column format of ACL
proceedings (http://www.acl2006.org/program/style/),
should be in PDF format, and should not exceed three (3) pages, including
references. As reviewing will be blind, please do not include the authors'
names and affiliations, and avoid revealing their identity in the text of
the paper. Please follow the submission instructions for the main INLG'06
session (details will be available on the Coling/ACL web site:
http://www.acl2006.org),
but note that you should submit your paper to the SPECIAL SESSION TRACK
(not the short paper track or any of the others). The deadline for
submissions is Friday April 28th (9 days later than the deadline for the
main INLG'06 session).
Abstracts for the Open Mic
part of the special session should not exceed 300 words in length and
preferably be in plain text format. Please send your abstract to Anja Belz
(asb@brighton.ac.uk) and Robert
Dale (rdale@ics.mq.edu.au) by no later than Friday May 27th.
Timetable
Submission of papers:
Notification of acceptance for papers:
Submission of abstracts for Open-Mic session:
Submission of camera-ready copy of papers:
Notification of acceptance for Open-Mic session:
INLG 2006: |
April 28th, 2006
May, 22nd, 2006
May, 27th, 2006
June 6th, 2006
June, 16th, 2006
July 15-16th, 2006 |
Questions regarding the special session, its topics and submission procedure, should be directed to the organisers, Anja Belz (asb@brighton.ac.uk) and Robert Dale (rdale@ics.mq.edu.au).
