Get me straight to the DOWNLOAD page

Many referring expression generation algorithms don't attempt to include spatial relations between the target referent and landmark objects into the descriptions they produce at all. Most systems that are able to deal with spatial relations only use them if the referent can't be uniquely identified without them. An interesting exception is the first one that appeared in the literature, Dale & Haddock's (1991) Relational Algorithm, which however showed extremely cumbersome descriptions in my first evaluation experiment of existing algorithms.

The prevalent hypothesis that spatial relations should only be used as a last resort (I call it the "absolute before relational" hypothesis) is usually justified by psycholinguistic findings suggesting that they cause a higher cognitive load for both speaker and listener. However, it seems quite obvious to me that in many situations a spatial relation would be both easier to produce and easier to undersand than a (possibly long) list of non-relational properties. For example, if I want a certain book that looks similar to a lot of the other books, mentioning the fact that it's just left of the big red one would be a better way to go than, for example, making the listener read the titles of all the little black books to find the one I mean.

To put the "absolute properties before relations" hypothesis to the test, I collected two corpora of referring expressions for very simple objects in very simple 3D scenes. The first corpus is the GRE3D3 Corpus which contains 630 descriptions, the second corpus is the GRE3D7 Corpus which contains 4480 descriptiosn. In all scenes it was possible to identify the target without using relations. However over a third of the descriptions in GRE3D3 and about 14% in GRE3D7 nonetheless contained spatial relations. A few hypotheses about the factors influencing the use of spatial relations can be drawn from the data.
The data gathering experiment and analysis of the GRE3D3 corpus are described in

Jette Viethen and Robert Dale (2008). The use of spatial relations in referring expressions. In Proceedings of the 5th International Conference on Natural Language Generation, 59-67, Salt Fork OH, USA. [ pdf | slides ]

Some further analysis of the corpus can be found in

Jette Viethen and Robert Dale (2008). Generating relational references: What makes a difference? In Proceedings of the 6th Australasian Language Technology Workshop, 160-168, Hobart, Australia. [ pdf ]

The collection of the GRE3D7 Corpus and some analyses are described in

Jette Viethen and Robert Dale (2011). GRE3D7: A corpus of distinguishing descriptions for objects in visual scenes. In Proceedings of the Workshop on Using Corpora in Natural Language Generation and Evaluation, 12-22, Edinburgh, Scotland. [ pdf | slides ]

A more detailed description of both corpora can be found in Chapter 5 of

Henriette Anna Elisabeth Viethen (2011). The Generation of Natural Descriptions: Corpus-Based Investigations of Referring Expressions in Visual Domains. PhD Thesis, Macquarie University, Sydney, Australia. [ pdf ]

You are welcome to download the corpora and use them for your own research, as long as you cite the papers that come with each download. It would be fantastic to hear about your plans, in case you do. Just email me.