Completions and continuations in dialogue: a preliminary account

Massimo Poesio (Uni Essex)Hannes Rieser (Uni Bielefeld)

CATALOGBarcelona, July 2004

Sentence cooperations: an example

Inst: So, jetzt nimmst DuWell, now you grasp

Cnst: eine Schraubea screw.

Inst: eine <-> orangene mit einem Schlitz.an <-> orange one with a slit

Cnst: Ja. Yes.

Sentence cooperations: an informal definition (Clark, 1996)

SENTENCE COOPERATION: At least two dialogue participants contribute to a sentence production

COMPLETION: sub-sentential structure continued by obligatory constituents

CONTINUATION: material added to already `complete’ sentence

The significance of sentence cooperations

Clear evidence that dialogue requires coordination at the sub-sentential level (see also Pickering and Garrod, in press)

Provide insights into incrementality and compositionality issues

A tool to investigate competing claims about coordination in dialogue

Purely intentional models Pickering and Garrod’s IAM based on simpler alignment

mechanisms.

Outline of the talk

Sentence cooperations in the Bielefeld Toy Plane Corpus (BTPC)An introduction to PTT (Poesio and Traum 1997, 1998; Matheson et al,

2000; Poesio, to appear) Intro to model of we-intentions we use, following

Bratman (1992), Tuomela (2000) and Grosz and Kraus (1996)

A PTT implementation of an intentional analysis of completions (May have time to sketch an IAM analysis)

The Bielefeld Toy Plane Corpus

The Bielefeld Toy Plane Corpus

22 video-filmed, speech recorded and transcribed dialogues two agents, Instructor and Constructor constructing a “Baufix” airplane different sight conditions: total screen, half-screen,

face to face

3675 contributions 160 sentence cooperations (4.34 %) in most of them cooperation other-initiated (95%)

Sentence cooperations in the BTPC (Skuplik, 1999)

126 sentence cooperations from the BTPC 54 completions (43%) 72 continuations (57%)

Production of the completion / continuation: 79% Cnst, 21% Inst

84% of compl. / contin. accepted by previous speaker (41% implicitly)

Release-turn signalled in 31% of cases

A few other observations

Completions become more frequent as dialogue procedes (routinization?)

The example, revisited

Inst: So, jetzt nimmst DuWell, now you grasp

Cnst: eine Schraubea screw.

Inst: eine <-> orangene mit einem Schlitz.an <-> orange one with a slit

Cnst: Ja. Yes.

CNST COMPLETION(70%) WITH AN OBLIGATORY NP

RESULTING IN A SENTENCE WHEN MERGED

SIGNALED BY LENGHTENING OF “DU”, LEVEL TONE

Outline of the talk

The Bielefeld Toy Plane Corpus (BTPC)

An introduction to PTT (Poesio, 1995; Poesio and Traum 1997,

1998; Matheson et al, 2000; Poesio, to appear) Intro to model of we-intentions we use, following

Bratman (1992), Tuomela (2000) and Grosz and Kraus (1996)

A PTT implementation of an intentional analysis of completions

PTT

A theory of semantics and interpretation in dialogue originally motivated by work on the TRAINS projectKey characteristics:

Building on (Compositional) DRT (Muskens, 1996) Common ground as a record of the discourse situation

(Barwise and Perry, 1983) An account of incremental semantic interpretation An account of GROUNDING So far, primarily concerned with aspects of dialogue

driven by obligations

Common ground: beyond assertion

A They have at their disposal enormous assets // and their policy

B //look can I just come in on that// last year

A //YES IN A MINUTE IF YOU MAY AND WHEN I’M FINISHED // then you’ll know

B // yes I’M SO SORRY

(Coulthard 1977)

Common ground: beyond assertion

B: Go to Elmhurst, pass the courthouse and go to Elmhurst and then to Elmhurst, uh north.A: mm hum.B: Towards Riverton, till you come to that Avila HallA: Oh yesB: Dju know where that//is?A: //uh huhA: Oh surelyB: Avilla Hall on the corner of Bor//donA: //uh huhB: Well there, on Bordon you turn back to town, left.

(George Psathas, "Direction-giving in Interaction," in Boden and Zimmerman, ed.)

From DRT to PTT

a. A: There is an engine at Avon.

b. B: It is hooked to a boxcar

DRT: [ x,w,y,u,s,s’| engine(x), Avon(w), s: at(x,w), boxcar(y), s’:hooked-to(u,y),

u=x]

Common ground and discourse situation in PTT

[ce1,ce2,K1,K2|

K1=[x,w,s| engine(x), Avon(w), s: at(x,w)],

ce1: assert(A,B,K1)

K2=[y,z,s’| boxcar(y), s’:hooked-to(z,y), z=x],

ce2: assert(B,A,K2)]

Locutionary acts in the common ground

“The fact that a speaker is speaking, saying the words he is saying in the way he is saying them, is a fact that is usually accessible to everyone present. Such observed facts can be expected to change the presumed common background knowledge of the speaker and his audience in the same way that any obviously observable change in the physical surroundings of the conversation will change the presumed common knowledge.” (Stalnaker, Assertion, p. 323)

The time-order of sentence processing

GARDEN-PATH phenomena shows that parsing is INCREMENTAL (Bever, 1974; Frazier, 1987)Marslen-Wilson 1973, 1975: semantic information ALSO accessed immediatelySwinney, 1979: lexical access incrementalJust and Carpenter,1980: IMMEDIACY HYPOTHESIS (“Every word encountered should be processed to the deepest level possible before the eye moves on to the next word”)Eye-tracking work (Tanenhaus et al, 1995, tomorrow): really fine-grained incrementality

Alignment at all levels Pickering & Garrod

Clarification questions (Ginzburg and Cooper, Purver

and Ginzburg)

A: Did Bo leave?B: BO? A: Bo Smith.B: Yes, half an hour ago.

Matthew: It wasn’t all that bad. At least the pool was clean.Lara: MR POOL?Matthew: The pool.Lara: Oh. <laugh>

(“What is the intended content of your utterance ‘Bo’?”)

(“Did you utter the words ‘Mr. Pool’?”)

Micro conversational events (Poesio, 1995)

boxcar [u|u:utter(A,”boxcar”), Noun(u), sem(u)=x [|boxcar(x)], + SYN INFO (NEXT)]

umm [u,ce| u: utter(A,”umm”), ce: keep-turn(A), generate(u,ce)]

MCEs in the example dialogue

[mce6| mce6:utter(Cnst,"Schraube"), Noun(mce6), sem(mce6)= v([ |screw(v)]];

[mce1,ce1| mce1:utter(Inst,“so"), Adv(mce1), ce1:take-turn(Inst), generate(mce1,ce1)];[mce2,ce2| mce2:utter(Inst,“jetzt"), Adv(mce2), ce2:keep-turn(Inst), generate(mce2,ce2)]; [mce3| mce3:utter(Inst,"nimmst"), Verb(mce3), sem(mce3)= Qx(Q(x’[e| e: grasp(x, x’)]))];[mce4| mce4:utter(Inst,"Du"), Pro(mce4), sem(mce4)= P.P (you)];[mce5| mce5:utter(Cnst,"eine"), Det(mce5), sem(mce5)= P’P([y| ]; P’(y); P(y))]

Syntactic interpretation with MCEs (Poesio, 1996)

MCE1 CE1

mce1:utter(Inst,“so"),ce1:take-turn(Inst), generate(mce1,ce1)];

MCE2 CE2

U1:S

MCE3:”nimmst”:V

U2:NP

U3:NP

MCE3

MCE4:”Du”:Pro

U4:NP

MCE4

Syntactic Interpretation with MCEs, II

MCE1 CE1

mce1:utter(Inst,“so"),ce1:take-turn(Inst), generate(mce1,ce1)];

MCE2 CE2 MCE3

U1:S

MCE3:”nimmst”:V

U2:NP

U3:NPMCE4:”Du”:Pro

MCE4

Semantic interpretation and compositionality

)()3(

)2(

,)1(,32 ,31

,

usem

usem

usemuuuu

BINARY SEMANTIC COMPOSITION

U3:()

U1: U2:

Intentions and obligations

OBLIGATIONS: [o | o:OblCnst ([|address(Cnst, ce1)])]

INTENTIONS: [i | i:IntInst&Cnst ([|join(Cnst, wing1,fuselage1)])]

INTENTIONAL STRUCTURE: Grosz&Sidner-like sp(i1) = i2 dom(i1) = i2

Grounding

As in proposals such as Clark and Schaefer (1989) and Traum (1994), establishment of common ground (‘G’) modeled in terms of CONTRIBUTIONS, or DISCOURSE UNITS, that may be ACKNOWLEDGED or REPAIRED

Discourse Units and Grounding Acts

MCE1 CE1

mce1:utter(Inst,“so"),ce1:take-turn(Inst), generate(mce1,ce1)];

MCE2 CE2 MCE3 …

U1:S

MCE3:”nimmst”:V

U2:NP

U3:NPMCE4:”Du”:Pro

DU1 …. DU17

DU17 =

ACK(DU17)

CONT(DU17)

REPAIR(DU17)

What prompts the completion? Two accounts

Intentional accountNeed to explain why help

Alignment accountWhat representation is aligned?

HR

Outline of the talk

The Bielefeld Toy Plane Corpus (BTPC)

An introduction to PTT (Poesio and Traum 1997, 1998; Matheson et

al, 2000; Poesio, to appear) Intro to model of we-intentions we use, following

Bratman (1992), Tuomela (2000) and Grosz and Kraus (1996)

A PTT implementation of an intentional analysis of completions

Public and private (partial) plans

Inst has a fully specified plan for building the toy-airplane (drawing or model)The public plan among Inst and Cnst is usually underspecified, but gets more refined throughout the construction dialogue Besides the shared partial plan, the agents have private plans which overlap to some extent with the shared plan.The difference between the public plan and the private plans leads to discrepancies and negotiations

The partial shared plan before the example

 

  

 

Join wing and fuselage

Assemble toy airplane

Assemble fuselage Assemble wing

get 5h bar

get 3h bar

get 7h bar1

get 7h bar2

join join alignwing&

fuselage

getbolt

putthroughget

boltgetbolt

getnut

We-intentions for shared cooperative

activity Tuomela’s (2000) modification of Bratman’s 1993 definition of SCA, adapted:

Inst and Cnst WE-INTEND that Cnst join wing and fuselage is equivalent to: It is Inst’s and Cnst’s mutual knowledge that

Inst intends that Cnst join wing&fuselage because Cnst intends that Cnst join wing&fuselage

and Cnst intends that Cnst join wing&fuselage because Inst

intends that Cnst join wing&fuselage

Tuomela’s definition of we-intention, formal

IntInst&Cnst(join(Cnst, W&F))

MK((IntInst join(Cnst, W&F))

/r IntCnst(join(Cnst, W&F))) and

(IntCnstjoin(Cnst, W&F)) /r

IntInst(join(Cnst, W&F))))

(where /r is the reason relation, which is factual)

Outline of the talk

The Bielefeld Toy Plane Corpus (BTPC)

An introduction to PTT (Poesio and Traum 1997, 1998; Matheson et

al, 2000; Poesio, to appear) Intro to model of we-intentions we use, following

Bratman (1992), Tuomela (2000) and Grosz and Kraus (1996)

A PTT implementation of an intentional analysis of completions

The situation before example 1

MP

The partial shared plan before the example

 

  

 

Join wing and fuselage

Assemble toy airplane

Assemble fuselage Assemble wing

get 5h bar

get 3h bar

get 7h bar1

get 7h bar2

join join alignwing&

fuselage

getbolt

putthroughget

boltgetbolt

getnut

Reaching the intention to perform a directive

2.: intertwined discourse / domain planjoin(Cnst,Obj1,Obj2) >Ash&Morr b & c & d & e & f,

(b) 1. direct(Inst, Cnst, grasp(Cnst, Bolt)),2. grasp(Cnst, Bolt),3. tell(Cnst, Inst, grasp(Cnst, Bolt ))

(c) 1.direct(Inst, Cnst, grasp(Cnst, Nut)),2. grasp(Cnst, Nut),3. tell(Cnst, Inst, grasp(Cnst, Nut))

(d) …put through… (e) …fasten.. (f) ….feedback.

1.: (partial) we-intention to join: [i | i:IntInst&Cnst ([|join(Cnst, wing1,fuselage1)])]

Deciding to perform a directive, II

5. Achieve partial Inst intention (& Cnst intention)[i2 | i2:IntInst ([K1,ce1|K1 = [e, x|bolt(x), e:grasp(Cnst,x)]

ce1:direct(Inst, Cnst, K1)])]

3. Distributivity of we-intention

4. Achieve partial we-intention to perform directive:[i1 | i1:IntInst&Cnst ([K1,ce1|K1 = [e,x|bolt(x), e:grasp(Cnst,x)]

ce1:direct(Inst, Cnst, K1)])]

Inst’s private plan

 

  

 

Join wing and fuselage

Assemble toy airplane

Assemble fuselage Assemble wing

get 5h bar

get 3h bar

get 7h bar1

get 7h bar2

join join alignwing&

fuselage

getbolt

…

getbolt

getbolt

orange-bolt-with-slit

Cnst’s private plan

 

  

 

Join wing and fuselage

Assemble toy airplane

Assemble fuselage Assemble wing

get 5h bar

get 3h bar

get 7h bar1

get 7h bar2

join join alignwing&

fuselage

getbolt

…..

getbolt

getbolt

bolt

Planning the directive

7. develop plan to perform utterance that generates directive:

[i4 | i4:IntInst ([u1.1 | utterance(u1.1), sem(u1.1) = K2,

generates(u1.1,ce2)])]

6. arrive at more specific intention (evidence: subsequent repair)

[i3 | i3:IntInst ([K2,ce2|K2 = [e, x|bolt(x),x=orange-slit-bolt,

e:grasp(Const,x)] ce2:direct(Inst, Cnst, K2)])]

Micro-plan8. plan to perform utterance in terms of MCEs

[i5a | i5a:IntInst ([u1.2 | u1.2: „so“:take-turn])];

9. lengthening signals problem - Inst doesn’t necessarily know which bolts are unused possibly does not know how to refer to bolt (NP type / content)

[i5b | i5b:IntInst ([u1.3 | u1.3:“jetzt“:keep-turn])];

[i5c | i5c:IntInst ([u1.4 .. u1.7|

S(u1.1) , u1.4:“nimmst“:V, u1.5:“Du“:NP, u1.7:VP,NP(u1.6),u1.5 u1.1, u1.7 u1.1, u1.4 u1.7, u1.6 u1.7]

Possible motivations for the completion

Interpret lengthening as request to continue obl(Cnst, cont(DU))

Interpret lengthening as request for acknowledgment (also standard PTT)obl(Cnst, ack(DU))

Cooperativeness`Blurting out’

A cooperativeness analysis

10. Cnst acquires intention to turn the directive in a joint action (cfr. Tuomela’s “unrequired contributory actions”)

[i6 | i6:IntCnst ([K3,ce3|K3 = [e,x|bolt(x), e:grasp(Const,x)]

ce3:direct(Inst&Cnst, Cnst, K3)])]

Derivation of 10 not axiomatized by Tuomela, but we assume here is the result of an intention to help.

11. Cnst produces plan to perform action to generate directive; analogous to Inst’s, but content (partial) K3:

[i7 | i7:IntCnst ([u1.1a | utterance(u1.1a), sem(u1.1a) = K3,

generates(u1.1a,ce3)])]

Further specification not possible

Micro-plan for Cnst’s completion

12. most of actions in plan already performed by Inst; Cnst plans missing action

[i8 | i8:IntCnst ([ K1.1d | u1.6:“eine Schraube“:NP,

K1.1d=[x1|bolt(x1)] sem(u1.6) = K1.1d ])]

(Same action would be planned to continue contribution and to acknowledge)

13. Instructor begins repair due to his private plan

An alternative analysis: the IAM model

Successful dialogue involves the development of aligned representations at all levelsAligned representations the result of priming mechanisms at every level of linguistic representation Mental state reasoning an option but not basic

Alignment at all levels

An IAM-analysis of the BTPC example

What leads Cnst to produce “eine Schraube”?

What is a situation model in this domain? “the key dimensions encoded in situation

models are SPACE, TIME, CAUSALITY, INTENTIONALITY, and REFERENCE to the MAIN INDIVIDUALS … “ (p. 7)

Two possible views of the situation model in BTPC domain

The partial shared plan (cfr. earlier analysis)

Cfr. mention of intentionalityWould make the IAM model much closer to the

model presented earlierThe state of Cnst’s plane assembly

The ‘state of assembly route’

Problems with this type of dialogues:Situation models are clearly NOT aligned

(they will only be so at the END of the conversation)

Cannot assume implicit common ground

Need to extend model to deal with directives

A proposal

Based on the notion of priming

Extend the notion of “routine” to non-linguistic actions

During the dialogue, an AGGREGATE FORMATION ROUTINE gets established

Development of the AF routine

JOIN(TAIL, FUSELAGE)

JOIN(TAIL, FUSELAGE)

JOIN(SIDE-RUDDER, FUSELAGE)

The routinePARAMETERS: 2 constituents to be joined through a port a fixing mechanism going through this port (currently

underspecified)

ROUTINE FOR AGGREGATE FORMATION: align material to be joined (requires port identif.) obtain fixing mechanism put fixing mechanism through port fasten fixing mechanism yields: aggregate

Generation of“eine Schraube”

Based not on global shared plan, but on local instantiation of AF-routine, primed in context

Only one parameter is still underspecified: the bolt

Local execution of routine at the ‘obtain fixing mechanism’ point

Syntactic structure to realize action presumably also available through routine

Open problems with this account

Some motivation for completion needed More in general, how is helping done? (Normal cooperativity axioms are based on

beliefs & intentions.)

Also need to say something more about the choice of that particular realizationEconomy principle?

Preliminary conclusions

PTT provides the technical tools to formalize a crucial feature of sentence cooperations: coordination at the micro conversational event levelMind-reading always difficult, but Tuomela’s theory of we-intention goes some way towards formalizing one of the possible motivations for completions, in terms of “help”A preliminary investigation of the alignment route also possible