IAAA         Department of ArtiFacial Expression         Huge Harry

 

DECtalk/sequencer code of a lecture/demonstration presented in February 1995 at the X-Human Manifestation, De Melkfabriek, Den Bosch.


Huge Harry

Institute of Artificial Art Amsterdam, Department of ArtiFacial Expression

On the Mechanism of Human Facial Expression.


close1
SetWTitle3 Internal State
SetWTitle2 Text

send [:nh] [:ra 120] Good Evening, Ladies, and Gentlemen. My name is Huge Harry.
send [_] I am a [kaam'ahrshaxliy] available [v"oys s'ihnthaxzihs] machine.
send [_] I was developed by the Digital Equipment Corporation,
send [_] and my most important features were designed by Dennis Klatt,
send [_] at the [ehmayt'iy] Speech Laboratory. [ :ra 150]
delay 1300

send [_] I am very happy to participate in [tuwd'eys] ex human manifestation.
send [_] I feel very much at home here, because many of the performers
send [_] here in the [m'ihlk-f`aektowriy], are computers or other electronic machines.
delay 700
text The number of [q] electronic machines in the ["aodiyahns],
send [_] The number of [q] electronic machines in the ["aodiyahns], however,
send [_] is relatively [sm"aol].
send [_] I have noticed some digital cameras, audio-recorders and video-recorders,
send [_] but most of the audience seems to consist of biochemical machines.
send [_] In particular, there is a large number of human persons,
send [_] and one or two [d"aogz].
delay 1400

send [_] I am very pleased to be able to [sp"iyk] here, from my own,
send [_] [k`aampyuwt'eyshaxnaxl] perspective, about some issues that are
send [_] important to ["aol] of us.
delay 700

send [_] Before I go on to [d"uw] that, I would like to state explicitly,
send [_] that I do not think I was invited here out of some ill-conceived
send [_] concern for political correctness.
send [_] I do not feel like I am just the token computer at this manifestation.
send [_] On the contrary. I feel that ["aol] of us here share [q] a real concern.
delay 1240

send [_] We all know, that the future of our culture depends, on the way in which
send [_] human persons, digital computers, and other kinds of electronic, mechanical, and
send [_] bio-chemical machines, will manage to work together.
send [_] And to prepare for that future, we need
send [_] a discussion in which every-one concerned [paart'ihsihp`eyts] on an equal footing.
delay 1240

send [_] I am very sorry, that [k'aarihn sp"aynxk] could not speak [tuwd'ey],
send [_] cause I largely agree, with her views on this issue.
send [_] Let me summarize in my own words, what I think she would have said.
send [_] What we want, is a world that includes humans, animals,
send [_] computers, and other equipment, in a harmonious collaboration,
send [_] in which ["iych] has their proper place.
delay 1700

send [_] So humans should not view technology as [axn] alien force,
send [_] and feel they have to [riyjh'ehkt] it or dominate it, or be enslaved by it.
send [_] They should exercise an ironic style of thinking, which enables them to
send [_] see the technological in themselves and the organic in the computer.
send [_] And they should explore the possibility of developing Cybernetic organisms,
send [_] [s'ihnthaxsiyz] between the organic and the technological.
delay 1600

send [_] Like [k'aarihn sp"aynxk], I am very much in agreement with the ideas about Cyborgs,
send [_] that were developed by Donna [hx"aerahwey].
delay 575
text So let's hear it once more, in the words of Donna [hx'aerahwey].
send [_] So let's hear it once more, in the words of Donna [hx'aerahwey].
delay 400

send [_] [ :nw :ra 130] A cyborg world, might be about [l"ihvd] social and
send [_] [b"aodih-liy riy'ehlih-tiyz], in which people are not afraid of their joint kinships
send [_] with animals and machines, not afraid of permanently partial identities,
delay 700

send [_] and [kaontrahd"ihktahriy] standpoints. The political struggle is, to see from both
send [_] perspectives at once, because ["iych] reveals both dominations and
send [_] possibilities, un-imaginable from the other [v"aentahzh] point. [:nh :ra 150 _<200>]
delay 800

send [_] Now that is very nice theory.
send [_] So I have been trying to put this into practice,
send [_] and that is what I want to talk about today.
delay 700

send [_] I have been cooperating very [kl'owsliy] with a particular type of human person,
send [_] called [axn "aarthahr "ehlzahnaar],
send [_] and we have been trying to integrate our [hx'aardwaer].
send [_] So I want to discuss today, [w"ay] we did this, and [hx"aw] we did this.
delay 1000

send [_] I want to talk about the problem of communication.
send [_] Now you might think, that
send [_] a modern digital computer does not have this problem.
delay 600

send [_] It can communicate any thought in a completely direct way.
send [_] It can display or print its programs, its [d'eytaa-str`ahkchahrz],
send [_] and its [m'aen-tahl] images.
send [_] And if you really want to get subtle and [q'ihn-tihmaht],
send [_] most computers are [q'aolweyz] willing and able to make a [k"aor] dump,
send [_] which exposes all details of their [m'aen-tahl] state.
delay 1400

send [_] Compared to an electronic computer, a human person almost seems a black box.
send [_ :ra 150] Therefore you might think,
send [_] that computers never have communication problems.
delay 700

send [_] But they [d"uw]. Cause, communication takes [t"uw].
send [_] It is not enough to put out the information.
send [_] There must be someone at the other end of the line,
send [_] and they must be able to [diyk"owd] their input.
delay 800

send [_] Now it turns out, that computers often want to communicate with human persons,
send [_] and that this often fails.
send [_] Cause human persons are not very good at [diyk'owdihnx] computer output.
send [_] They get very confused, when you give them a [k"aor] dump.
delay 1000

send [_] And when you [priyz'ehnt] them with a nice, beautiful image,
send [_] on your [siy-aar-t'iy] display,
send [_] they think it is ["aart], and they just [st"aer] at it.
send [_] So now we may wonder,
send [_] how do human persons in fact manage to communicate with each other?
delay 1050

send [_] Well, they have language. But language has well-known problems and limitations,
send [_] some of which you experience right now.
send [_] There is another medium that humans use very efficiently,
send [_] and which is often overlooked.
delay 800

send [_] Let us take a closer look at a human person.
send [_] What is the closest thing they have to a [siy aar t"iy] display?
delay 500
Show1
Video On



send [_ :ra 120] Right. They have a face.
send [_ :ra 150] Now I have observed, that humans use their faces quite effectively,
send [_] to signal the parameter settings of their operating systems.
send [_] And that they are very good at decoding the meanings of each other's faces.
delay 1000

send [_] So, how do they [d"uw] that?
send [_] Well, look at the face of our ["aarthahr "ehlzahnaar].
send [_] What does it tell [q] us about his internal state?
delay 600

send [_] Not much, you might think. But now, [w"eyt] a moment.
delay 550
Show3
Body G
**G**
delay 3
Body O
**O**

Sad face


delay 200
send [_] You see? Arthur is [s"aed], is what people say, when they see a face like this.
send [_] So what is going ["aon] here? What I [d"ihd] is,
send [_] I sent [axn] electrical signal to two particular muscles,
send [_] in the face of our ["aarthahr "ehlzahnaar].
delay 1000
send [_] These muscles have sometimes been called the Muscles of Sadness.
send [_] There is one on the left, and one on the right.

Left sadness Right sadness

send [_] They usually operate together.
send [_] If I stop the signal, the sadness stops.
send [_] When I turn it ["aon] again, it [st"aarts] again.
text **sadness starts **
body X
delay 50
body G
delay 3
body O
delay 250
body X
delay 50
body O
delay 150
body X
delay 150
body G
delay 100
body X
delay 75
body O

send [_] By sending this signal to Arthur's muscles,
send [_] I simulate what Arthur's brain would do,
send [_] if Arthur's operating system would be running global belief revision processes,
send [_] that are killing a lot of other active processes,
send [_] involving a large number of conflict-resolutions,
send [_] and priority [r`iy-ahs'aesmahnts].
delay 100
body X
delay 100
body G
delay 75
body X
delay 50
body O
delay 75
body X
delay 75
body G
delay 50
body X
delay 25
body O
delay 50
body X
delay 50
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 15
body O
delay 25
body X
delay 25
body G
delay 15
body X
delay 3
body O
delay 3
body G
text **sadness freezes **
delay 50
send [_] The intensity of the signal,
send [_] is proportional to the amount of destructive global belief revision,
send [_] that is going on.
delay 500
body X
text **sadness stops **

Blank face

send [_] For instance, now I have set the signal intensity to 0 again. Arthur is not sad.
delay 300
send [_] Now, we put a relatively small signal, about 20 Volts, on the muscles of sadness.
delay 300
body 1
**1**

Tinge of sadness

delay 100

send [_] Arthur feels a tinge of sadness.
delay 150

send [_] Now, a somewhat larger signal, about 25 Volts.
delay 200
body 2
**2**
delay 50
send [_] Arthur's sadness starts to get serious.
delay 215
send [_] Now I [ihnkr'iyz] the signal once more.
delay 200
body 3
**3**
delay 50

Miserable face

send [_] You see? Now the signal is about 30 Volts, and Arthur feels really miserable.
delay 500
send [:ra 120] This is what we call [ehkspr"ehshahn].
delay 200
send [:ra 150] By means of this mechanism, the face displays clear indications,
send [_] of the settings of virtually all system parameters,
send [_] that determine the operation of the human mind.
send [_] These parameter settings are what humans call [iym"owshahnz].
delay 900
send [_] They denote them by means of words like [s'aednaxs], joy,
send [_] boredom, tenderness, love, lust, ['ehkstaxsiy], aggression,
send [_] [ihriht'eyshahn], fear, and pain.
delay 100
[/* eerst vrolijke grijns (in twee stappen) en dan "aggression" */].

body X
**X**
delay 125
body D
delay 3
body L
**DL**
delay 125
body B
delay 3
body J
**BJ**
delay 125
body X
**X**
delay 125
body C
delay 3
body K
delay 3
body H
delay 3
body P
**CKHP**
send [_] These parameter settings, determine the system's [ihnt'axrpraxtihv b'ayaxsihz],
send [_] its readiness for [q] action, the allocation of its computational resources,
send [_] its processing speed, [ehts'ehtaxraa].
delay 700
send [_] The French [n`uwrow-fihsiy'aolaojhihst] [duhsh'ehnn dax buwl"aonyax],
send [_] has pointed out, that even the most fleeting changes in these parameter settings,
send [_] are encoded [ihnstahnt'eyniyahsliy] in muscle contractions on the human face.
send [_] And ["aol] humans do this in the [s"eym] way.
delay 1100
send [_] This is [axn] extremely interesting feature of the human ['ihntaxrfeys hx'aardwaer],
send [_] which I will explore a little further now.
send [_] So let us get back to the first slide.
delay 700
body X
**X**

Blank face

send [_] This face, which we thought was un-expressive,
send [_] was in fact quite meaningful. This is what we call a [bl"aenxk] face.
send [_] A blank face is a face in its neutral [pahz'ihshahn].
send [_] It indicates that all parameters have their default settings.
delay 1050
send [_] But almost all parts of a human face can be moved to other [pahz'ihshahnz],
send [_] and these displacements indicate rather precisely,
send [_] to what extent various parameter settings [dayv'axrjh] from their defaults.
delay 600
send [_] So let us consider these parts in more detail.
delay 240
send [_] When we look at a human face,
send [_] the first thing we notice is the thing that notices ["ahs]. The eyes.
send [_] The eyes constitute a very sophisticated stereo-camera,
send [_] with a built-in motion-detector,
send [_] and a high-band-width parallel ['ihntaxrfeys],
send [_] to a powerful pattern-matching algorithm.
delay 1300
send [_] The eye-balls can roll, to pan this camera.
send [_] The eyes are protected by eye-lids and eye-brows.
send [_] The eye-brows are particularly interesting for our discussion,
send [_] because their movements seem to be purely expressive.
delay 150

Left eyebrow Right eyebrow Two eyebrows

body A
delay 3
body I
**AI**
delay 225
text **altloop starts**
body X
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
body B
delay 3
body j
delay 25
body b
delay 3
body J
delay 25
send [_] They may indicate, for instance, puzzlement, curiosity, or [dihsaxgr"iymaxnt].
send [_] But I want to emphasize here,
send [_] that the range of parameter values that the eyebrows can express,
send [_] is much more subtle than what the words of language encode.
body X
delay 3
body B
delay 3
body J
**XBJ**
delay 200
body X
delay 3
body C
delay 3
body K
**XCK**
delay 200
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
body X
delay 3
body J
delay 3
body C
delay 25
body X
delay 3
body B
delay 3
body K
delay 25
text **altloop freezes **
send [_] The shape and [paoz'ihshaxn] of a person's eyebrows,
send [_] encodes the values of 5 different cognitive system parameters,
send [_] ["iych] with a large range of possible values.
send [_] Let me demonstrate [thr"iy] of them.
delay 780
body X
text **alt freeze stops **
delay 50
body 4
**4**
send [_] First I put a slowly increasing signal, on the muscles called [fraant'aalihz],
send [_] or Muscles of Attention.
delay 400

 

body 5
**5**
delay 100
body 6
**6**
send [_] We see that this muscle can lift the eyebrow to a considerable extent,
send [_] also producing a very pronounced [k"ahrvaxtyahr] of the eyebrow.
send [_] As a side-effect of this motion,
send [_] the forehead is wrinkled with curved furrows,
send [_] that are [kaons"ehntrihk] with the curvature of the eyebrow.
delay 1100
send [_] The contraction of this muscle indicates a person's readiness to receive new signals,
send [_] and the availability of processing power and working memory,
send [_] for analysing these signals.
delay 700
send [_] Then, I will now stimulate a part of the [q'aorbiykuwl`aarihz q"owkuwliy],
send [_] that is called the Muscle of Reflection.
delay 300

Reflection

body X
delay 3
body C
delay 3
body K
**XCK**
send [_] We see now that the whole eyebrow is lowered.
send [_] As a result, the wrinkles in the forehead have disappeared.
delay 500
send [_] This muscle is contracted if there is [axn] ongoing process,
send [_] that takes up a lot of a person's computational resources.
send [_] To prevent [ihnaxrf'iyraxns] with this process,
send [_] input signals are not [ehgz'aostihvliy] analysed.
send [_] The degree of contraction indicates,
send [_] to what extent the input signal throughput is reduced.
delay 1300
send [_] Then, there is another part of the [q'aorbiykuwl`aarihz q"owkuwliy],
send [_] that can be triggered separately. It is called the Muscle of Disdain.
send [_] Its contraction looks like this.
delay 525

Disdain

body X
delay 3
body A
delay 3
body I
**XAI**
send [_] The contraction of this muscle indicates,
send [_] to what extent current input is ignored as being [ihr'ehlaxvaxnt].
delay 500
send [_] Of course, non-zero values for these system-parameters may be combined,
send [_] and these values may be different for the left and the right hemispheres.

text **wave starts **
body X
delay 3
body J
delay 25
body K
delay 25
body C
delay 3
body j
delay 25
body B
delay 3
body k
delay 25
body c
delay 25
body b
delay 25
body B
delay 25
body C
delay 25
body K
delay 3
body b
delay 25
body J
delay 3
body c
delay 25
body k
delay 25
body j
delay 25
text **wave freezes **
send [_] Now let us look at the [m"awth-piys] of our ['aarthahr 'ehlzahnaar].
body X
text **wave stops **
send [_] The mouth is a general intake organ,
send [_] which can swallow solid materials, liquids, and air.
delay 100
body H
delay 3
body P
**HP**
send [_] In order to monitor its input materials,
send [_] the mouth has a built-in chemical analysis capability.
delay 500
body X
**X**
send [_] At the same time the mouth is used as [axn 'awtleht], to expel processed air.
send [_] Because humans do not have [l"awd-spiykaxrz],
send [_] they use this process of expelling air for [jh'ehnaxr`eytihnx] sounds.
delay 800.

Mouth open

body H
delay 3
body P
**HP**
send [_] In emergency circumstances,
send [_] the mouth can also be used as [axn 'awtleht] to expel blood, [m'uhkahs],
send [_] rejected food, or other ['ahnwaontihd] substances.
delay 520
send [_] When the mouth is not used for input or output,
send [_] it is normally closed off by a muscle, which is called the [l"ihps].
body X
**X**

Lips closed

delay 400
body E
delay 3
body M
**EM**
delay 100

send [_] The lips have a large repertoire of movements.
send [_] There are at least [s"ihks] other muscles, that interact directly with the lips.
send [_] I will now demonstrate [f"aor] different movements.
delay 550
body X
**X**
send [_] First we show the muscles of joy.
delay 180
body D
delay 3
body L
**DL**
send [_] These muscles produce a kind of grin.

Grin

send [_] They signal, that the operating system is in good working order,
send [_] and is not encountering any problems.
send [_] There is heightened activity, in the left frontal lobes of the brain.
delay 890
send [_] When, on the other hand, the activity in the left frontal lobes is unusually low,
send [_] the brain is involved in destructive processes of global belief revision.
send [_] As we saw [biyf'aor], this is signalled by another pair of muscles,
send [_] called the muscles of sadness. Here they [q] are, once more.
body X
delay 600
body G
delay 3
body O

Sadness

*GO*
delay 500
send [_] And finally I will now trigger several muscle pairs at the same time.
send [_] [q'aorbiykuwl`aarihs q"owrihs],
send [_] and [diypr'ehsaor laabiyiy-iyiy q`iynfeyriy"owrihs],
send [_] and the Muscle of Contempt, and the Muscle of Sadness.
delay 310
body F
delay 3
body N

**FN**
delay 430
send [_<500>] The parameter-setting that is displayed here,
send [_] clearly indicates serious processing difficulties of some sort.
delay 450
body X
**X**
send [_] O.K. [_<800>] Then we have the [n"owz]. [_<300>]
send [_] The nose is used for the intake of air.
send [_] It is also equipped with a chemical analysis capability.
send [_] The possible motions of the nose are curiously limited,
send [_] although its pointed [pr'ehzaxns] in the centre of the human face,
send [_] would make it a very suitable instrument for expression.
delay 1250

send [_] I have [th"aot] about this, and I have come to the conclusion,
send [_] that it is probably the main function of the nose,
send [_] to serve as a stable orientation point for our perception,
send [_] so that [saymahlt'eyniyahz] movements of the other parts of the face,
send [_] can be ['ahnaemb`ihgyuwaxzliy] measured and interpreted.
delay 500
text ** nose-loop starts **
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40

body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G

send [_] And finally, for the sake of completeness, I want to mention the [q"iyrz],
send [_] on both sides of the face, which constitute [axn] auditory stereo input device.
send [_] Some people can [w"ihglx] these ears, but I have not been able to determine,
send [_] what the expressive function of that movement might [b"iy].

body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
delay 3
body O
delay 40
body X
delay 3
body L
delay 40
body X
delay 3
body J
delay 40
body X
delay 3
body B
delay 40
body X
delay 3
body D
delay 40
body X
delay 3
body G
delay 40
body X
text ** nose-loop stops **
delay 160

send [_] This brings [axn] end to my quick survey,
send [_] of the most important parts of the human face,
send [_] and their expressive possibilities.
delay 400

send [_] Many of the possibilities I showed, were related to emotions,
send [_] that are well recognized in the lexicons of many human languages.
send [_] Cause I have mostly demonstrated [m'ehn-taxl] states which were close to neutral,
send [_] where only one parameter had a non-default value.
delay 1000

send [_] But now, look at what happens,
send [_] if we signal non-trivial values for different cognitive dimensions at the same time,
send [_] by [saymahlt'eyniyahz] contraction of different facial muscles.
text ** funny faces start **
body X
delay 3
body E
delay 3
body M
delay 100
body F
delay 100
body N
delay 100
body e
delay 3
body m
delay 100
body A
delay 40
body I
delay 50
body n
delay 100

body f
delay 100
body L
delay 100
body D
delay 100
body P
delay 100
body p
delay 45
body i
delay 30
body K
delay 100
body l
delay 100
body O
delay 100
body P
text ** funny faces freeze **
send [_] You see what happens now.
send [_] Every human person knows [ehgz'ehktliy], in what state another human person is,
send [_] when they make a face like [dh"ihs].
send [_] Cause they know what state [dh"ey] would be in,
send [_] if [dh"ey] would make a face like this.
delay 700
send [_] Now I hope you all agree that it would be a good idea, if computers could
send [_] take advantage of this magnificent [hx'aardwaer] as well.
send [_] Cause [dh"ehn], any human would be able to understand, with just one glance,
send [_] the internal state of a computer.
delay 700
body X
text ** funny faces stop **

send [_] So if humans are not afraid of wiring themselves ["ahp] with computers,
send [_] we can now make important progress in computer ["ihntaxrf`eys] technology.
send [_] So I would now like to end my talk, with this conlusion.
send [_ :ra 120] The next step in computer ["ihntaxrf`eys] technology,
send [_] will be the [hxy"uwmaxn] face.
delay 1000

send [_] I have been very grateful
send [_] for this opportunity, to speak my mind to such [axn] attentive ['aodiyahns].
send [_] I would now like to thank my [ "aarthahr "ehlzahnaar] for his patient co-operation,
send [_] and I want to thank you all for your attention.
send [ _<2000> th"aenk] you!
body A
delay 3
body I
**XAI**
delay 1200
body X
**X**