Download Hierarchical Control in the Execution of Action Sequences and more Schemes and Mind Maps Ethology in PDF only on Docsity!
Hierarchical Control in the Execution of Action Sequences: Tests of Two Invariance Properties Saul Sternberg, Ronald L. Knoll, & David L. Turock
3 Abstract 3 Introduction 4 Properties of Sequences Under Hierarchical Control 4 Concepts of Hierarchy 6 Two Invariance Properties of Hierarchically Controlled Sequences Augmenting of hierarchical structure with hierarchical control. Low-Ievel.and high-level invariance as properties of hierarchical control. The testing of invariance. Command/signal transmission as an alternative to transfer of control. 9 '" .Fundamental Discreteness of Continuous Action 10 Alternative Criteria for Action Units 12 Existing Evidence for Hierarchical Control in Sequence Production 14 Tests of Low-Level Invariance 14 Existence of an Action Unit in Rapid Speech Production 18 Further Analysis by Segmentation of Single Speech Units 20 '" .Tests in Keystroke Sequences Containing Repeated Pairs of Distinct Strokes Evidence of hierarchical control in strings of eight finger taps. Modifications and elaborations in a keystroke experiment. Results of the modified experiment. Two tests of low-level invariance. Application of the binary-hierarchy test. 28 .... Test of Constituent Invariance in Strings of Distinct Strokes Assumptions for restricted and general tests; expected linkage proportions. Observed linkage proportion: Upper bound. 34 .... Further Analysis by Segmentation of Single-Stroke Finger Trajectories 39 '" .Invariance Test of Embedded Doublets 41 .... Summary of Evidence For and Against Low-level Invariance 43 Tests of High-Level Invariance 43 Sequence Duration Under Singlet-Doublet Exchange 46 Serial-Position Effect Under Singlet-Doublet Exchange 47 Joint Tests of High-Level and Low-Levellnvariance 49 Effects of Utterance Length Under Speech-Unit Exchange 49 Effects of Number of Strokes Under Keystroke Exchange 50 Summary of Evidence For and Against High-Level Invariance 50 Conclusions 51 Acknowledgements 52 References
Association Lecture of the International Association for the Study of Attention and Performance, Arc-et-Senans, France, June 1988. In M. Jeannerod (Ed.) Attention and Performance XIII. Hillsdale, N. J.: Erlbaum, 1990. Pp. 3-55.
Hierarch ical Control in the
Execution of Action
Sequences: Tests of Two
Invariance Properties
Saul Sternberg
University of Pennsylvania
Ronald L. Knoll
AT&T Bell Laboratories
David L. Turock
Bell Communications Research
ABSTRACT
What might it mean for execution of an action sequence to be controlled hierarchically? We argue that if production of a sequence consists of the execution of nested constituent subsequences, then it should be characterized by two invariance properties-properties that limit the effects of one part of the sequence on another. Because one such constituent structure merely partitions the stream of action into action units, these properties have wide applicability. According to low-level in variance , the process that executes a constituent should not be influenced by changes in any higher level constit uent. According to high-level invariance , changes in a constituent should have at most limited and local effects on higher level constituents. We report on tests of these two properties in the rapid production of brief utterances and short strings of keystrokes, in which we examine the effects of sequence length, serial position, and unit size on measures of timing. The tests support the existence of hierarchical constituents at the level of the stroke in typing and the stress group in speech, but provide only limited evidence for deeper hierarchical structure.
INTRODUCTION
In this chapter we investigate one sense in which execution of action se quences might be hierarchical, by virtue of their being composed of sep arately controlled constituent subsequences. We argue that to claim merely that a continuous stream of action can be decomposed into a string of
3
1. HIERARCHICAL CONTROL OF ACTION SEQUENCES 5
sisting of a set of elements (nodes) at different levels, partially ordered by a relation (branches), usually antisymmetric and transitive (Wall, 1972). Examples of relations are inclusion and control. If the relation is inclusion, we have a classification hierarchy in which each class (at one level) consists of a set of subclasses (at the next), and a subclass can belong to only one class. If the relation is control, each element (at one level) controls a set of elements (at the next), and an element can be controlled by at most one higher level element. In applying this idea to action sequences we are interested in strings of rapid actions that can be said to contain one or more separately controlled substring constituents (or units), larger than a "single" action (that is, susceptible to further analysis), but smaller than the whole string (Gallistel, 1980, pp. 288-290). The validity of such an assertion hinges theoretically on the specification of criteria that define a substring as a constituent, and empirically on the demonstration that such criteria are satisfied. This sense of hierarchy should be distinguished from the idea of different levels of specificity, such that more detailed aspects of the same action are controlled autonomously at lower levels, closer to its execution (Szentagothai & Ar bib, 1975), and where control branches can converge and cross (Gallistel, 1980). Perhaps the most familiar domain where hierarchically organized se quences have been used to characterize human behavior is language. The constituent structure of sentences is represented in linguistics by the recursively branching phrase marker (Wall, 1972), which we intro duce to add precision to the idea of hierarchical structure. The phrase marker can be expressed as either an ordered tree, (Fig. 1.1A), or a labeled-bracket structure (Fig. LIB). An ordered tree consists of a set of nodes (from a root at the highest level to leaves at the lowest, or first) connected by diverging and non-crossing branches. Any pair of nodes is related either by dominance (for instance, in Fig. 1.1 VP dom inates the right-hand N, and is thus at a higher "level") or by prece dence (for instance, the upper NP precedes V) but not by both. Each node corresponds to a substring constituent of the sentence. The nesting of constituents is more evident in the labeled-bracket structure, in which dominance becomes inclusion. A hierarchically organized string may thus be defined as a set of con stituent (sub)strings, partially ordered by inclusion, such that any string either fully contains, or is fully contained in, or is disjoint from any other string, and such that disjoint constituents are temporally ordered. A string of "action unit" substrings defines the most shallow structure that can be described as hierarchical; in a deeper hierarchy the substrings would be further partitioned into smaller disjoint substrings.
6 STERNBERG. KNOLL. AND TUROCK
A s
NP VP
/\ /\
Art N V^ NP
A
Art N
I I
The reader likes^ the^ analogy
B
( (The reader) (^ likes^ (the analogy)))
S NP VP NP
FIG. 1.1. Two forms of the phrase marker representation of a sentence. A: An ordered tree composed of nodes and branches. B: A partially equiv alent structure of nested labeled brackets.
Two Invariance Properties of Hierarchically Controlled Sequences
Augmenting of hierarchical structure with hierarchical control. The phrase
marker is a hierarchical structure with no commitment to any particular embodiment in real time or to any specification of the flow of control. To convert such a structure into one that can actually control the execution of the sequence it describes, we must augment it with an interpretive
8 STERNBERG. KNOLL, AND TUROCK
the label of that node, and not on any other nodes. Low-level invariance, or an approximation thereof, is inherent in the idea of a unit, an entity that must preserve some or all of its properties across different contexts. The second consequence is high-level invariance: relative to changes in a constituent substring, the higher level process that controls its execution along with the execution of disjoint other constituents, and hence those other constituents themselves, should be invariant. Changes in the content of a constituent should thus have only local effects (limited to the constit uent itself) on the higher level constituents of which it is a substring. An analogy in language is the existence of syntactic dependence, such as num ber agreement between subject and verb, that is unaltered by expanding an intervening constituent and thus increasing their separation. It might be argued that these invariance criteria are too stringent; the hypothesis of hierarchical structure should perhaps not be rejected in every instance in which one or the other criterion is found wanting. We might instead ask to what degree of approximation these properties obtain, and for which aspects of performance, in cases where we have other reasons to argue for hierarchical structure. Thus, Simon (1962) suggests that hi erarchical systems might be "nearly decomposable," rather than displaying full invariance. On the other hand, it seems reasonable to start with the expectation of full invariance, and discover how far it will carry us.
The testing of invariance. To test either form of invariance one assesses the effects of manipulating an experimental factor on some measure. To provide an interesting test, the factor should influence some aspect of the performance under study. For this reason we refer to a factor used in this way as an influencer. Although the invariance properties apply to any aspect of performance, our tests to date are limited to duration measures. To emphasize this limitation we sometimes use the term duration invariance.
Command/signal transmission as an alternative to transfer of con trol. Strict interpretation of "transfer of control" requires that there be no temporal overlap of operations associated with different levels (Broad bent, 1977). Durations of operations at different levels must then contribute additively to sequence duration, and time measurements become especially revealing. Given such a process, however, we must reconcile the apparent continuity of action not only with the discreteness of the action unit, dis cussed later, but also with the possibility of time-consuming operations at high levels, which (on this account) should lead to epochs during which action is frozen: It may thus be necessary to at least augment a process of transfer of control such that action controlled by a low-level subprogram can outlast the execution of that subprogram, filling the gap during which
- HIERARCHICAL CONTROL OF ACTION SEQUENCES 9
the next subprogram is being selected and invoked, but without delaying its execution (Sternberg, Knoll, Monsell, & Wright, 1989, Section 4.3). These considerations, together with aspects of coordination of the hands in typewriting described later, also suggest that we consider a fundamen tally different class of control processes that might operate on a hierarchical structure, in which transfer of control is replaced by command/signal trans mission: A program invokes a subprogram by transmitting a command downward rather than relinquishing control; completion (or forthcoming completion) may be indicated by an upward signal, but return of control is not needed. Processes at different levels, or even at the same level, may thus overlap in time and proceed concurrently. Our invariance properties should still obtain, but their consequences may be obscured, depending on the extent of temporal overlap and other details; the interpretation of time measurements may be more complicated. If higher level operations must await a signal of completion from below, such a process could mimic transfer of control.
Fundamental Discreteness of Continuous Action
Action is in general continuous; even when it is punctuated by a series of distinctive time markers, such as keystroke actuation points in typing or articulatory closures in speech, the system is unlikely to be static during intervals associated with these events, nor do we wish to limit our analyses to those cases where it might be. Given such continuity, it is well to bring out a common but often implicit assumption: The apparently continuous action stream is fundamentally discrete. Without this assumption we could not readily consider the planning of action sequences, and we would not be attempting to identify action units. The idea is commonplace in thinking about speech production, which is often thought to have phonemes as low level units, with their concatenation smoothed by coarticulation (Fowler, 1985; Perkell, 1980). In relation to manual movement trajectories, espe cially in handwriting, Morasso and others have shown how an inherently discrete and discontinuous mechanism can produce smooth movements by a particular form of coarticulation: the partial time overlap of consecutive units, achieved by superposition of their kinematic characterizations. 3 If such smoothing were produced by a process that operated on the output of a hierarchical control mechanism, then the resulting perturbations at constituent boundaries might obscure instances of expected invariance; If
3S ee Morasso (1983) and Morasso, Mussa Ivaldi, and Ruggiero (1983). In an experimental test using planar arm-movements, such superposition failed to describe the resultant trajectory (Massey, Schwartz, & Georgopoulos, 1986).
1. HIERARCHICAL CONTROL OF ACTION SEQUENCES 11
- Equivalence of effects of units on other actions. Any effect on the remainder of a sequence of adding a unit should be independent of the size or content of that unit (Sternberg, Knoll, et al., 1989). Note that this is equivalent to high-level invariance, according to which one constituent can be replaced by another, with no effect on the remainder of the control structure.^5
- Equivalence of effects of other actions on units. An effect on a unit of lengthening a sequence should be independent of the content of that unit (Sternberg, Knoll, et al., 1989). Note that although this requirement appears at first glance to be equivalent to low-level invariance, it is weaker: Low-level invariance requires no effect of changing the embedding struc ture, rather than merely an invariant effect.
- Preferential influence. Segments within the same unit should influ ence each other more than segments of different units. Fowler (1981, 1985) uses this criterion to support the idea of a stress-group unit in speech production. In a perceptual analog, Palmer (1977) proposes that elements within units of a hierarchically organized percept have relatively more influence on each other. Simon (1962) describes hierarchical systems as "nearly decomposable," displaying stronger interactions within than be tween units.
- Internal coherence. Parts of a unit should be difficult to separate. Segments within the same unit should thus never (or seldom) participate independently in errors of omission or misordering (Drummond, 1981; Fentress, 1983; Fromkin, 1981; Gallistel, 1980), and it should be more difficult to produce a prescribed rearrangement of a prepared sequence that breaks up units than one that leaves them intact (Gordon & Meyer, 1987).
- Process discreteness. A unit is a segment of performance that results from a single rendition of some process. This criterion is, of course, more model-dependent than the others because in general, a process is hypo thetical and requires a model to be related to observables (Sternberg, Knoll, et al., 1989, Section 2.5).
- Bell-shaped velocity profile. In arm and hand movements, a unit can be distinguished by a bell-shaped velocity profile, and is thus a segment of the trajectory delimited by velocity minima (Abend, Bizzi, & Morasso, 1982).
5The equivalence can be seen as follows: Suppose equal effects on some univariate property P (^) a of sequence a, of adding (concatenating or incorporating) constituents U (^) x and U (^) y • That is, Pa(a+uJ - Pia) = Pia+uy) - Pia). It follows that Pia+ux) = Pa(a+u y), so that replacing Ux by u (^) y leaves Pa invariant. Conversely, it can be seen that invariance under replacement implies equal effects.
12 STERNBERG, KNOLL, AND TUROCK
Existing Evidence for Hierarchical Control in Sequence Production The idea that fluently performed action sequences tend to be structured hierarchically, and that skill learning includes the creation of higher-level action units, dates back at least to the turn of the century (Book, 1908). However, there is a disparity between the enthusiasm for the hypothesis of hierarchical organization in the skilled execution of action sequences (e.g., Gallistel, 1980, Ch. 12; Johnson, 1970; Keele, 1987; MacKay, 1982; Miller, Galanter, & Pribram, 1960) and the persuasiveness of much of the cited evidence. For example, contrary to frequent citation, Bryan and Harter's (1899) claims were for hierarchical structure in receiving, rather than sending Morse code. Other evidence does not exclude a perceptual locus (Leonard & Newman, 1964). The often-cited conclusions of Book (1908) about multiple-stroke units in typewriting depend exclusively on introspective evidence. Furthermore, they have been only partially con firmed by the effects on transcription speed at different skill levels of the length of familiar sequences incorporated in the copy (Fendrick, 1937; West, 1969, Ch. 3).6 More recently, some evidence consistent with hier archical structure of the representation that controls the execution of action sequences has begun to emerge: Gordon and Meyer (1987) have applied the internal coherence criterion listed in the previous section to speech production, and Rosenbaum et al. (1983), whose work we discuss later, have applied the temporal separation criterion to finger tapping (but see also Klein, 1983, and Rosenbaum, 1983). Because their problem is the timing of "identical" actions rather than the choice of distinct ones, Vorberg and Hambuch's (1978, 1984) attempts to determine conditions (if any) for hierarchical control in the timing of repeated rhythmic patterns fall outside the domain of central interest here; they are nonetheless important in any review. Suppose a series of taps is produced under instructions to make them' approximately isochronic but to group them in triple meter. Denote the times of occurrence of each of the successive taps as
According to a serial-timing model, each tap is timed from the preceding one. In a typical hierarchical-timing model, the timing of one or more
6Inferences in these studies about the size of the action unit depend on two questionable assumptions: First, skill level is assumed to have no influence on the perceptual or translation processes in transcription; second, given no change in unit size, skill level is assumed to have a multiplicative effect on transcription speed for different kinds of copy.
14 STERNBERG, KNOLL, AND TUROCK
our tests thus far are restricted to the duration invariance of elements and sequences.
TESTS OF LOW-LEVEL INVARIANCE
Existence of an Action Unit in Rapid Speech Production
The analysis we initially present sets the stage for some of the others and also leads to an important elaboration of the idea of low-level invariance. Subjects were asked to recite word sequences of different composition and length, specified well in advance of a "go" signal, so as to complete the utterance as soon as possible (Sternberg, Knoll, et al., 1989). The utter ances were either simple counting sequences of lengths one through five ("one, two" or "one, two, three, four," for example; digit condition), or the same strings with and interpolated ("one and two and three," for example; and-digit condition) or the same strings with minus interpolated ("one minus two minus three," for example; minus-digit condition). Within each condition one can regard the strings of increasing length as being generated by starting with one and adding elements of a particular "size," the elements being digit, and digit, and minus digit, for the three conditions, respectively, where digit represents a digit name and receives a primary stress. Rather than increasing linearly with the number of elements, n, the durations of utterances produced with this procedure (sequence durations) are usually well fitted by a quadratic function:
Dn = & + ~n + "in(n - 1). (1)
The nonlinearity reflects the fact that longer sequences are produced at lower rates. We adjust for end effects (such as prepausal lengthening) by forming Dn - &, divide by number of elements, n, and add "i, to arrive at the mean element duration: 8 D - & d. (^) n = n + "i = ~ + "in. (2) n
That is, the mean time between corresponding points of successive elements increases linearly with the number of elements in the utterance (utterance length), by an amount given by the parameter "y. Quadratic functions also
8Duration of an element here includes the durations of all events that occur from its beginning to the beginning of its successor or (for the last element) to the end of the utterance. The addition of 'Y is unimportant for our present purposes; it is explained in Sternberg, Knoll, et al. (1989).
1. HIERARCHICAL CONTROL OF ACTION SEQUENCES 15
account well for the durations of sequences of n distinct keystrokes in typing (Sternberg, Monsell, et al., 1978) where a similar analysis also leads to a linear function of n for mean element duration. 9 Next we consider the relation between the arbitrary element, (such as the word or syllable) in terms of which we design experiments and analyze the data, and the action unit, a theoretical entity. We do so here because the result enables us to develop an argument for the existence and identity of an action unit in these rapid utterances: Decomposition of a sequence into units defines a minimal hierarchical structure, and thus permits a test of low-level invariance. Suppose there exists a unit of speech production, an entity that is defined by the scope of an execution process and that is invariant across utterances of different lengths and compositions. We define the unit period as the time from the beginning of one unit to the beginning of the next; note that the unit period includes both the time to execute the unit (a low-level process) and the time to invoke its successor (a higher level process). Now consider the effects of increasing the number of equal size elements in an utterance. Suppose the following equivalence require ment for action units: Regardless of the elements comprising the utterance or of the size of the unit (e.g., the number of phonemes it contains) mean unit period is either invariant or is increased by the same amount per added unit. (If unit execution time is itself altered, we have a failure of low-level invariance. If any increase in unit period can be identified with the higher level process, however, then low-level invariance applies to unit execution time itself, and the requirement of a hierarchical control structure is sat isfied. The invariance relative to unit size expressed by the equivalence property is then an expression of high-level invariance.) We wish to de termine whether there exists such a unit and, if so, to identify it. Let the entities we designate as the elements of a particular utterance each contain k such units (k > 0). Let 8 be the mean increase in unit period per added unit. Then the mean increase in unit period per added element (i.e., per added k units) is k8, and the mean increase in element duration (i.e., in the summed periods of k units) per added element, or the slope of the element-duration function, is k(k8) = k28. (For example, suppose
the unit is the syllable and elements are disyllabic words, so that k = 2.
Let 10 msec be the mean increase in the time from the beginning of one
syllable to the next, per added syllable. Because k2 = 4 the mean increase
9For typewriting (see Fig. 1.11) we use a slightly different parametrization of the quadratic function [Dn = ex + l3(n - 1) + "(n(n - 1)] and definition of mean element duration [d. (^) n
= (Dn - ex) I (n - 1) = 13 + "(n]. These differences are dictated by an interpretation of
the duration of an utterance as running approximately from the beginning of the first element to the end of the last, whereas in those typing experiments where only strokes (taken to be the ends of elements) are recorded, we consider that sequence duration runs approximately from the end of the first element to the end of the last.
1. HIERARCHICAL CONTROL OF ACTION SEQUENCES 17
IELEMENTI
a..^1 ImirlJs^ 21··· 0 ::J^280 C'l^ l- 260 en en Q)l (^240)
..... en
'-u 220 en Q) E^200 z^180 0 160 ~ (^) I 0:::: (^140) ::::> 0 t- 120 z w 100 1 ~^ ••• ~ w ...J 80 W z 60 <! w (^40) ~ 1:' I^ 'J 1 234 5 NUMBER OF STRESS GROUPS, n FIG. 1.2. lV1ean element-duration functions based on stress-group ele ments for control utterances (digit condition) and for utterances with in terpolated unstressed words (and or minus), with estimate of ± Sf and fitted linear functions: 58 + 8.0n (digit condition); 151 + 6.2n (and-digit condition); 235 + 8.7n (minus-digit condition). Results are averaged over four subjects and three days; about 140 (70) observations per point for control (other) conditions.
equality of the nonlinear part of the sequence-duration function (Equation 1). Sequence-duration functions for utterances containing units of different size can then differ by at most a linear function. We see later how this constraint follows from our two invariance properties. For our present purposes the main point is that we have a set of data that is accounted for very nicely by a particular definition of the action unit. Having identified a unit, we now ask whether that unit functions as a constituent in a hierarchical structure. We first consider whether it obeys low-level invariance; later we discuss a test of high-level invariance, as units of different size are exchanged.
18 STERNBERG, KNOLL. AND TUROCK
Further Analysis by Segmentation of Single Speech Units
For the test of low-level invariance, the property of the unit that we consider is its mean duration; the influencer is the number n of units in the utterance. It is clear that if we identify unit duration with unit period, as plotted in Fig. 1.2, we have a violation of low-level invariance. The three approxi mately parallel functions indicate two effects on unit period, effects that combine additively. First, the vertical displacement of the three functions shows, unsurprisingly, that the size of the unit-the number of phonemes or syllables it contains-has a substantial effect on its period. That this effect is independent of n suggests the possibility that unit execution time, a component of the unit period, might be invariant relative to sequence length. Second, the fact that instead of being flat, the element-duration functions rise systematically, shows that the number of units in the utterance also has an effect on unit period. Here we have what appears initially to be a clear violation of low-level invariance: a characteristic of a purported hierarchical constituent depends on its context. However, the analysis of how the effect of utterance length on unit period is distributed within that period provides a means by which low-level invariance can be preserved. In two experiments conducted under conditions similar to those already described (Sternberg, Wright, et al., 1980; Sternberg, Knoll, et al., 1989), utterances were composed of words selected such that segmentation based on the acoustic signal would be feasible: Words were initially stressed disyllables (e.g., copper, token) in which each syllable starts with an artic ulatory closure. Each word in the second experiment (1989) was decom posed into 6 segments, corresponding roughly, for each syllable, to the closure, release, and vowel: C1, Rl, VI, C2, R2, V2. The upper panel of Fig. 1.3 displays the element-duration function from that experiment, com parable to the functions shown in Fig. 1.2, here with a slope of 11 msec/ word. The lower panel shows the segment-duration functions, which in dicate how the effect of the number of units in the utterance is distributed within the stress-group unit. ll^ Rather than pervading the unit, the effect of utterance length is almost entirely restricted to V2, the vowel of the second syllable; durations of the 5 other segments are virtually invariant. It is possible that the second vowel is prolonged until higher level control structures have invoked the subprogram for the subsequent unit. Low-level invariance is required of the time to execute a subprogram, not of the time to invoke it. We thus have what appears to be a qualified form of low
liThe segment-duration function for segment i (here, 1 :=:; i :=:; 6) is obtained by starting, for each n, with the summed durations of the n ith segments in utterances of length n, and using the resultant set {D"J to determine a set of n mean segment durations {d."J for the ith segment, just as utterance durations {D,,} are used to determine mean element durations {d.J. The observed and fitted element-duration functions are then sums of the observed and fitted segment-duration functions, respectively.
20 STERNBERG. KNOLL, AND TUROCK
level invariance which, on reflection, is what we might have expected: Given that speech is relatively continuous, that the time required to invoke a subprogram may vary with sequence length (because length influences the control structure), and that smoothing occurs to provide for continuity, not all speech segments would be expected to demonstrate duration in variance. That the effect of utterance length is seen in one epoch per stress group (indicating one invoking process per stress group) is consistent with the conclusion about unit size based on properties of the mean element duration function. The unit period thus appears to include two additive components, one attributable to the unit and dependent on unit size alone, and the other attributable to higher level control operations and dependent on number of units. A difficulty remains, however. Although we find approximate duration invariance relative to sequence length for 5 of the 6 segments of the stress group unit when the mean is taken over serial position, low-level invariance also requires that these segment durations be invariant over serial position; and this is not uniformly the case: instead, serial position influences the duration of VI, the first syllable's vowel, which tends to increase with serial position (Sternberg, Wright, et al., 1980). One possible way to rescue the invariance property would be to argue that prosodic effects-including effects of serial position on timing-are introduced after the hierarchical control structure has done its work, so as to modulate its output. 12 Low level invariance would then characterize a hypothetical representation that antedates the action sequence itself. With this elaboration of the hierar chical control model we turn our attention away from speech production and toward sequences of finger movements.
Tests in Keystroke Sequences Containing Repeated Pairs of Distinct Strokes
Evidence of hierarchical control in strings of eight finger taps. Rosenbaum, et al. (1983) have provided what appears to be persuasive evidence of hierarchical control in the production of patterned strings of finger taps. They studied the performance of strings of eight taps made by the fingers on alternating hands and containing repeated pairs. An example is R 1 , L (^) b
R 1 , L b R 2 , L 2 , R 2 , L 2 , where Rand L denote left hand and right hand,
and subscripts 1 and 2 denote index finger and middle finger, respectively. Subjects were asked on each trial to produce one of a set of such strings repeatedly without interruption and as fast as possible, for six renditions.
12 The idea that the motor system temporally modulates or "interprets" the output of a higher level control system by introducing delays that might have some structure has been suggested for isochronic taps by Wing (1977).
1. HIERARCHICAL CONTROL OF ACTION SEQUENCES 21
Evidence for hierarchical control was derived mainly from this pattern of mean intervals between successive taps: The mean intertap intervals (ITIs) defined by the four pairs of taps 1-2, 3-4, 5-6, and 7-8 (first set) were similar and shortest; they were longer and similar for the two intervals defined by taps 2-3 and 6-7 (second set); and they were longest for the two intervals defined by pairs 8-1 and 4-5 (third set). Furthermore, the increases in mean ITI from first set to second and from second set to third were approximately equal, consistent with a recursive process. The control structure suggested by Rosenbaum et al. is an ordered binary
tree (Fig. 1.4, n = 8) a special case (restricted to binary) of the phrase
marker discussed earlier. Thus, each of the two nodes below the first bifurcation in the tree representation of the eight-tap sequence specifies a four-tap constituent, each of the four nodes below the next bifurcations specifies a two-tap constituent, and each of the eight final, bottom-level nodes specifies a single tap. The tree structure is augmented, for control purposes, by a ("preorder") tree-traversal process (Aho, Hopcroft, & Ull man, 1983) that starts at the top (root) node and proceeds to the lowest level nodes from left to right, in turn. To repeat the sequence the process passes from rightmost node to root, and then proceeds as before. The intervals between successive taps are assumed to increase linearly with the number of^ nodes that must be traversed; the resulting model fits^ the^ mean ITI data reasonably well. (In an isomorphic conceptualization of such a tree and its control process, each node corresponds to a subprogram of the program associated with its parent node, and tree traversal is replaced by a "subprogram-retrieval process" with properties as in Sternberg, Mon sell, et aI., 1978.) Tree traversal is one control process associated with a hierarchical struc ture that operates by transfer of control and requires low-level and high level invariance of the sequences it produces. Thus (low-level invariance) once control is passed to a lower level node the traversal process for that node is independent of the structures connected to it through higher level nodes (from which control was derived earlier and to which it is transferred later); similarly (high-level invariance) the traversal processes in those other higher and disjoint parts of the structure are independent of what the process does in relation to the lower level node. Although strings of taps known to have the suggestive pattern of mean ITIs described above form a very limited set, it is worth testing the invariance properties in such strings, partly because hierarchical control is strongly indicated by the ITI data, and partly because the model advocated by Rosenbaum et al. (1983) requires these properties.
Modifications and elaborations in a keystroke experiment. We therefore proceeded to test low-level invariance in strings with repeated pairs, by modifying and elaborating the experiment of Rosenbaum et al. (1983).
