Research Methods

Test of Spatial Assembly (TOSA)

Instruction Manual

Brian N. Verdine Roberta Michnick Golinkoff*

University of Delaware

Last Revision Date: 04-09-2017

Before citing this manual or in the event that you have questions about the coding, please contact

Brian Verdine ([email protected] or [email protected]) or Roberta Golinkoff

([email protected]).

Development of this test was funded by a National Institutes of Health Stimulus Grant

1RC1HD0634970-01 to Roberta Golinkoff and Kathy Hirsh-Pasek and the National Science

Foundation through the Spatial Intelligence and Learning Center (SBE-1041707).

This manual was used in the following published work. Please cite one or more of these if you use the TOSA. Please also tell us about how you used it, how it worked for you, and ask any questions you might

have about creating or giving the test.

Verdine, B. N., Golinkoff, R. M., Hirsh-Pasek, K., & Newcombe, N. S. (2017). Links

between spatial and mathematical skills across the preschool years. Monographs of

the Society for Research in Child Development, 82(1), 1–150.

https://doi.org/10.1111/mono.12263

Verdine, B. N., Golinkoff, R. M., Hirsh-Pasek, K., Newcombe, N. S., Filipowicz, A. T., &

Chang, A. (2013). Deconstructing building blocks: Preschoolers’ spatial assembly

performance relates to early mathematics skills. Child Development.

doi:10.1111/cdev.12165

Verdine, B. N., Irwin, C. M., Golinkoff, R. M., & Hirsh-Pasek, K. (2014). Contributions of

executive function and spatial skills to preschool mathematics achievement. Journal

of Experimental Child Psychology, 126, 37–51. doi:10.1016/j.jecp.2014.02.012

* We are grateful to the following individuals on our project who assisted in the creation of the

scoring scheme and preparation of the manual for this test: Angeliki Athanasopoulou, ToriAnne

Davies, Kelsey Lucca, Gabrielle Farmer, Andrew Filipowicz, Maya Marzouk, and Jelani

Medford.

TOSA: Manual Contents and Introduction

2

Manual Contents

 Introduction of the TOSA

 Administration Instructions

 Scoring Instructions

 Score Sheets for Data Collection

 Instructions for Creating Stimuli

Introduction

The motivation behind creating the TOSA was to design a spatial measure for very young

children that captured a broad spectrum of spatial skills and that provided enough variability in

the age group to study individual differences in spatial skill at very young ages. The scoring

system for the TOSA allows children to receive “partial credit” for preserving spatial

relationships between pieces, creating a wider range of scores that capture more meaningful

variance in 3-year-olds’ performance than the very limited number of other tests which reach

down to such a young age.

Basic Procedure

For each of the 2-D and 3-D portions of the TOSA, participants received a training trial

followed by six test trials. During each test trial, the child was shown a target figure composed of

either 2-D geometric shapes or 3-D interlocking blocks. The child was then given individual

pieces, matching those in the target, and was instructed to make their pieces look just like the

target figure. The task required participants to exercise spatial visualization, shape

decomposition and composition, and manual construction of complex shape forms in planar

space and 3-dimensional space.

Background

This task was part of a larger study that sought to investigate geometric-spatial

knowledge as a precursor to the fundamental STEM (science, technology, engineering, and

mathematics) competencies that we need our children to possess as they enter a globally

competitive workforce. This project examined geometric-spatial abilities and their potential

correlations with mathematical ability over time in preschool children of diverse socio-economic

statuses. The work was designed to diagnose, analyze, and understand the role of spatial

assembly skills in early mathematical competency.

Ages for Which the TOSA is Applicable

Our papers report findings with 3-year-olds between the ages of 37 and 48 months, for

which the psychometric properties of the test yield good results (internal reliability; α = .747)

and predict later skill on standardized spatial measures for the same children at ages 4 and 5

years. The test was also administered to 4-year-olds with mixed results. A number of the items

had significant ceiling effects and, therefore, the test (as currently outlined below) is not as

appropriate for that age group, particularly toward the latter half of the 4th year. We are working

to find items and procedures to expand the effective range of the test.

TOSA: Manual Contents and Introduction

3

Test Administration

The TOSA is composed of 2-dimensional (2-D) and 3-dimensional (3-D) trials that score

children’s ability to use individual pieces to copy a target design. Scores are tabulated as to

whether the children match the design and according to the errors that they make in the process

of model building.

2-D TOSA Trials

Six constructions (see Table 1) and a training trial, each composed of between two and

four geometric shapes, were used as the target designs (see instructions for creating the materials

below).

Table 1. Six designs used for the 2-D TOSA testing trials.

Design

1

2

3

4

5

6

Children received trials in the same order, starting with the training trial and proceeding

as indicated in Table 1. Each board had a model picture at the top and the component shapes

placed randomly at the bottom of the board, nearest the child.

For the practice trial, the experimenter pointed to the shape pieces and indicated that

he/she was “going to try to make my pieces look just like this picture [experimenter points to

model].” To determine whether the child understood the task, the experimenter placed the shape

pieces incorrectly two times, confirming that the child could identify a non-matching design.

The experimenter then placed the shapes in the correct formation and corroborated the match

with the child. Finally, the experimenter placed the shape pieces in front of the child and

instructed the participant to “make [his/her] pieces look just like the picture.” Most children

correctly perform the task on the first try and many even spontaneously (and correctly) help the

experimenter when they make the errors during training.

TOSA: Manual Contents and Introduction

4

For the six test trials, children received each magnetic board sequentially with the target

design visible throughout a trial and no feedback. The task was untimed, and the participant

indicated completion of each design. The designs were then stacked and, before resetting the

boards for the next child, photographed for later, offline coding of construction accuracy.

3-D TOSA Trials

The 3-D portions of the TOSA included six constructions and a practice item selected to

provide a range of difficulty. The pieces to be constructed were made of interlocking Mega

Bloks® and constructions included two to four blocks. Since the original creation of the task we

were able to find individual Duplo® blocks for sale which we are now using instead (see

instruction on creating the items below)

Table 2. Six designs used for the 3-D TOSA testing trials.

1 2 3 4 5 6

Training and administration of the test trials was the same as in the 2-D trials except for

the materials used, which were a model for 3-D trials made up of an intact, glued-together

construction (rather than a picture of the design) and a set of separated bricks arranged randomly

on the table in front of the child. The separated bricks used for the construction matched the

model in color and number. The model was always left on the table in plain sight and children

were allowed to pick it up, although they were discouraged from playing with it and were not

allowed to add their pieces to it. Participants received all six test trials in a fixed order, as

specified above. As with the 2-D trials, pictures were taken of the designs for later coding.

Design

TOSA: Scoring Manual

5

Scoring Manual

Scoring

The TOSA is unique because the items permit finer grained analysis of children’s

geometric-spatial strategies and allows children to receive “partial credit” for preserving spatial

relationships between pieces, as opposed to using an “all or none” scoring system similar to most

standardized tests. Scoring is accomplished through a scheme that analyzes the child’s

construction across a number of dimensions. The advantage to this approach is that the test

produces a wider range of analyzable variability in performance for very young children in

comparison to standardized tests on which very young children may only get one or two items

completely correct. The wider range of scores for young children allows this test to more easily

differentiate skill differences between children in the “normal” range whereas most standardized

spatial tests only produce enough variability at age 3 to detect children who are scoring

exceptionally well or who are particularly delayed.

Scores for the two different types of trials (2-D and 3-D) are calculated independently, as

outlined below. Each participant is then given a z-score for their performance on each set of

trials. The average z-score for the two trial types then serves as the child’s overall score for the

TOSA.

In the instructions below you will notice that we discuss how to code “basic” scores for

both 2-D and 3-D trial types. These scores are much more akin to the “all of none” scoring of

many of the standardized tests and therefore share some of the drawbacks of that system. Basic

scores have a limited range (0-12 as opposed to 0-76 for dimensional scoring) which results in

distributions that are less normal and have less variability. Also, children receive credit only

when they are 100% correct, even if they display some competence in completing the trials,

which may obscure relatively small but potentially important differences in spatial competence at

this age. Preliminary analyses of this data show that they are correlated with the more detailed

scoring and that they do correlate with other measures, but the correlations are generally lower

and the psychometric properties of the resulting data are not as good. For example, correlations

between the TOSA given at age 3 and the Woodcock-Johnson Spatial Relations subtest given at

age 4 were .437 for the dimensional scoring system presented below and .398 for the match

scoring system. Correlations between the TOSA at age 3 and the WPPSI Block Design subtest

at age 4 were .555 and .426 respectively. In future development of this test we aim to better

quantify these differences and determine whether the basic scores can be useful as a “quick and

dirty” way to score the test. In the short-term we recommend using the detailed coding scheme

unless practical concerns (e.g., the need to immediately code designs online) dictate the use of

the basic scoring scheme.

Necessary Assumptions

A number of assumptions are necessary for the detailed scoring of the TOSA trials. First,

each figure is viewed as a whole made of component pieces. We assessed the placement of each

component piece with respect to a main, “base” piece. The central and/or largest piece was

TOSA: Scoring Manual

6

chosen as the “base” of each figure. Across designs the base piece tends to serve as the connector

between the most pieces. We use the base piece to provide a referent by which to judge the

position of component pieces.

A problem in trying to score complex figures is that, as component pieces are added, the

dimensions on which you might award points for “correctness” begin to become more

dependent. These coding schemes strive to limit dependencies, which might cause a single

mistake by a child to prevent them from getting many points while bigger mistakes might receive

smaller penalties.

TOSA: 2-D Scoring

7

2-D Trial Coding

Detailed Scoring

Materials

 Photographs of the participants’ designs o To ease coding and allow a permanent record of the data a camera jig -see below-

is used such that the pictures are taken from straight down and print outs can be

made to the original scale (1:1), allowing us to make accurate measurements and

mark up the images

 x,y-axis overlay for coding horizontal and vertical direction o These were just a set of axes printed with the axes parallel to the edges of a

transparent overlay

 Laminated cards featuring the target models for adjacent pieces

 Coding overlay for scoring translation in relative position o These are the target images printed on a transparent overlay at a scale that

matches the size of the component pieces. If a child’s design perfectly matched

the target image the overlay should perfectly match the child’s design.

 A ruler with centimeters and millimeters

 A protractor to measure angles

 A copy of the scoring sheet

Camera Jig:

General Instructions:

Throughout this manual “model” will be used to refer to the target design that the child is

supposed to reproduce. The word “copy” will be used to refer to the design that the child

produces. “Figure” is also a term used to refer to the whole design.

TOSA: 2-D Scoring

8

Each figure is pre-assigned a base piece (see Figure 1 at the bottom of the Detailed Scoring

section). The base piece is never scored. The other pieces in the design, called component

pieces, are all scored.

This coding scheme was designed to work specifically for Items 1-6 in Figure 1 of this manual,

but the rules were intended to be flexible enough to work for new and more difficult versions of

2-D designs.

You will be coding independent dimensions designed to capture how children conceptualize the

spatial relationship displayed in the model. Each dimension is explained below. The first

dimension (adjacent pieces) is designed to capture whether children appreciate that each model is

a unified figure. The second two dimensions (horizontal and vertical direction and relative

position) are designed to capture children’s understanding of where each component piece

belongs.

Figure 1. Base Pieces and Home Quadrants for Component Pieces

Item 1:

Item 2:

BASE

BASE

X

NE

TOSA: 2-D Scoring

9

Item 3:

Item 4:

Item 5:

BASE

BASE

BASE

NE

X

X

SE

X

Y

TOSA: 2-D Scoring

10

SW

Item 6:

Adjacent pieces - “pieces touching” - measurement

Adjacent Pieces awards points for a child that correctly places a piece next to the correct

neighboring pieces regardless of whether there is an error in the direction or relative orientation

of those pieces. Thus it acknowledges when children duplicate the relationship between two

adjacent pieces in their copy.

For each component piece, look at each piece that is directly attached to it in the model (these

shapes are referred to as “neighbors”). In the current set of shapes outlined in Figure 1, most

component pieces have only 1 neighbor, which is also the base piece. The blue pentagon in Item

5 is the only exception and should be scored against two neighbors (the pink square and the

green hexagon).

To score Adjacent Pieces, ask the series of questions below (example in Figure 3):

1) For each component piece in the child’s copy, is the nearest shape the correct neighbor shape?

If it is, score a 1.

If it is not, proceed to the next question.

2) Is the correct neighbor within 1 cm., in any direction and to any point on the shape, of the

component piece being scored?

If it is, score a 1.

BASE

SW

NW

NE

SW

TOSA: 2-D Scoring

11

If it is not, give the relationship a 0.

For Item 5, or future items in which pieces have more than one neighbor, ignore other valid

neighbor(s) when answering question 1 and only proceed to question 2 if a non-neighbor is

closer to the component piece that is being scored. See Figure 3 for an example of how the

pieces would be scored for Item 5.

The 1 cm qualification, mentioned in question 2 above, is to allow for when a child jams the

pieces close together and may have a non-neighbor closer to the component piece being scored,

even though the correct neighbor is still quite close by. Without the additional qualification that

the neighbor can be within 1 cm., a non-neighbor piece that is very near the component piece

being scored would prevent the child from getting any points for that relationship even though

the correct neighbor could still be very close by.

Figure 3: Adjacent Pieces Example The black triangle would receive a 1 for its relationship with the pink square, its neighboring piece, because that is

the closest piece to it. The green hexagon would receive a 1 for its relationship with its neighboring piece, the blue

pentagon, again because it is the closest piece. The blue pentagon would receive a 1 for its relationship with the

green hexagon because, although the triangle is the closest piece, the green hexagon is within 1 cm. of the blue

pentagon. However, the blue pentagon would receive a 0 for its relationship with the pink square since the pink

square is not the nearest non-neighbor piece (the triangle is closer) and the square is farther than 1 cm. from the blue

pentagon.

Model Child’s Copy

Horizontal and vertical direction – “correct side or correct top or bottom” – 1

possible point per piece

Horizontal and vertical direction captures whether the child knows that the component pieces

belong either above or below or to the left or the right of the base piece. To score this dimension

more readily, we created transparent x, y-axes to superimpose over the child’s base piece.

Because this is relative to the model, we superimpose the transparency on the child’s copy such

that it is aligned with the top and bottom and sides of the board. The center of the base piece of

TOSA: 2-D Scoring

12

the child’s copy can be determined by drawing lines between opposing vertices of the shape (see

dotted lines in Figure 4).

In order for the child to lose credit for the position of a component piece, 50% or more of the

volume of the piece must have crossed the x or y-axis by comparison to the model. For each

component that is misplaced more than 50% across either the x or y-axis, award 0 points. For

each component that is not misplaced, award 1 point. Once points are assigned, also assign an

error code to each component as per the rules below.

The above rules for assigning points apply to the current set of 6 items as follows:

Shapes assigned 1 of the 4 “home quadrants” in Figure 1 (NE, SE, SW, NW) will lose points if

more than 50% of those shapes are shifted across either the x or y-axis (i.e., 50% or more of the

shape is outside of the home quadrant). Pieces perfectly straddling an axis (those marked X or Y

in Figure 1; e.g., the pink triangle in Item 3) should be coded as incorrect if the piece crosses the

axis it is supposed to be resting on completely. By virtue of these X and Y pieces being on an

axis in the model, 50% of the shape is supposed to be across an axis. Therefore, in order for

50% more of the piece to have moved across the axis, the shape will have moved 100% of its

volume above/below (for pieces marked X) or to the right/left of the axis (for pieces marked Y).

These X and Y pieces should be subjected to the same rule as all the other pieces for the axis

they are not straddling. For example, in order for the pink triangle in Item 3 (an X piece) to lose

points for crossing the y-axis (a shift to the right), only 50% or more of the triangle would need

to appear to the right of the y-axis. It would not need to cross the y-axis 100%, since the triangle

starts completely to the left of the y-axis.

Error Codes:

If a piece is correct, it gets an error code of 0.

If a piece is marked as incorrect, go on to record the direction of the error depending on which

axis or axes the piece was moved across and the quadrant the piece was incorrectly moved into:

Error Direction

0 = no error

1 = up/down error – 50% or more of the piece moves across the x-axis

2 = left/right error – 50% or more of the piece moves across the y-axis

3 = both errors – 50% or more of the piece moves across both the x- and y- axes

Error Quadrant

NW, NE, SE, SW, or NA (for no error)

These additional codes are only to account for the types of errors and are not to be factored into

the overall score for the 2D design assembly task.

TOSA: 2-D Scoring

13

Figure 4: Horizontal and vertical direction Example for Placing the X, Y-Axis Overlay

Figure 5: Horizontal and vertical direction Scoring Example of Participant Copy The child has moved the blue square from its home quadrant in the NE (top right) more than 50% across both the x

and y axes into the SW (bottom-left corner). Therefore, that piece would receive a score of 0 and the error code

would be a “both” error (i.e., 3). The pink triangle, which straddles the x-axis, has not moved completely into either

the NW or SW quadrants. It also has not crossed the y-axis. Therefore, it is “correct” and receives a score of 1 with

an error code of 0. If the pink triangle were just a little farther up, it would cross completely into the NW quadrant

(more than 50% of the shape would move across the x-axis), in which case it would receive a score of 0 and an error

code of 1 for an up/down error.

base piece

Y-Axis

X-Axis

guides to find center of piece

TOSA: 2-D Scoring

14

Relative position – “pieces in correct spot relative to the base” – 1 possible

point per piece

This dimension is intended to award points to children who maintain the most relationships

between the component pieces they see in the model. This allows us to score whether children

get the relationships between the pieces even if the overall orientation of their copy is wrong. To

do this, we introduce the term “home orientation” to capture how the child has aligned their

figure with the model.

To find a figure’s home orientation you need to try every possible rotation of the child’s copy in

which the base piece matches the model. You are doing this to find the location where the

child’s score will be highest. Therefore, take note of the number of component pieces that are

correctly placed (use scrap paper to help remember the number of correct pieces), as you will be

comparing different possible home orientations to see which achieves a higher score.

While following the steps below, it may be helpful to reference Figure 6.

Step 1: Rotate the entire overlay clockwise until the base piece on the overlay is aligned with the

base piece for the child’s copy. Take note of the number of component pieces that are correctly

placed (use scrap paper if necessary). In order for a location to be considered “correct,” there

cannot be a translation error. A “translation error” occurs when any point on the participant’s

component piece falls farther than 1 cm. from the correct location of that same point (i.e., more

than 1 cm. from the same spot on the overlay-- see Figure 7).

Y-Axis

X-Axis

TOSA: 2-D Scoring

15

Step 2: Continue rotating the overlay clockwise into another orientation in which the base piece

on the participant’s copy is aligned with the base piece for the overlay. Again take note of the

number of the number of pieces that are in the correct location.

Step 3: If necessary (depending on the shape of the base piece), continue rotating the overlay

clockwise, noting the number of correct component pieces for every orientation of the child’s

base piece that can match the orientation of the model’s base piece (e.g., rectangular bases only

have 2 valid orientations, triangles have 3, etc.)

Step 4: Once all possible orientations of the base piece have been considered, select the “home

orientation” in which the largest number of pieces is in the correct locations. If multiple

orientations yield the same number of points, select the “home orientation” that required the

smallest degree of rotation to match the child’s base piece compared to the orientation of the

model’s base piece. Please do not trust your “eye” for making this judgment unless it is very

obvious. As Figure 8 may illustrate depending on your own spatial skills, angle differences of as

large as 10 degrees can be hard to judge. Once you have selected the “home orientation” that

maximizes relative position scores, write down a score (1 or 0) for each component piece based

on that orientation.

Optional - Step 5: Rotation: Using a protractor, the coder should score how far they had to

rotate the overlay from the “home orientation” to match the orientation of the overlay to the

model This measurement should be taken as the smallest magnitude rotation in either direction

(clockwise or counter-clockwise) with counter-clockwise rotations receiving a negative sign in

front of them (see Figure 8). Rotation scores will be bounded between -180 and 180 degrees. If

0 points were awarded for relative position, just record the smallest amount of rotation necessary

to make the child’s base piece match the orientation of the base piece on the model. Scoring

rotation is considered optional because it was not used in the overall scoring for the TOSA but

has been reported in some papers and could be interesting depending on your own motivations

for using the test.

Figure 6: Relative position Example (page 1 of 3)

Model Child’s Copy in Original Orientation

TOSA: 2-D Scoring

16

Step 1: Set the overlay so that the base piece matches the child’s copy. Note how many of the

component pieces are in the correct location compared to the overlay (in this case the score is

just 1, the black hexagon)

Step 2: Rotate the overlay clockwise until the base pieces are again aligned. Note how many of

the component pieces are in the correct location (in this case the score is just 1 again, the pink

triangle)

Step 3: There are only 2 possible orientations for a rectangle so no further rotations are

necessary.

Step 4: The “home orientation” would be the orientation from Step 1 since both orientations

result in 1 point and the orientation from Step 1 requires the smallest magnitude of rotation from

the original orientation of the child’s copy to match the model.

TOSA: 2-D Scoring

17

Model Orientation Orientation - Step #1 Orientation - Step #2

Step 5: Measure the amount of rotation necessary to match the child’s “home orientation” to the

model orientation. In this example, it is -18 degrees (black outline is the child’s original design,

and the blue outline is the orientation of an orthogonal part of the base piece as it would need to

be to be aligned with the base piece in the model. The red lines and text show the angle of

rotation necessary to make them match. Another way to think about this is to take the overlay in

the “home orientation” and rotate it to exactly match the model. The number of degrees you

rotate the overlay to match the actual model is the angle of rotation that needs to be measured*.

If you turn the overlay clockwise the sign is positive and if you turn the overlay counter-

clockwise the sign is negative.

*There are many ways to measure the angle of rotation. In cases where there is a reasonably

large degree of rotation, using a pencil and the image of the model you can draw lines extending

the same edges of the base piece on the child’s design and the model. Where the lines intersect

you can measure the degree of rotation. In cases where these lines do not intersect, you can often

measure the angle from where it intersects the edge of the board, since many of the models are

aligned with the sides of the board. Measuring the angle of rotation can be a bit of a geometry

problem.

-18

TOSA: 2-D Scoring

18

Figure 7: Translation Error Examples

Figure 8: Rotation Scoring Examples

Example 1 Model Example 2

Negative Rotation Positive Rotation

(-35 degrees) (25 degrees)

If this error is less than

1 cm. this would be

considered correct

and the component

would get a 1. If the

error is more than 1

cm., this component

piece would receive a

score of 0. As shown,

you need to measure

from the overlay to

the same location on

the shape.

Coding overlay

TOSA: 2-D Scoring

19

Basic Scoring

In addition to the detailed scoring scheme above, a more simplified scoring scheme is used as a

“quick and dirty” method of scoring the test. Correlations between this scoring system and the

more detailed scoring tend to be around 0.7 - 0.8. This scoring scheme also tends to predict to

later spatial skills, although not as well as the detailed scoring scheme, particularly as the time

between TOSA administration and administration of the other spatial tests increase.

The basic coding scheme essentially asks, “Was the participant’s reconstruction 100% correct?”

and could have been used without doing the detailed scoring. In practice we used the following

rules, which were based on data from the detailed scoring, to determine a participant’s basic

score for each trial:

1. For the component piece did the participant have less than 1 cm. (10mm.) of error for

adjacent pieces, less than 30 degrees of rotation error, and receive a point for piece

relationships, horizontal and vertical direction, and relative position?

If yes, the component piece receives a basic score of 1.

If no, the component piece receives a basic score of 0.

2. Did all component pieces receive a basic score of 1?

If yes, the Item Basic Score is 1.

If no, the Item Basic Score is 0.

2-D TOSA Scoring Sheet: Participant ID: ________________ TOSA: 2-D Score Sheet Coder Name: _________________ School: ______________ Testing Date: ______________ 20

Item 1

Horizontal and Vertical Direction Relative Position Adjacent Pieces

Basic

Score

(0/1)

Score

(0/1)

Quadrant

(NE, SE,

SW, NW,

N/A)

Error

(L/R =1;

Up/Dn = 2;

Both = 3;

N/A = 0)

Score

(0/1)

Rotation

(degrees)

Score (0/1)

Item 1a – Yellow Circle

Item 2

Horizontal and Vertical Direction Relative Position Adjacent Pieces

Basic

Score

(0/1)

Score

(0/1)

Quadrant

(NE, SE,

SW, NW,

N/A)

Error

(L/R =1;

Up/Dn = 2;

Both = 3;

N/A = 0)

Score

(0/1)

Rotation

(degrees)

Score (0/1)

Item 2a – Pink Pentagon

Item 3

Horizontal and Vertical Direction Relative Position Adjacent Pieces

Basic

Score

(0/1)

Score

(0/1)

Quadrant

(NE, SE,

SW, NW)

Error

(L/R =1;

Up/Dn = 2;

Both = 3;

N/A = 0)

Score

(0/1)

Rotation

(degrees)

Score (0/1)

Item 3a – Blue Square

Item 3b – Pink Triangle

2-D TOSA Scoring Sheet: Participant ID: ________________ TOSA: 2-D Score Sheet Coder Name: _________________ School: ______________ Testing Date: ______________ 21

Item 4

Horizontal and Vertical Direction Relative Position Adjacent Pieces

Basic

Score

(0/1)

Score

(0/1)

Quadrant

(NE, SE,

SW, NW)

Error

(L/R =1;

Up/Dn = 2;

Both = 3;

N/A = 0)

Score

(0/1)

Rotation

(degrees)

Score (0/1)

Item 4a – Green Triangle

Item 4b – Pink Circle

Item 5

Horizontal and Vertical Direction Relative Position Adjacent Pieces

Basic

Score

(0/1)

Score

(0/1)

Quadrant

(NE, SE,

SW, NW)

Error

(L/R =1;

Up/Dn = 2;

Both = 3;

N/A = 0)

Score

(0/1)

Rotation

(degrees)

Score (0/1)

Item 5a – Black Triangle

Item 5b – Blue Pentagon

To Hexagon

To Square

Item 5c – Green Hexagon

2-D TOSA Scoring Sheet: Participant ID: ________________ TOSA: 2-D Score Sheet Coder Name: _________________ School: ______________ Testing Date: ______________ 22

Item 6

Horizontal and Vertical Direction Relative Position Adjacent Pieces

Basic

Score

(0/1)

Score

(0/1)

Quadrant

(NE, SE,

SW, NW)

Error

(L/R =1;

Up/Dn = 2;

Both = 3;

N/A = 0)

Score

(0/1)

Rotation

(degrees)

Score (0/1)

Item 6a – Pink Triangle

Item 6b – Black Hexagon

Item 6c – Yellow Circle

2-D TOSA Scoring Sheet: Participant ID: ________________ TOSA: 2-D Score Sheet Coder Name: _________________ School: ______________ Testing Date: ______________ 23

Table 3. 2-D Trials: Total Points Possible for Each Shape Design by Dimension.

Design

1

2

3

4

5

6

Total

# of Pieces 2 2 3 3 4 4 18

Adjacent

Pieces -- -- 2 2 4 3 11

Horizontal

and Vertical

Direction

1 1 2 2 3 3 12

Relative

Position 1 1 2 2 3 3 12

Total 2 2 6 6 10 9 35

TOSA: 3-D Scoring

24

3-D Trial Coding

For the 3-year-old sample initially tested, items 1 and 2 from the 3-D trials were at ceiling

performance. Therefore, trials 1 and 2 were not included in the final scoring. We continue to

include these items because pilot testing has shown that children at the youngest part of that age

range (age 36 months) are capable of understanding and completing parts of the test and those

items may be necessary to include with younger age groups. A focus of our future development of

the test is to create items and procedures to administer the test to a wider age range.

Scoring Step 1: Base Piece Coding

For Step 1, we score how the non-base pieces are located relative to the base piece.

The base piece is always the largest piece. It is always located on the bottommost layer.

There are three scoring dimensions. Each dimension is to be scored on its own merits without

regard to the other scoring dimensions.

 Vertical location – whether a piece is located on the correct vertical level with respect to the base (above, below, same level). A piece that should be above (or below) the base piece but is

too far above/below the base piece (because another piece was incorrectly placed in between)

loses a point for vertical location.

Correct: Incorrect:

 Rotation – whether the piece is oriented in the correct way with respect to the base (parallel or perpendicular)

o 2x2 blocks cannot be scored for piece rotation

Correct: Incorrect:

 Translation – whether a piece is located in the correct horizontal position relative to the base o Remember, vertical location is not a factor. If a piece is on the wrong vertical level but

connected to the same pips as in the model, then it still receives full credit

o For pieces adjacent (but not connected) to the base:  If the piece is oriented correctly (receives a score for piece rotation) the pieces

must look exactly as they are in the model or it will receive no points.

Correct: Incorrect:

TOSA: 3-D Scoring

25

o If the non-base piece does not receive a point for piece rotation (i.e. is oriented incorrectly), then it must be located adjacent to the correct pips (or correct column of

pips) on the base.

Correct: Correct:

Incorrect:

o For pieces connected to the base, the piece must connect to the same pips as the piece in the model in order to receive full marks.

 Examples of errors that still receive points for translation:

Correct: Correct: Correct:

o Clarification example: In the child’s design below, the blue piece is covering the correct pips on the base piece (yellow), but it is also connected to an extra pip on the

base piece due to a rotation error. If a child makes a rotation error like that shown, they

would lose the rotation point but would still get the translation point, since the blue

piece is attached to the correct 2 pips. This lets us keep rotation points as independent

as possible from translation (otherwise a rotation error would often result in a

translation error as well).

Model: Child’s Design:

Scoring Step 2: Subset Scoring

Constructions 3-6 are more difficult than the preceding constructions in that there are multiple non-

base pieces, meaning these pieces have spatial relationships that are independent of the base piece.

To accurately code children’s answers and to give them more credit on these particularly difficult

constructions, the non-base pieces for these constructions are coded separately on their relations to

each other.

For these constructions, the non-base pieces are grouped into dyads. Each subset dyad is assigned a

ground piece in the same manner as used in the more general 3d scoring, with the largest piece

acting as the ground piece.

TOSA: 3-D Scoring

26

The dimensions used are the same as the base piece coding and should be scored exactly as they

would have been in general scoring:

 Vertical location: Whether a given piece is placed on the correct vertical level with respect to the ground piece.

 Translation: Position of a block on the horizontal axis relative to the ground piece.

 Rotation: Whether a given piece is rotated in the correct direction related to the ground piece.

Notes:

 Construction 6, subset 2 has the ground piece on top of another piece – this is the only time this happens.

 These pairs are graded without regard to their position relative to the base.

 As in base-relations coding, the absolute cardinal direction of the model is not scored.

 Not all dimensions can be scored for all subsets – this is reflected on the score sheet. For example, subset 1 of Construction 4 has no piece rotation score available, as 2x2 blocks

cannot rotate.

 On the score sheet, pictures are provided showing all possible correct designs for the given item.

o Note that for Construction 6, subset 2, that there are some flipped versions of the subset that are incorrect even though they are very similar to the correct design –

pay close attention to this subset while scoring. Examples of the correct and

incorrect designs follow:

o Correct:

o Incorrect:

3-D TOSA Scoring Sheet: Participant ID: ________________ TOSA: 3-D Score Sheet Coder Name: _________________ School: ______________ Testing Date: ______________ 27

Base Piece Scoring

Construction 1 – Small Step

Vertical

Location

Piece

Rotation Translation

Basic Score (perfect on other

dimensions)

Score N/A

Possible 1 0 1 2/2 = 1

Construction 2 – Big Step

Vertical

Location

Piece

Rotation Translation

Basic Score (perfect on other

dimensions)

Score

Possible 1 1 1 3/3 = 1

Construction 3 – Dinosaur

Vertical

Location

Piece

Rotation Translation

Basic Score (perfect on other

dimensions)

Score

Possible 2 1 (No

Square) 2 5/5 = 1

Construction 4 – Helicopter

Vertical

Location

Piece

Rotation Translation Basic Score

(perfect on other

dimensions)

Score

Possible 2 1 (No

Square) 2 5/5 = 1

3-D TOSA Scoring Sheet: Participant ID: ________________ TOSA: 3-D Score Sheet Coder Name: _________________ School: ______________ Testing Date: ______________ 28

Construction 5 – T-Shape

Vertical

Location

Piece

Rotation Translation

Basic Score (perfect on other

dimensions)

Score

Possible 3 2 (No

Square) 3 8/8 = 1

Construction 6 – Long Arm/Short Arm

Vertical

Location

Piece

Rotation Translation

Basic Score (perfect on other

dimensions)

Score

Possible 3 2 (No

Square) 3 8/8 = 1

Subset Scoring

Construction 3 – Dinosaur

(Note – base piece must be used to determine the distance between the ground and non-ground piece for

this design – vertical location does not take the base piece into account).

Subset 1

Vertical

Location

Piece

Rotation Translation

Basic Score (perfect on other

dimensions)

Score N/A

Possible 1 N/A 1 2/2 = 1

3-D TOSA Scoring Sheet: Participant ID: ________________ TOSA: 3-D Score Sheet Coder Name: _________________ School: ______________ Testing Date: ______________ 29

Construction 4 – Helicopter

Subset 1

Vertical

Location

Piece

Rotation Translation

Basic Score (perfect on other

dimensions)

Score N/A

Possible 1 N/A 1 2/2 = 1

Construction 5 – T-Shape

Subset 1

Vertical

Location

Piece

Rotation Translation

Basic Score (perfect on other

dimensions)

Score N/A

Possible 1 N/A 1 2/2 = 1

Subset 2

Vertical

Location

Piece

Rotation Translation

Basic Score (perfect on other

dimensions)

Score N/A

Possible 1 N/A 1 2/2 = 1

3-D TOSA Scoring Sheet: Participant ID: ________________ TOSA: 3-D Score Sheet Coder Name: _________________ School: ______________ Testing Date: ______________ 30

Construction 6 – Long Arm/Short Arm

Subset 1

Vertical

Location

Piece

Rotation Translation

Basic Score (perfect on other

dimensions)

Score N/A

Possible 1 N/A 1 2/2 = 1

Subset 2

Vertical

Location

Piece

Rotation Translation

Basic Score (perfect on other

dimensions)

Score

Possible 1 1 1 3/3 = 1

Subset 3

Vertical

Location

Piece

Rotation Translation

Basic Score (perfect on other

dimensions)

Score N/A

Possible 1 N/A 1 2/2 = 1

3-D TOSA Scoring Sheet: Participant ID: ________________ TOSA: 3-D Score Sheet Coder Name: _________________ School: ______________ Testing Date: ______________ 31

Table 4. 3-D Trials: Total Points Possible for Each Shape Design by Dimension.

# of Pieces 2 2 3 3 4 4 18

Block

Lengths 2,3 3,4 2,3,6 2,3,4 2,3,4,6 2,3,4,6

Vertical

Location 1 1 3 3 5 6 19

Rotation 0 1 1 1 2 3 8

Translation 1 1 3 3 5 6 19

Total 2* 3* 7 7 12 15 46

Note. Piece rotation was not scored for component blocks that are 2 studs x 2 studs because they are

symmetrical.

*Since the first two trials were at ceiling for 3-year-olds, these trials were left out of the final score for this portion of the test, yielding a total score of 41 points.

Design Total

1 2 3 4 5 6

TOSA: Constructing Stimuli

32

Constructing the 2-D and 3-D

TOSA Stimuli

2-D Trials

Recommended Materials List:

 Self-adhesive magnetic sheets: o ProMag Adhesive Magnetic Sheet- 4 sheets at 4" x 6"

 Self-adhesive foam sheets: o Creative Hands smART Foam Sheets (5.5” x 8.5”, 50-Pack, Rainbow Colors)

 Exacto knife or other sharp “hobby knife”

 Black magic marker

 Double-sided scotch tape

 Whiteboards o 11" x 8.5" Quartet Magnetic Whiteboards o The actual white, magnetic work surface measured 9-1/2" x 7-5/8"

 Model images and exploded shape template (pages below) printed on card stock, photo paper, or heavy paper.

Creating the 2-D Stimuli:

Models:

 Print out the models page and cut around the black rectangles for the models.

 Thick card stock or photo paper is preferable and/or laminating helps the models stay in good shape (children will try to grab and pick at them periodically.)

o Double-sided scotch tape on the back of the models is a good non-permanent way to affix them because the tape is thin enough that children cannot get their fingers underneath the cards easily.

 Affix the models to the top-center of each whiteboard.

Foam Shapes:

 Prepare your shape templates: o Print out the exploded shape template on card stock or photo paper. o Carefully cut out each shape from the template page with scissors. o Use the card stock pieces as a stencil for cutting out the foam shapes as described below.

 Affix magnet to the back of the foam before cutting out shapes: o Group the shapes by color and lay them on top of a similar colored sheet of foam close together but

far enough that you can cut around them.

o Mark a rectangle on the foam that will go around all of the shapes and cut out that rectangle. o Affix a similar sized piece of magnet to the back of the foam.

 Cut out the foam shapes: o Trace around shape templates onto foam using the black magic marker. o Use a hobby knife to cut around the shapes and through the magnet.

TOSA: Constructing Stimuli

33

 This will require a reasonable amount of pressure to get through the foam and magnet, so be careful. It helps a lot of if you are using a new blade. Holding the knife so that the handle is

at a 45 degree angle or less to the table (as opposed to the handle being closer to

perpendicular to the table – i.e., 90 degrees) will also help with cutting.

 Our lab has started to use a Cricut to cut out shapes, which results in clean cuts and consistent copies of the same shape. This piece of equipment is likely more money and

trouble than it is worth for making only a couple copies of the foam shapes, but works well if

you have one at your disposal or if you are creating many shape sets.

3-D Trials

When referring to block dimensions in this section we refer to the blocks by the number of pips (or pegs) they

have when looking down on the top of the block. All of the blocks are 2 pips wide.

Using half of the blocks, assemble the models and glue the pieces together using hot glue. Leave the other half

of the blocks loose for the participants to build with.

Full set of 3-D Trial Models:

Practice

Test #1 Test #2 Test #3

Test #4 Test #5 Test #6

TOSA: Constructing Stimuli

34

Lego Duplo Blocks

 Lego Duplo Blocks can be ordered individually. Lego offers a service called “Bricks & Pieces” which is

more extensive than the standard online brick orders and can be found here (https://www.lego.com/en-

us/service/replacementparts/sale). The Element ID’s are very helpful in finding the right pieces.

 These are the necessary quantities for one complete set.

Image Element ID Size Color Quantity

230021 2x6 Red 2

4558881 2x3 Red 6

343721 2x2 Red 4

301121 2x4 Red 2

4168579 2x2 Green 4

4168312 2x6 Green 2

230024 2x6 Yellow 2

301124 2x4 Yellow 6

343724 2x2 Yellow 2

4558884 2x3 Yellow 2

4613700 2x3 Blue 4

4167177 2x4 Blue 2

4166960 2x2 Blue 2

TOSA: Constructing Stimuli

35

2-D Models:

TOSA: Constructing Stimuli

36

2-D Exploded Shape Template:

TOSA: Constructing Stimuli

37

Appendix

1. We tried some different adhesives and solvents for bonding plastics to join the blocks. From our research,

Lego builders apparently use something similar to methyl ethyl ketone (MEK) to permanently bond bricks and

this probably would work. However, using hot glue and quickly sticking the blocks together while the hot glue

is still really hot seems to work well and will allow you to separate the blocks later without children doing it in

the normal course of use.

2. At the time of the creation of the original version of the TOSA we had used Mega Bloks which required us

to alter the blocks. We have since found Duplos in the sizes needed. With this change the colors of the blocks

used in the TOSA have varied somewhat. We think it is highly unlikely that the colors matter very much for

performance on the test as long as: 1) the models use the same colors as the pieces you provide to the child for

building, 2) models which were originally designed with 2 different length blocks of the same color (items 5

and 6 below) are built so that the color of those pieces match one another, and 3) other than the two specific

instances in models 5 and 6, none of the blocks match color. The instructions for the designs above satisfy

these requirements with readily available materials.

Test #5 Test #6

(2 yellow) (2 blue)