Measurement, Data Display and Interpretation
How behavior becomes data you can trust: what to count, how to count it, how to check the counting and how to read the graph that comes out of it.
- Concepts
- 12
- Sections
- 6
- Diagrams
- 6
In this chapter
What to measure, and where the data come from
Every number in this chapter starts with two decisions: what exactly counts as the behavior, and whether it is recorded as it happens, reported afterwards or read from what it left behind.
Writing operational definitions
An operational definition tells an observer exactly what to record. It describes what the person does in terms that can be seen or heard and measured, instead of a feeling, an intention or a diagnostic label. The dead man’s test is a heuristic for spotting targets defined only by an absence, such as not talking; it does not prohibit measuring an actively maintained posture when that posture is defined with observable criteria. A complete definition also settles the borderline cases: when an occurrence starts and when it ends, plus examples and nonexamples for the situations observers would otherwise argue about. A definition can be function-based, covering every form that has the same effect on the environment, or topography-based, specifying the form itself when the form is what matters. It has done its job when trained observers working independently record the same instances with it.
Example
“Aggressive” is not a definition. For this observation, define hitting as moving an open hand or fist into contact with another person’s body, excluding high fives, gentle taps to request attention and accidental contact. Count one hit for each contact following separation of the hand from the body. This specifies what is included, excluded and counted separately.
Common confusion
A definition that its author applies consistently, or that a parent approves of, has not yet been shown to work. The evidence is agreement between independent observers using only the written definition. Also avoid targets that describe an absence, such as “not leaving the seat”; state what the person does instead.
Key points (4)
- An operational definition names what the person does in observable, measurable terms, never an inferred state, intention or diagnostic label, and the target passes the dead man's test.
- A complete definition gives observers what they need for borderline cases: onset and offset criteria and examples and nonexamples, so that scoring does not rest on judgment.
- Function-based definitions include every form that produces the same effect on the environment; topography-based definitions specify the form itself and suit cases where the form matters or the function is not the question.
- A definition is adequate when independent trained observers using it record the same instances; the author's own scoring or a stakeholder's approval does not show that.
Direct, indirect and permanent-product measures
Direct measurement scores the target behavior itself through observation, either live or from a recording. The observer can be a behavior analyst, a teacher, a parent or the person themselves; the source of the record, rather than the observer’s job title or the time of scoring, distinguishes direct observation from a recalled account. Indirect measurement records something other than the behavior itself and infers the behavior from it, most often accounts given after the fact: interviews, rating scales and checklists filled in from memory. Because they depend on recall and judgment, they help with planning but are weaker evidence of how much behavior occurred. Permanent-product measurement counts the lasting effects a behavior leaves in the environment, such as solved problems on a page or assembled kits, after the behavior is over. It is sound when every occurrence leaves the same product, the product cannot appear some other way, and it can be traced to the person whose behavior is being measured.
Example
A store wants to count kits assembled by an employee. Tallying each completed kit while observing her work is direct measurement. Asking the manager on Friday how many she assembled is indirect. Counting her uniquely labeled completed kits at closing is permanent-product measurement, provided each completed kit leaves one retained product, no one else produces a product with her label, and none are removed or counted twice.
Common confusion
A data sheet, log or tally is where a direct observation is written down; it is not a permanent product of the behavior. And permanent-product data are not indirect just because nobody watched: they count real results of the behavior, not somebody’s memory of it.
Key points (4)
- Direct measurement records the target behavior itself, live or from a recording; what makes it direct is what is measured, not whether the observer is a behavior analyst, a caregiver or other staff.
- Indirect measurement records something other than the behavior itself and infers the behavior from it, usually reports made after the fact, such as interviews, rating scales and checklists completed from memory, so it rests on someone's recollection.
- Permanent-product measurement counts lasting effects attributable to the target behavior and person, with other sources excluded. Inferring response count requires a known response–product relation; measuring products such as new injuries need not establish the number or duration of responses.
- A form, log or tally that records behavior is a data sheet, not a permanent product of the behavior; choosing among the three kinds of measure weighs convenience against how much inference the data require.
How often, how long and how soon
Behavior has more than one dimension. It can happen often or rarely, last a moment or an hour, start right away or after a long wait. The measure has to match the dimension that is actually the problem.
Figure 1
Three clocks on one timeline
Time (seconds) →
Measuring occurrence: count, rate and percentage
A count is the number of times a behavior occurred. On its own it can mislead, because ten responses in five minutes and ten in an hour are very different performances, so a count always needs the length of observation or the number of opportunities beside it. Rate divides the count by the observation time and is reported with its unit, such as responses per minute; that makes sessions of different lengths comparable. Percentage divides the responses by the opportunities and multiplies by 100. It suits responding that can only happen when an opportunity is presented, such as answers to questions, but it hides the numbers behind it: with four opportunities, a single answer moves the result 25 points. Behavior that can happen at any moment, called free-operant behavior, is usually summarized as rate. Event recording fits behavior with a clear beginning and end.
Example
On Monday a child asks for help 12 times in 30 minutes; on Tuesday, 12 times in 60 minutes. The counts match, but the rates are 0.4 and 0.2 per minute. In a spelling drill he gets 9 of 10 words right: 90%. Another student’s 90% comes from 18 of 20 words, twice as many opportunities behind the same number.
Common confusion
Check numerator and denominator: rate is count over time, never time over count, and percentage is responses over opportunities. Comparing raw counts from sessions of different lengths, or reporting a percentage without the opportunities behind it, hides exactly what a reader needs to judge it.
Key points (4)
- A count is the number of occurrences; it can be interpreted or compared only together with the length of the observation period or the number of opportunities.
- Rate is count divided by observation time, stated with its unit; it allows comparisons across sessions of different lengths and can be converted between units (per minute, per hour).
- Percentage is responses divided by opportunities times 100; it hides how many responses and opportunities lie behind it, and with few opportunities one response changes it a lot.
- Free-operant behavior, which can occur at any time, is summarized by rate; responding whose opportunities are set by trials is summarized by percentage or count of opportunities; event recording suits behaviors with a clear beginning and end.
Measuring temporal dimensions: duration, latency and IRT
Duration is how long a response lasts, from its onset to its offset. It is reported either as total duration within a period or as mean duration per occurrence, and the two can move in opposite directions. Latency is the time that elapses between the start of a stimulus, such as an instruction, and the start of the response; the clock stops when the response starts, not when it ends. Interresponse time (IRT) is the time between two successive responses, commonly from the end of one to the start of the next. Other things equal, longer IRTs go with a lower rate of responding. The concern decides which measure to use: a slow start calls for latency, episodes that go on too long call for duration, and responses that come too close together or too far apart call for IRT.
Example
A parent says, “Shoes on, please.” Eleven seconds pass before the child picks up a shoe: a latency of 11 seconds. Putting on both shoes takes 90 seconds: duration. A teenager who eats very fast is taught to wait before the next bite; the pause from finishing one bite to starting the next is the IRT the program is trying to lengthen.
Common confusion
Latency runs from a stimulus to a response; IRT runs from one response to the next. Neither is how long a response lasts, which is duration. And a falling total duration is not automatically progress: ten one-minute episodes become three three-minute episodes, the total drops from ten minutes to nine, and every episode has become three times longer.
Key points (4)
- Duration is how long a response lasts, from onset to offset; it is reported as total duration in a period or as mean duration per occurrence, which can change in opposite directions.
- Latency is the time from the onset of a stimulus (such as an instruction) to the onset of the response; it ends when the response starts, not when it finishes. Some practice guides start the clock when the instruction ends; check how the question defines the starting point.
- IRT is the time between successive responses. With response duration approximately unchanged, longer offset-to-onset IRTs generally accompany lower response rates.
- The temporal measure follows the concern: slow to start calls for latency, lasting too long for duration, and pacing or spacing of responses for IRT.
Choosing a measure for representative data
Start with the dimension that is the problem: how often the behavior occurs (count, rate), how long it lasts (duration), how quickly it starts (latency), how responses are spaced (IRT), how forceful it is (magnitude) or what form it takes (topography). The measure has to capture that dimension. The observations must also be representative: scheduled when and where the behavior is likely, and long and frequent enough to sample it fairly. Practical limits, such as one observer for several students, can push the choice toward time sampling or permanent products, as long as the data still answer the question. Finally, the measure must be able to show change. A task already at ceiling, intervals that are too long, or a summary that averages away the dimension of concern can make real change invisible.
Example
A child has two tantrums a week, each lasting about 40 minutes. A program cuts them to five minutes each, but a weekly count would read 2 before and 2 after. Duration is the measure that can show the improvement. And observing only on calm afternoons, when tantrums are rare, would miss them whatever the measure.
Common confusion
The convenient measure is not automatically the representative one. Observations scheduled when the behavior rarely happens, or counts taken when the concern is how long or how hard, produce tidy data that do not answer the question that was asked.
Key points (4)
- The measure follows the dimension of concern: repeatability (count, rate), temporal extent (duration), latency, IRT, magnitude or topography.
- Representative data come from observations scheduled when and where the behavior is likely and long and frequent enough to sample it.
- Environmental constraints (observer availability, competing duties, several behaviors or people) shape the choice, for example toward time sampling or permanent products, as long as the data still answer the question.
- A measure must be sensitive enough to show change: ceiling or floor effects, intervals that are too long and summaries that hide the target dimension make change invisible.
Recording everything, or sampling
Some procedures catch every occurrence in the observation period; others look at part of it and estimate the rest. Estimates are useful, but their error depends on the sampling rule and the pattern of behavior.
Figure 2
One record, four answers
Behavior as it happened · Actual time in behavior
Partial interval · Any occurrence in the interval
Whole interval · Behavior throughout the interval
Momentary time sampling · Behavior at the end of the interval
Continuous versus discontinuous measurement
Continuous measurement detects every occurrence within the observation period. Event recording and timing duration, latency or IRT are continuous. The word refers to that period: a 20-minute session recorded this way is continuous even though the rest of the day goes unobserved. Discontinuous measurement samples the period, so some occurrences, or parts of them, are missed. Interval recording and time sampling are discontinuous, and their result is the percentage of intervals or moments in which behavior was scored. That percentage estimates the behavior; it is not a count and not a rate. Sampling lowers the demand on the observer and lets one person track several behaviors or several people at once, at a cost in accuracy, so the choice weighs what the data must show against the observer’s time and duties.
Example
One aide needs data on hand-flapping for three children during the same 15-minute circle time. Counting every flap for all three is impossible. Checking each child in turn at the end of every 30-second interval is manageable and gives an estimate for each child, reported as the percentage of checks with flapping.
Common confusion
Interval data cannot be turned back into a count or a rate: one scored interval may hold one response or ten. Observing every day does not make a sampling procedure continuous, and a short observation period does not make event recording discontinuous.
Key points (3)
- Continuous measurement (event, duration, latency, IRT recording) detects every occurrence within the observation period; continuity refers to that period, not to the whole day.
- Discontinuous measurement samples the period, so some occurrences go undetected; its result is a percentage of intervals that estimates the behavior, never a count or a rate.
- Discontinuous procedures lower observer demands and allow several people or behaviors to be measured at once, at the cost of accuracy; the choice weighs what the data must show against the observer's resources.
Interval recording and time sampling
The observation period is split into equal intervals, and each method applies a different scoring rule. Partial-interval recording scores an interval if the behavior occurs at any point in it, even briefly. Because a short occurrence earns the whole interval, it tends to overestimate the proportion of time the behavior occurs. Whole-interval recording scores an interval only if the behavior lasts the entire interval, so a one-second pause leaves it unscored; it tends to underestimate. Momentary time sampling scores only whether the behavior is occurring at the instant each interval ends. Its error has no fixed direction; more frequent observations can improve estimation, but shorter intervals do not guarantee less error in every record. Planned activity check, or PLACHECK, applies momentary time sampling to a group by counting how many members are doing the target activity at each moment. Choose the method and interval length to match the behavioral dimension and the observation resources. Whole-interval scoring is often used for sustained behavior to increase and partial-interval scoring for behavior to decrease, but these bias directions do not guarantee conservative estimates of change; check sensitivity and, when feasible, compare with continuous measurement.
Example
For reading engagement, whole-interval recording scores a 15-second interval only when the student is engaged throughout it. A change from 30% to 60% means more intervals contained uninterrupted engagement; it does not show that actual engagement time doubled. Partial-interval recording of calling out marks any interval with at least one call-out, but cannot show whether that interval contained one call-out or several.
Common confusion
For equal-length intervals scored correctly, partial-interval percentage is at least as high as the actual percentage of time, while whole-interval percentage is at most as high. Either can equal the true percentage for some records; longer intervals generally increase distortion and can hide change. Momentary time sampling looks at one instant per interval, not at what happens across the interval.
Key points (4)
- Partial-interval recording scores an interval if the behavior occurs at any point in it and tends to overestimate the proportion of time the behavior occurs.
- Whole-interval recording scores an interval only if the behavior lasts the entire interval and tends to underestimate the proportion of time the behavior occurs.
- Momentary time sampling scores whether behavior occurs at each check; its error has no fixed direction. More frequent checks can improve the estimate, but do not guarantee less error for every temporal pattern. PLACHECK applies momentary sampling to a group.
- Choose an interval method according to the intended level estimate: whole-interval provides a lower bound on time in behavior, partial-interval an upper bound. Those bounds do not guarantee conservative change estimates. Select intervals short enough to help detect the relevant changes.
Can the data be trusted?
Three questions about any set of data sound alike and are not: did we measure the right thing, did we get its true value, and would we get the same value again? Interobserver agreement answers only part of the third.
Figure 3
Accurate, reliable and valid are three different questions
Reliable, not accurate
Consistent, but all off by the same amount.
Reliable and accurate
Consistent and on the true value.
Neither
Scattered: individual measurements cannot be trusted.
Valid?
Is this the behavior that matters, on the dimension that matters, observed where it happens?
Counting words read per minute perfectly is still invalid if the concern is understanding the text.
Figure 4
Same records, different agreement
Interval data: was the behavior scored in each interval?
| Interval | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| Observer A | Scored | Scored | Not scored | Scored | Not scored | Not scored | Scored | Not scored | Not scored | Not scored |
| Observer B | Scored | Not scored | Not scored | Scored | Not scored | Not scored | Scored | Scored | Not scored | Not scored |
| Agreement | Agreement | Disagreement | Agreement | Agreement | Agreement | Agreement | Agreement | Disagreement | Agreement | Agreement |
-
Interval-by-interval
All intervals
8 ÷ 10 × 100 = 80%
-
Scored-interval
Only intervals that at least one observer scored
3 ÷ 5 × 100 = 60%
-
Unscored-interval
Only intervals that at least one observer left unscored
5 ÷ 7 × 100 = 71%
Event data: how many responses in each interval?
| Interval | 1 | 2 | 3 | 4 | 5 | Total |
|---|---|---|---|---|---|---|
| Observer A | 2 | 0 | 3 | 1 | 2 | 8 |
| Observer B | 1 | 1 | 3 | 2 | 1 | 8 |
-
Total count
Smaller total ÷ larger total
8 ÷ 8 × 100 = 100%
-
Mean count-per-interval
Mean of smaller ÷ larger in each interval
(0.5 + 0 + 1 + 0.5 + 0.5) ÷ 5 × 100 = 50%
-
Exact count-per-interval
Intervals with identical counts
1 ÷ 5 × 100 = 20%
Validity, accuracy and reliability of measurement
Validity asks whether the right thing was measured: the socially significant behavior itself, on the dimension that matters, at times and places representative of the concern. Accuracy asks whether the recorded values match what really happened, which can be checked against a reference such as a carefully scored video. Reliability asks whether measuring the same events again gives the same values. A measure can be reliable but inaccurate, when two observers make the same mistake, and accurate but invalid, when words read per minute are counted perfectly while the concern is comprehension. Interobserver agreement (IOA) compares independent observers and is reported as a percentage. Event data: total count IOA divides the smaller total by the larger; mean count-per-interval IOA averages that ratio over short intervals; exact count-per-interval IOA counts intervals with identical counts; trial-by-trial IOA counts trials scored the same way. Timing data: total duration IOA divides the shorter total by the longer; mean duration-per-occurrence IOA averages that ratio over occurrences. Interval data: interval-by-interval IOA counts agreements over all intervals; scored-interval IOA uses only intervals where at least one observer scored the behavior, the stricter choice for low rates; unscored-interval IOA uses only intervals where at least one observer did not, the stricter choice for high rates. Observer drift, reactivity and observer expectations threaten the data and are managed with training, calibration and regular checks.
Example
Two observers time a child’s crying in one session: 300 and 270 seconds. Total duration IOA is 270 ÷ 300 × 100 = 90%. That agreement makes the record believable and consistent; it does not prove that either stopwatch matched what really happened, and it says nothing about whether crying was the right target.
Common confusion
High IOA shows believability, not accuracy and not validity. Total count IOA can reach 100% when the observers recorded different instances that happened to add up to the same total. Observer drift is a change in how an observer applies the definition over time; reactivity is a change in the client’s behavior because someone is watching.
Key points (5)
- Valid measurement directly measures the socially significant behavior, on the dimension that matters, at times and places representative of the problem.
- Accuracy is agreement with the true value; reliability is consistency of repeated measurement; measurement can be reliable and still inaccurate, and high interobserver agreement shows believability, not accuracy.
- IOA methods differ in stringency: total count, mean count-per-interval, exact count-per-interval and trial-by-trial for event data; total duration and mean duration-per-occurrence for timing data; interval-by-interval, scored-interval (low rates) and unscored-interval (high rates) for interval data.
- IOA is computed as smaller over larger (or agreements over agreements plus disagreements) times 100; total count agreement can be high even when observers recorded different instances.
- Threats include observer drift, reactivity to being observed, observer expectations and poorly designed measurement systems; they are controlled by training, calibration, retraining and periodic checks.
Graphing and reading data
A graph is where measurement turns into a decision. The type of graph decides what can be seen, and a few disciplined questions decide what can be concluded.
Figure 5
The same ten sessions, three graphs
-
Line graph
↑ Correct answers
0510 Baseline InterventionSessions →
Shows every session
-
Bar graph
↑ Mean: correct answers
0510 4 Baseline 8 InterventionShows only a summary
-
Cumulative record
↑ Total correct answers
03060 Baseline InterventionSessions →
Slope shows answers added per session
Figure 6
Level, trend and variability
-
Level
↑ Responses
0612 Baseline InterventionSessions →
Flat data, higher after the phase change.
-
Trend
↑ Responses
0612 Baseline InterventionSessions →
Rising before and after: the intervention phase continues baseline’s trend.
-
Variability
↑ Responses
0612 Baseline InterventionSessions →
Similar mean, much less bounce.
Dashed line: mean of the phase
Graphing data
An equal-interval line graph plots behavior on the vertical axis against sessions or time on the horizontal axis, and it is the standard display for watching change. Its conventions do real work: labeled axes, a phase change line between conditions, a label naming each condition, and data paths that are not joined across the phase change line. A bar graph summarizes separate sets of data, such as the mean of each condition; it is good for comparison but hides session-by-session trend and variability. In a cumulative record each response is added to a running total, so the line rises or stays flat but never goes down: a steep slope means a high rate and a flat stretch means no responding. Slope represents responses per unit of the horizontal axis. When that axis is sessions, comparing elapsed-time response rates requires sessions of equal known duration or a separate time denominator. On a semilogarithmic chart, such as the Standard Celeration Chart, equal vertical distances stand for equal ratios, so going from 5 to 10 looks the same as going from 50 to 100, which allows relative change to be compared. Scaling choices matter too: a truncated or stretched vertical axis, or different scales on graphs meant to be compared, can inflate a small change or hide a large one.
Example
A coach records how many free throws a player makes during each ten-minute practice observation. On a line graph, a slump shows as the line dipping. On a cumulative record of the same practices, the slump shows as the line climbing more slowly, because the total can only grow.
Common confusion
A flatter stretch of a cumulative record means a lower rate, not fewer total responses, and the line never turns down. On a semilogarithmic chart, equal distances are equal ratios, not equal amounts. In a conventional noncumulative line graph, do not connect the data path across phase boundaries; a cumulative total may remain continuous when the record continues across conditions.
Key points (5)
- Equal-interval line graphs show behavior across sessions; conventions include labeled axes, phase change lines, condition labels and data paths that are not connected across phases.
- Bar graphs summarize and compare discrete sets of data (such as means per condition) and do not show session-by-session trend or variability.
- In a cumulative record each response adds to the running total, so the line never goes down; the slope shows the rate, and a flat segment means no responses.
- On semilogarithmic charts such as the Standard Celeration Chart, equal distances show equal ratios of change, so relative change can be compared; celeration is the change in response rate per unit of time, such as ×2 per week.
- Scaling choices, such as a truncated or stretched y-axis or different scales for compared graphs, can exaggerate or hide change.
Interpreting graphed data
Visual analysis starts inside each condition. Level is where the data sit on the vertical axis, often summarized by a mean or median. Trend is the direction and steepness of the data path over time. Variability is how far the data bounce around that path. Level and trend are separate properties: data can sit high and fall, or sit low and rise. Between conditions, look at how immediately the data change at the phase line, how much the data points of the two phases overlap, and whether similar phases produce similar patterns; a condition mean alone can hide all of this. Before changing conditions, baseline should be stable or moving away from the hoped-for direction. A highly variable baseline weakens prediction, and a baseline that is already improving makes an intervention effect difficult to distinguish from the trend that was already there; assess changes in level, trend and variability against that baseline pattern and seek replication. Whether a trend is good depends on the goal: rising is improvement for a skill and worsening for a problem behavior.
Example
A student’s correct answers in baseline run 4, 5, 6, 7 and 8, and in the intervention phase continue 9, 10 and 11. The intervention mean is higher, but the data simply carry on the baseline trend, so the graph offers no clear evidence of an effect. A flat baseline around 4 followed by an immediate jump to 9 would tell a very different story.
Common confusion
A higher mean in the second phase is not enough. Check the trend baseline already had, the overlap between phases and how variable the data are. And an upward trend is good news only for a behavior that is supposed to increase.
Key points (4)
- Within a condition, visual analysis looks at level, trend and variability; level and trend are different properties.
- Between conditions it looks at immediacy of change, overlap of data points and consistency across similar phases, not only at condition means.
- An improving baseline can support prediction, but continuing that trend alone does not demonstrate an effect. Consider more baseline data or an appropriate design; evaluate changes beyond the projected trend and their replication.
- Whether a trend is good depends on the goal: an upward trend is improvement for behavior to increase and worsening for behavior to decrease.
Measuring the procedure itself
Data on the client are only half the record. The other half says what the procedure cost to reach its goal and whether it was delivered the way it was written.
Measuring efficiency
Effectiveness asks whether a goal is reached at all; efficiency asks what it costs to reach it. Trials to criterion counts how many response opportunities it takes to meet a stated mastery criterion, and fewer trials to the same criterion means a more efficient procedure. Training time to criterion, in minutes, sessions or days, is another efficiency measure, and the two can disagree when sessions contain different numbers of trials. A cost-benefit analysis widens the view by weighing staff time, materials, money and the effort asked of the client against the outcome. Efficiency comparisons are fair only when the procedures share the same criterion and comparable targets.
Example
Two ways of teaching comparable coin names use the same criterion: at least nine correct in each of two consecutive, nonoverlapping ten-trial blocks. Method A reaches it after 120 trials in 12 ten-trial sessions. Method B reaches it after 80 trials in 16 five-trial sessions, combining each pair of sessions into one assessment block. B uses fewer trials and A fewer sessions; session duration and scheduling are needed to compare time or cost.
Common confusion
Two procedures can be equally effective and very differently efficient. Comparing trials to criterion means nothing if one procedure used an easier criterion or easier targets, and counting only correct trials, or stopping before the criterion is really met, understates the true number.
Key points (4)
- Trials to criterion is the number of response opportunities needed to reach a stated mastery criterion; fewer trials to the same criterion indicates a more efficient procedure.
- Training duration (minutes, sessions or days to criterion) is another efficiency measure; sessions to criterion and trials to criterion differ when sessions contain different numbers of trials.
- Efficiency differs from effectiveness: two procedures can both reach criterion while one needs fewer trials, less time or fewer resources.
- Cost-benefit analysis weighs resources (staff time, materials, money, effort for the client) against outcomes; efficiency comparisons are fair only with the same criterion and comparable targets.
Measuring procedural integrity
Procedural integrity, also called treatment integrity, is the extent to which an intervention is carried out as written. Without integrity measurement, confidence that the intended procedure produced the observed change is reduced; observed deviations also limit what can be concluded about the procedure as written. Integrity has several dimensions. Accuracy is whether each step is done correctly. Dosage is how much of the intervention is delivered, how often and for how long. Timing, such as how soon a consequence follows a response, can matter too. Integrity is usually scored with a step-by-step checklist, which shows not only an overall percentage but also which steps are missed. Representative data sample sessions across implementers, settings and times, including observations that are not announced. Constraints shape the method, from direct observation and video review to permanent products and implementer self-report; self-report is convenient but tends to be less accurate than direct observation.
Example
A token program has eight steps. Over a week an observer finds 88% integrity overall, but the checklist shows that the same step, delivering the token within five seconds, is missed in almost every session. The overall figure looked fine; the step-level data show exactly what to retrain.
Common confusion
Integrity data describe what the implementer did, not what the client did, and they are not IOA, which compares two observers’ records. Accuracy and dosage are separate: every step done perfectly twice a week, when the plan calls for daily sessions, is accurate but underdosed.
Key points (4)
- Procedural (treatment) integrity is the extent to which the intervention is implemented as written; without it, client data cannot be attributed to the intervention.
- Integrity has several dimensions: accuracy (steps implemented correctly) and dosage (how much, how often and how long the intervention is delivered), among others such as timing.
- Representative integrity data sample sessions across implementers, settings and times, including unannounced observations, and are scored with a step-by-step checklist that also shows which steps are missed.
- Environmental constraints shape the method (direct observation, video review, permanent products, self-report); implementer self-report is convenient but tends to be less accurate than direct observation.
Keep going
Practice the whole chapter in the course
Original questions on every task of area C, with explanations and a record of which tasks you have mastered.
Try a sample free, without an account