Experimental Design

How behavior analysts show that a procedure, and not something else, changed behavior: variables, validity, single-case logic and the designs built on it.

Concepts
9
Sections
4
Diagrams
5
In this chapter
  1. 01 Variables and validity
  2. 02 The logic of single-case research
  3. 03 Four single-case designs
  4. 04 Refining treatments and choosing a design

Variables and validity

Every design in this chapter exists to answer one question about a graph: did the change we see come from what we did? Start with the words for what is changed, what is measured and what else could be responsible.

Figure 1

Two questions to ask of every change

The dependent variable changed

Internal validity

Was it the independent variable?

  • History

    An outside event lines up with the phase change

  • Maturation

    The person changes with time: growth, fatigue, recovery

  • Measurement

    Observer drift, expectancy, new instruments, reactivity

  • Sequence effects

    An earlier condition colors a later one

  • Regression toward the mean

    After selection on an unusually extreme score, a later score may be less extreme without an intervention effect.

  • Attrition

    Participants leave, and the data change with them

  • Treatment integrity

    The procedure was not delivered as written

External validity

Does it hold elsewhere?

  • Other participants
  • Other settings
  • Other behaviors
  • Other times
  • Variations of the procedure

Built through direct and systematic replication, not through sample size alone.

Internal validity is threatened by anything other than the independent variable that could have moved the data; each card is one of the usual suspects. External validity is a separate question, about other people, places and procedures, and single-case research answers it by replication.
D.1

Dependent and independent variables

The independent variable is what the investigator deliberately changes: a procedure, a schedule, a kind of prompt. The dependent variable is the behavior measured to see whether anything changed, stated as a dimension with a unit, such as responses per minute, seconds of latency or percentage of trials correct. Anything else that could move the data, such as a new medication, a noisy room or a different observer, is an extraneous variable, to be held constant or at least monitored. A functional relation is claimed only when the dependent variable changes reliably each time the independent variable is changed.

Example

A clinic tests whether a visual timer shortens how long a boy takes to start a worksheet. The timer, on or off, is the independent variable. The dependent variable is latency: seconds from the instruction to his first pencil mark. A substitute teacher arriving that week is an extraneous variable worth marking on the graph.

Common confusion

The dependent variable is not the person, the room or the goal. "Leo" or "better focus" is not a dependent variable; "seconds to begin, per worksheet" is. Changing from paper to a tablet counter is a measurement change unless the investigator deliberately tests that change as the independent variable. Identify what is manipulated to answer the research question; it need not be switched simply on and off.

Key points (4)
  • The dependent variable is the measured dimension of the target behavior; the independent variable is the condition or procedure the investigator deliberately changes.
  • A dependent variable names what is measured and how (count, rate, duration, latency, percentage), not the participant, the setting or the goal.
  • Extraneous variables are uncontrolled factors other than the independent variable that may affect the dependent variable; investigators hold them constant or monitor them.
  • A functional relation is shown when the dependent variable changes reliably each time the independent variable is manipulated, and only then.
Examples and a practice question: D.1 Dependent and independent variables Practice this: D.1 Dependent and independent variables
D.2

Internal and external validity

Internal validity asks one question of a particular study: can the change in the dependent variable be credited to the independent variable, rather than to something else happening at the same time? It grows with experimental control, which in single-case work comes from repeating the effect inside the design. External validity asks whether the relation holds beyond the study, with other people, settings, behaviors, times or versions of the procedure; behavior analysts call this generality. Single-case research builds it through replication: direct replication repeats the procedure as it was, systematic replication deliberately varies some feature of it.

Example

A token system cuts call-outs for one third-grader, and the effect disappears and returns each time the tokens are removed and restored: strong internal validity. Whether it will work for her classmates, in music class or with another teacher is a separate question, answered only by trying it there: external validity.

Common confusion

The two are judged separately. A study can show tight control with one person and still say little about anyone else, and a large sample does not by itself make a result trustworthy or general. Social validity is a third, different question: whether the goals, procedures and results matter to and are accepted by the people involved. Interobserver agreement is a fourth: how much the measurement itself can be trusted.

Key points (4)
  • Internal validity is the degree to which a change in the dependent variable can be attributed to the independent variable rather than to uncontrolled factors.
  • External validity is the degree to which a functional relation holds for other participants, settings, behaviors, times or versions of the procedure (generality).
  • In single-case research, generality is built through direct and systematic replication rather than through large samples.
  • Internal validity, external validity, social validity and measurement reliability answer different questions and are judged separately.
D.3

Threats to internal validity

A threat to internal validity is any uncontrolled factor that could produce, or hide, the change credited to the intervention. History is an outside event that happens to coincide with a change of condition. Maturation is change inside the person over time: growing older, tiring, recovering. Measurement threats include observer drift (an observer slowly applying the definition differently), observer expectancy, changes in instruments and reactivity, where being watched alters behavior. Sequence effects, regression toward the mean, participants dropping out and a procedure not carried out as written, which is poor treatment integrity, complete the usual list.

Example

Tantrums drop in the very week a new plan starts, and in that same week the building work next door, which had been keeping the child awake, comes to an end. Better sleep from a quieter street is a history threat: an outside event lined up with the phase change. A toddler whose speech improves over six months of a program while also simply getting older illustrates maturation.

Common confusion

History happens outside the person; maturation happens inside the person and is tied to the passage of time. Regression toward the mean applies when someone was chosen because of an unusually extreme score, so a less extreme next measurement was likely anyway; it does not explain every improvement. Before crediting or blaming a treatment, check it was delivered as designed: poor integrity can hide a real effect or fake one.

Key points (4)
  • History is an event outside the study that coincides with a change in conditions and could produce the observed change.
  • Maturation is change within the participant over time, such as development, growth or fatigue, that is independent of the procedure.
  • Measurement- or observation-related threats include observer drift, expectancy, instrumentation changes and reactivity. Reactivity changes behavior itself; the others can alter its record.
  • Sequence effects, regression toward the mean, attrition and poor treatment integrity can also produce or hide changes that are not due to the independent variable.

The logic of single-case research

Behavior analysts usually study one person at a time and measure that person again and again. The choice has its own reasoning, its own strengths next to group studies and its own way of reading a graph.

Figure 2

A reversal design read with baseline logic

↑ Hand raises per class

A

B

A

B

Prediction Verification Replication

Sessions →

A BaselineB TreatmentA WithdrawalB Treatment

Illustrative values, not data from a real case. The dashed line projects the first baseline forward (prediction). The return toward baseline levels in the second A phase shows the projection would have held (verification), and the second B phase produces the effect again (replication).
D.4

Defining features of single-case designs

In a single-case design, each case provides its own comparison through repeated measurement across conditions or values of the independent variable; the comparison may involve baseline and treatment or two active treatments. Behavior is measured repeatedly within every condition, and the investigator usually waits for stable data before switching. The reasoning is called baseline logic. Prediction: a stable baseline, projected forward, shows what would probably happen if nothing changed. Verification: evidence that the projection would have held, such as data returning to baseline levels when the treatment is removed. Replication: the effect produced again. Arrange comparisons so that the planned independent-variable difference can be separated from other changes; if a whole package is introduced together, the result concerns that package rather than the contribution of each component.

Example

A teenager's hand-raising is counted in every class for two weeks and stays near two per period: that steady line is the prediction. Praise for raising a hand is added, and hand-raising climbs to eight. Praise is withdrawn and the count drifts back toward two (verification). Praise returns, and so does the higher rate (replication).

Common confusion

Single-case does not mean one data point, and a study can include several people; it means each person is the comparison for himself or herself, so no separate control group is needed. Verification and replication are different steps: one confirms the baseline prediction, the other repeats the treatment effect. A single measurement before and another after treatment is not a single-case design.

Key points (4)
  • Each participant serves as his or her own control: performance under the treatment is compared with the same person's performance without it.
  • Behavior is measured repeatedly within every condition, and conditions are usually changed after responding reaches a steady state.
  • Baseline logic has three elements: prediction (the projected baseline path), verification (showing the prediction would have held) and replication (repeating the effect).
  • Change one planned factor at a time to isolate a comparison; causal attribution also requires replicated effects and control of rival explanations. Changing components together evaluates the package, not each component's contribution.
D.5

Single-case versus group designs

Group designs often compare aggregated outcomes across groups assigned to different conditions, such as an intervention, another active intervention, usual care or no intervention; random assignment strengthens causal inference when feasible. They suit the question of how large an effect is, on average, across a population, which matters for program and policy decisions. Single-case designs show how each individual's behavior changes from session to session, and they let the investigator adjust conditions when the data call for it. Their generality comes from replication across individuals and settings, not from the number of people in one study.

Example

A school district wants to know whether a reading program raises average scores across forty classrooms: a group comparison answers that. The Instructor deciding whether the same program is working for one particular child, and what to change if it is not, needs that child's repeated data.

Common confusion

An average can hide what happened to individuals: a modest mean gain may come from a few large improvements while many people did not change or got worse, and those results are not errors to discard. Neither family of designs is better for every question; pick the one that matches what you need to know.

Key points (4)
  • Single-case designs show how each individual's behavior changes and let the investigator adjust conditions in response to ongoing data.
  • A group mean can hide individual variability: an average improvement may describe few or none of the participants.
  • Group designs with random assignment and large samples are suited to estimating average effects across a population, such as for program or policy decisions.
  • Single-case findings gain generality through replication across individuals and conditions, not through sample size.
D.6

Interpreting and critiquing single-case data

Single-case data are read through visual analysis of the graph. Within each phase, look at level (where the data sit), trend (their direction and slope) and variability (how much they bounce). Between adjacent phases, look at how quickly the data change and how much the two data paths overlap. A convincing effect usually shows a shift in level or trend soon after the condition changes, with little overlap. Experimental control needs that effect repeated within the design: one change from baseline to treatment, an A-B comparison, cannot rule out other explanations.

Example

In baseline a girl completes 3, 4, 3 and 4 puzzles per session; with a new prompting procedure she completes 9, 10, 11 and 10, and later replications look the same. A stable baseline, an immediate jump in level and no overlap: the pattern a reader hopes to see.

Common confusion

If baseline is already trending in the direction you expect the treatment to push, the same trend continuing after treatment begins is not evidence for the treatment. A very variable baseline makes any later change hard to read. If a tier changes before it receives the intervention, or behavior never comes back toward baseline in a reversal, suspect a confound or a limit of the design rather than counting it as success.

Key points (4)
  • Visual analysis examines level, trend and variability within and between phases, plus the immediacy of change and the overlap between phases.
  • A baseline already trending in the expected direction of treatment, or highly variable, makes a later change hard to attribute to the intervention.
  • Experimental control requires the effect to be replicated within the design; a single A-B change does not demonstrate a functional relation.
  • Change in a tier or condition that has not yet received the treatment, or a failure to recover baseline levels, signals a possible confound or limit of the design.

Four single-case designs

Four arrangements cover most single-case research. Each one gets its repetitions of the effect in its own way: by taking the treatment away, by starting it at different times, by switching quickly between conditions or by moving a criterion.

Figure 3

Multiple baseline across settings

↑ Seconds per transition

Baseline Picture schedule HomeClinicSchool

Sessions →

Illustrative values. The intervention starts at a different time in each setting while the later settings stay in baseline. Settings that stay steady until their own start support the prediction, and each staggered change repeats the effect; a setting that changed before its start would point to a confound.

Figure 4

Alternating treatments: two conditions, one graph

↑ Correct responses (%)

Video model Spoken instructions

Baseline

Alternating treatments

Better condition alone

Sessions →

Illustrative values. After a short baseline the two conditions alternate from session to session in a counterbalanced order, each signaled by its own stimulus. The separation between their data paths supports the comparison; the more effective condition is then used on its own. Carryover between conditions is the design's main risk.

Figure 5

Changing criterion: behavior follows each step

↑ Minutes of daily reading

Criterion in force
Baseline 812161220 ↓ Step back

Sessions →

Illustrative values. In this arrangement, reinforcement depends on meeting the current reading-duration criterion. Each horizontal line is the criterion in force, and the data settle near each new level soon after it changes. Steps of different length and size, and one step back to an earlier criterion, make it hard to explain the match by anything other than the criterion.
D.7

Reversal, multiple-baseline, multielement and changing-criterion designs

A reversal (withdrawal) design introduces the independent variable, takes it away and brings it back (the A-B-A-B sequence); control appears when behavior shifts each time the condition does. A multiple-baseline design staggers intervention onset across behaviors, settings or participants that can change independently. Plan at least three distinct intervention starts to provide repeated opportunities to demonstrate the effect. A multiple-probe variant reduces continuous baseline measurement but retains planned probes, including checks before each tier’s intervention, to test whether untreated tiers remain unchanged. A multielement, or alternating-treatments, design switches rapidly between conditions, each tied to its own distinct stimulus, and compares their data paths. A changing-criterion design makes reinforcement contingent on meeting successive performance criteria for one behavior and tests whether behavior follows those criterion changes.

Example

To test whether a picture schedule shortens transitions, an Instructor introduces it first at home, two weeks later at the clinic and two weeks after that at school, recording transition times in all three places throughout: a multiple baseline across settings. If transitions shorten only where and when the schedule arrives, each staggered start repeats the effect.

Common confusion

A reversal removes the treatment; a multiple baseline never does, it staggers when the treatment starts. Alternating treatments switches conditions from session to session or faster, not phase by phase. In a changing-criterion design, control is shown by behavior tracking each new criterion, not by withdrawal or alternation. "Across behaviors", "across settings" and "across participants" name what the tiers are, not three different logics.

Key points (4)
  • A reversal (withdrawal) design shows control by introducing, withdrawing and reintroducing the independent variable, as in A-B-A-B.
  • A multiple-baseline design introduces the independent variable in a staggered way across behaviors, settings or participants; a multiple-probe design uses intermittent baseline probes instead of continuous measurement.
  • A multielement (alternating-treatments) design rapidly alternates two or more conditions, each paired with a distinct stimulus, and compares their data paths.
  • A changing-criterion design changes the performance criterion for one behavior in steps; control is shown when behavior changes to match each new criterion.

Refining treatments and choosing a design

Once something works, the next questions are which version works best and which design fits the case in front of you. Both are settled by the question asked and by what the behavior allows.

D.8

Comparative, component and parametric analyses

These analyses refine a treatment instead of only asking whether it works. A comparative analysis sets two or more different interventions against each other to see which is more effective or efficient. A component analysis takes a treatment package apart, removing elements from the full package or adding them one at a time, to learn which parts are necessary or sufficient. A parametric analysis keeps a single independent variable and compares different values of it: its amount, intensity, duration or how often it is delivered.

Example

A feeding program combines praise, a sip of a favorite drink after each bite and a picture chart of the plate. Running it with and without the chart is a component analysis. One sip per bite against one sip per three bites is parametric. The whole program against an entirely different video-modeling procedure is comparative.

Common confusion

Component means parts of one package; comparative means different treatments; parametric means different values of one variable. A functional analysis asks which environmental contingencies influence the target behavior; comparative, component and parametric analyses describe what is compared or varied, so these labels can also describe comparisons made within a functional analysis.

Key points (4)
  • A comparative analysis tests two or more different interventions against each other to find which is more effective or efficient.
  • A component analysis separates the parts of a treatment package, by removing or adding them, to find which are necessary or sufficient.
  • A parametric analysis compares different values of one independent variable, such as amount, intensity, frequency or duration.
  • These analyses are chosen to make treatments more efficient: dropping unnecessary components, choosing the better procedure or finding an effective level.
D.9

Selecting and applying single-case designs

Start from the question. To show that one intervention changes a behavior, a reversal or a multiple baseline can work; to compare interventions quickly, alternating treatments; to show control over gradual change in a behavior already in the repertoire, a changing criterion. Then look at the behavior. A skill that will not fade once learned, like reading a word, cannot be reversed, so use a multiple baseline or multiple probe. If withdrawing an effective treatment would be unsafe or unacceptable, as with severe self-injury, avoid designs that need withdrawal. Multiple-baseline tiers must be independent of each other yet likely to respond to the same intervention.

Example

An Instructor wants to show that a new teaching procedure works for reading three sets of community signs. Once a child reads STOP, removing the procedure will not make him forget it, so a reversal would fail. A multiple probe across the three sets fits: teach the first set while checking the other two now and then, then move on.

Common confusion

Choosing a reversal for a skill that will not reverse, or for dangerous behavior, is the classic mistake. A second one is building tiers that will change together, such as two closely related skills likely to spread to each other, or two siblings who share the same routine. Repeated testing can be burdensome or teach the skill, making planned probes useful; delaying needed treatment is a separate ethical concern, and fewer measurements do not shorten that delay. If waiting is unacceptable, choose a design and service arrangement that provides timely care.

Key points (5)
  • The design is chosen to answer the question asked: demonstrating one treatment's effect, comparing treatments, or showing control over gradual changes.
  • Behavior that is unlikely to return to baseline once learned is evaluated with a multiple-baseline or multiple-probe design rather than a reversal.
  • When withdrawing an effective treatment would be unsafe or unacceptable, a design that does not require withdrawal is preferred.
  • Multiple-baseline tiers need to be independent of each other yet likely to respond to the same treatment; prolonged or reactive baselines favor multiple probes.
  • A changing-criterion design fits a behavior already in the repertoire that should change gradually in steps toward a target.

Keep going

Practice the whole chapter in the course

Original questions on every task of area D, with explanations and a record of which tasks you have mastered.

Open the course

Try a sample free, without an account