Noonan syndrome

The effects of feedback on the acquisition and retention of mathematical fact series by elementary school students classified with mild mental retardation was examined in 4 studies. Immediate feedback was provided by either an educator or the Immediate Feedback Assessment Technique (IF AT), at the end of a test series by a review of correct solutions (delayed feedback), or for control purposes, with a Scantron form. Reductions in errors and inaccurate perseverative responding during intervention periods were higher, and the repetition of errors during maintenance test was lower, when feedback was provided by an educator in Study 1. These results were replicated and extended in Study 2 to the operation of subtraction. In Study 3, the concurrent presentation of both forms of immediate feedback was more effective than the presentation of either form separately. In Study 4, the adjunctive value of the IF AT to facilitate the teaching-learning process was established, with higher levels of independent learning and retention demonstrated when the IF AT was available. The IF AT, as a simple paper and pencil tool, can assist the educator through the provision of individualized performance feedback and the encouraging of students to continue responding while simultaneously promoting independent learning.

The past 20 years have witnessed considerable changes in educational policies and practices related to instruction of students with learning difficulties (Hitchcock & Noonan, 2000). The number of students with learning difficulties educated in regular classrooms increases annually, presenting new challenges to educators and prompting development of new methods of classroom instruction (Parker & Schuster, 2002). The design of instructional strategies for children with learning difficulties requires an effective set of teaching procedures (Noonan & McCormick, 1993), with most studies typically examining the effects of classwide peer tutoring (CWPT), constant time delay (CTD), and computer-assisted instruction (CAI) on a number of learning outcomes.

A central ingredient of CWPT, CDT, and CAI is the provision of immediate corrective feedback. There is little disagreement that feedback is an effective method for enhancing learning, but there has been relatively little agreement about the timing of its presentation (Bruning, Schraw, & Running, 1999; Robin, 1978). Kulik and Kulik (1988) reported that immediate feedback is more effective than delayed feedback for applied, but not laboratory, studies. Corrective feedback on objective examinations completed in the classroom, in the absence of computers, cannot be provided until the examination has been completed. In comparison, the conditions and equipment within the laboratory permit the immediate delivery of corrective feedback on an item-by-item basis. Until recently, the simple and practical means through which immediate feedback might be provided in the classroom in the absence of computers has not been available. In recent studies conducted in our laboratory, we have developed and validated a classroom assessment technique through which individualized performance feedback is coupled with the opportunity to answer until correct using simple paper and pencil media (Dihoff, Brosvic, & Epstein, 2003; Epstein, Brosvic, Dihoff, Lazarus, & Costner, 2003; Epstein, Epstein, & Brosvic, 2001; Epstein et al., 2002).

The tool that has been refined and validated in our classrooms and laboratories is the Immediate Feedback Assessment Technique, or IF AT, which manifests the theoretical and practical foundations of the teachingtesting machines described by Pressey (1950) and Skinner (1968), transforming the passive receiver of information into the active demonstrator of skills and knowledge. The IF AT form (see Figure 1) is a multiple-choice answer sheet with rows of rectangular answer spaces (e.g., A, B, C, D) that is nearly identical in layout to the ubiquitous machine-scored answer sheet available from Scantron Corporation. Participants scrape off an opaque, waxy coating covering an answer space on the IF AT form to record their answer. If a symbol (e.g., a star) is printed beneath the covering, the student receives instant feedback that a correct choice was made; the absence of a symbol provides instant feedback that an incorrect choice was made. However, rather than simply exiting the question, the student reviews the remaining response options, continues to respond until the correct answer is discovered (a self-correction procedure), and thus exits each question knowing the correct answer.

The effectiveness of the IF AT has been demonstrated in a number of studies conducted in our classrooms and laboratories, with enhancements in student performance observed during cumulative assessments (Epstein et al., 2002, 2003). In these studies, the provision of corrective information during classroom and laboratory tests reduced the likelihood of incorrect perseverative responding on subsequent item administrations presented in their original or with a modified wording (Dihoff et al., 2004). The correction of initially inaccurate responses was maintained across retention intervals of 2 to 10 weeks and across populations ranging from preschool children with developmental delays (Epstein et al., 2002) to university students preparing for classroom examinations (Dihoff et al., 2004). Similar gains were not observed when responses were recorded with Scantron forms, and only intermediate gains were observed when feedback was provided either after the completion of a test or after a 24-hour delay (Dihoff et al., 2003). Students without known learning difficulties (elementary school students through college students) reported that immediate feedback and the opportunity to respond until correct provided realistic assessments of performance, more involvement in the testing process, and the opportunity to exit an item with the correct solution. Students with diagnosed learning difficulties reported that immediate feedback enhanced the acquisition of basic academic materials (e.g., colors, shapes), enhanced rote memorization once a concept and the function of that concept were grasped (e.g., learning multiplication skills in discrete units), and that the answeruntil-correct procedure enabled the completion of each test item as a discrete unit that could be resolved before approaching the next test item.

Among the most common advantages of the IF AT reported by educators with special needs children in their classrooms and resource rooms was that the IF AT not only provided feedback, but that the answeruntil-correct procedure prompted students to continue to respond until the correct solution was attained. This latter advantage was not achieved when feedback was provided by an educator, as students required continuous prompting to maintain responding. When the IFAT was used, feedback coupled to the opportunity to answer-until-correct could be provided to an entire class, whereas an educator could provide this combination to only one student at a time, which suggests a tremendous potential for the IF AT as an adjunctive tool with special-needs learners under appropriate classroom conditions. For example, the IF AT would not be effective for teaching concepts and their applications and functions, but it would be highly effective for assisting learners during rote memorization drills, such as those used while rehearsing mathematical fact series. The concept and function of an arithmetic operation, in keeping with NCTM 2000 principles and standards (National Council of Teachers of Mathematics, 2000), must be presented by an educator. However, the encouragement of active learning, the provision of immediate feedback, and the use of an answer-until-correct procedure that maximizes time on task, are three cardinal principles of pedagogy (Rickey, 1995), and they are easily employed with the IF AT.

The present studies were undertaken to determine how immediate feedback and the opportunity to answer until correct could assist the learning of students classified with mild mental retardation. Each child had participated in early intervention and preschool intervention services, repeated at least one grade, received the majority of instruction in regular classrooms, and presented considerable learning difficulties in the area of mathematics. A review of each student's mathematics examinations indicated high rates of inaccurate perseverative responding, while observation of their learning environments indicated that each student required high rates of verbal prompting to maintain responding. These learning difficulties and classroom behaviors suggested the opportunity to evaluate the adjunctive potential of the IF AT. The four studies described below included more participants than reported in prior studies employing single-subject procedures, and all teaching and testing occurred within the regular classrooms and resource rooms in which the students received instruction.

Study 1 compared the effects of immediate feedback provided by an educator with immediate feedback provided by the IF AT, delayed feedback, and the absence of feedback on the acquisition of an arithmetic fact series. Study 2 compared the effects of immediate feedback, provided by either an educator or the IF AT, separately, on the acquisition of a fact series for addition, subtraction, division, and multiplication. Study 3 compared the interactive effects of immediate feedback, provided by an educator and the IF AT, on the acquisition of fact series for the operations of addition, subtraction, division, and multiplication. Study 4 examined the adjunctive utility of the IF AT during the teaching process.

Study 1

Analysis of the Effects of Timing of Feedback on the Acquisition of a Multiplicand

Method

Participants. Twelve male and four female students meeting the criteria for the diagnosis of mild mental retardation (AAMR, 2002) and enrolled in an urban public elementary school served as voluntary subjects. The representative student was either an African-American or a Caucasian male, of lower socioeconomic status (United States Department of Health and Human Services, 2002), 10.5 years of age, had participated in early intervention and preschool intervention services, currently enrolled in second grade after repeating at least one academic year, and received the majority of instruction in regular classrooms. Three students were being treated for mild seizure disorders, two were being treated with antidepressants, and one was classified with Down syndrome. The median score on the WISC-III was 66 (range: 59 to 69) and the median developmental standing on the Vineland Adaptive Behavior Scale (Sparrow, BaIIa, & Cicchetti, 1984) was 7.1 years (range: 6.05 to 8.9).

Materials. Multiplicands of four served as the test stimuli. Materials included 3- x 5-inch index cards for the presentation of multiplicands, with four solutions (marked A, B, C, D) presented in multiple-choice format. For each member of the fact series there was one card (e.g., 4 x 0) and four solutions from which to select. The solutions were presented in multiple-choice format (e.g., A = 4, B = 8, C = 2, D = O), with the solutions for each member of the fact series printed on separate sheets of paper. Responses were recorded on either Scantron answer sheets using a pencil to darken the appropriate circle or the Immediate Feedback Assessment Technique form (E3 Corporation).

Design and procedure. Thirty test sessions were completed by each participant, with five sessions completed daily during the baseline and the intervention periods; during the maintenance period, only one session was completed per day on each of five consecutive days. Each test session consisted of the presentation of 10 single-digit multiplication facts with a multiplier of 4 (i.e., 4 x 0 through 4 x 9), each time in a random order. After completion of the baseline period, 1 female and 3 male participants were randomly assigned to complete the intervention period using one of four feedback procedures. Participants in the four feedback conditions were matched for scores on the WISC-III, the Vineland Adaptive Behavior Scale, and overall classroom performance (e.g., grades in mathematics, overall class grades, participation, and attendance). The participants reviewed the multiplicands of one through three, received instruction on the concept and function of the multiplicand of four, and then transitioned to resource rooms for experimental participation; all participants were taught by the same educator. Answers in the control condition were recorded with a pencil on a Scantron form. Answers in the end-of-test-feedback condition were recorded on a Scantron form, but after the completion of the test session, all pencils were removed and the participants were permitted to review the multiplicand series, the correct solutions, and their original answers for 10 minutes; these participants could review their work, but could neither discuss it with other participants nor ask questions about their solutions. Participants in the other conditions remained seated during this time and worked quietly on non-course materials under educator supervision. Participants in the immediate feedback condition scraped off the opaque, waxy coating covering an answer space on the IF AT form to record each answer. If a symbol (e.g., a star) was printed beneath the covering the student received instant feedback that a correct choice was made; the absence of a symbol provided instant feedback that an incorrect choice had been made. Answers in the educator-feedback condition were recorded on a Scantron form, and verbal feedback was provided by the educator. Correct responses were reinforced with "that is correct, 4 x 1 is 4"; incorrect responses were met with "that is not correct" and a verbal prompt to make an additional response. A maximum of two additional responses was permitted before the correct solution was provided by the educator, and thus the maximum number of responses permitted and the performance information provided in the IF AT and educator-feedback conditions was comparable.

Participants in the four conditions rated confidence in the accuracy of their solutions on a 100-point scale ranging from 1 (no confidence) to 10O (complete confidence) after each response during the baseline, intervention, and maintenance periods. Upon the conclusion of the intervention period, each participant completed a 15-item questionnaire assessing ease of understanding and of completing response requirements, perceived fairness of and preference for an answer-untilcorrect procedure, and involvement in the test-taking process, as described previously by Epstein and Brosvic (2002).

Results

The results of ANOVAs with feedback condition as the betweensubjects factor indicated that scores on the WISC-III and the Vineland scales, as well as on the measures of classroom performance, did not differ, all F .87.

Mean errors per session are plotted in Figure 2 as a function of feedback condition and test session. Potential differences in mean errors were analyzed using an ANOVA with feedback condition as the betweensubjects factor and test session as the within-subjects factor, with significance observed for each main effect and their interactions, all F > 21.21, all p

Mean confidence ratings are plotted in Figure 3 as a function of feedback condition and test session. Potential differences in confidence ratings were analyzed using an ANOVA with feedback condition as the between-subjects factor and test session as the within-subjects factor, with significance observed for each main effect and their interactions, all F > 19.02, all p

Mean responses on the scales (see Table 1 ) measuring test anxiety and time requirements did not differ as a function of feedback condition, all F .5. Mean responses on the scales measuring satisfaction with response format, clarity of response requirements, the desirability of the response form, and the benefits of testing differed significantly as a function of feedback condition, all F > 23.78, all p

Study 2

Effects of Timing of Feedback on the Acquisition of Multiple Mathematical Operations

Rationale: The results of Study 1 indicated that the fewest errors were made when feedback was provided by an educator, and that fewer errors were made when the IF AT form rather than end-of-test feedback was provided. Study 1 employed multiplicands as test stimuli, and thus Study 2 was undertaken to examine how the two types of immediate feedback promote the learning of simple fact series for the mathematical operations of addition, subtraction, and division.

Method

Participants. Thirteen male and three female students meeting the criteria for the diagnosis of mild mental retardation (AAMR, 2002) and enrolled in an urban public elementary school served as voluntary subjects. The representative participant was either an African-American or a Caucasian male, of lower socioeconomic status (United States Department of Health and Human Services, 2002), 11 years of age, had participated in early intervention and preschool intervention services, currently enrolled in third grade after repeating at least one academic year, and received the majority of instruction in regular classrooms. One participant was being treated with an antidepressant, two were being treated for minor panic attacks, one was being treated for seizure disorder, and one was classified with Down syndrome. The median score on the WISC-III was 64 (range: 60 to 67) and the median developmental standing on the Vineland Adaptive Behavior Scale (Sparrow et al., 1984) was 8.2 years (range: 7.1 to 8.9).

Materials. The fact series for multiplication by four, division by two, the addition of three, and the subtraction of four served as test stimuli. The format of the materials and their means of presentation were identical to those described in Study 1.

Design and procedure. After matching for scores on the WISC-III and Vineland scales, as well as on measures of overall classroom performance, two groups, each with 8 students, were formed. One group was randomly assigned to receive educator feedback, and the other to receive feedback using the IF AT. A latin-squares design was used to balance the order through which the participants within each group completed the four mathematical operations; all participants were taught by the same educator. Thirty test sessions were completed by each participant in each experimental condition, with five sessions completed daily during baseline and intervention periods; during the maintenance period, only one session was completed per day on each of five consecutive days. Each test session consisted of the presentation of 10 single-digit arithmetic facts related to multiplication by four, division by two, the addition of three, or the subtraction of four, as appropriate to the experimental condition. Participants began each experimental condition in the regular classroom with baseline testing followed by instruction on the concept and practice of the arithmetic operation specific to each experimental condition. Participants in each condition rated confidence in the accuracy of their solutions on a 100-point scale ranging from 1 (no confidence) to 100 (complete confidence) after each response during the baseline, intervention, and maintenance periods. Upon the conclusion of the intervention period, each participant completed the 15-item questionnaire described in Study 1.

Results

The results of ANOVAs using source of feedback (educator, IFAT) as the between-subjects factor indicated that scores on the WISC-III and the Vineland scales, as well as on the measures of classroom performance, did not differ between the feedback conditions, all F .47.

Mean errors are plotted in Figure 4 as a function of source of feedback, experimental period, and arithmetic operation. Potential differences in mean errors were analyzed using an ANOVA with source of feedback (educator, IF AT) as the between-subjects factor and experimental period (baseline, intervention, maintenance) and mathematical operation (addition, subtraction, division, multiplication) as within-subject factors. Significance was observed for each main effect and the three-way interaction of source of feedback, period, and mathematical operation, all F > 5.56, all p .5. Scheffe comparisons during intervention and maintenance indicated that errors were lowest in the educator-feedback condition for the operations of subtraction and multiplication, all p

Mean confidence ratings are plotted in Figure 5 as a function of source of feedback, experimental period, and mathematical operation. Potential differences in confidence ratings were analyzed using an ANOVA with source of feedback (educator, IF AT) as the betweensubjects factor and period (baseline, intervention, maintenance) and arithmetic operation (addition, subtraction, division, multiplication) as within-subject factors; significance was observed for neither the main effects nor their interactions, all F .71. Mean responses on the six scales (see Table 2) measuring test anxiety, time requirements, satisfaction with response format, clarity of response requirements, the desirability of the response form, and the benefits of testing did not differ between the two feedback conditions, all F .63.

Study 3

Examining the Interactive Effects of Combining Feedback From an Educator With Feedback From the IF AT

Rationale: The results of Studies 1 and 2 demonstrated the superiority of immediate feedback over delayed feedback, with the most favorable outcomes observed when feedback was provided by an educator. Study 3 was undertaken to examine the interactive effects of combining immediate feedback provided by the IF AT with immediate feedback provided by an educator.

Method

Participants. Eleven male and five female students meeting the criteria for the diagnosis of mild mental retardation (AAMR, 2002) and enrolled in an urban public elementary school served as voluntary subjects. The representative participant was either a Hispanic or an African-American male, of lower socioeconomic status (United States Department of Health and Human Services, 2002), 11.25 years of age, currently enrolled in third grade after repeating at least one academic year, had participated in early intervention and preschool intervention services, and received the majority of instruction in regular classrooms. Two participants were being treated for minor panic attacks, one was being treated for seizure disorders, one was being treated for thyroid hormone deficiency, one was being treated for lead poisoning, one was being treated for phenylketonuria, and one was classified with Down syndrome. The median score on the WISC-III was 64 (range: 60 to 66) and the median developmental standing on the Vineland Adaptive Behavior Scale (Sparrow et al., 1984) was 8.65 years (range: 7.4 to 9.4).

Materials. The fact series including multiplication by 4, division by 2, the addition of 3, and the subtraction of 4 served as test stimuli. The format of the materials and their means of delivery were identical to those described in Study 1.

Design and procedure. Four participants were assigned to each feedback condition after matching for scores on the WISC-III and Vineland Scales, as well as on the measures of overall classroom performance. The procedures for the Scantron, educator feedback, and the IF AT were identical to those described in Study 1. The fourth condition combined the procedures for educator feedback with those for the IF AT, with the educator providing verbal feedback which duplicated the printed feedback provided by the IFAT. All participants were taught by the same educator.

Thirty test sessions were completed by each participant, with five sessions completed daily during baseline and intervention periods; during the maintenance period, only one session was completed per day on each of 5 consecutive days. Each test session consisted of the presentation of 10 single-digit mathematical facts (e.g., addition: 3 + 0 through 3 + 9), each time in a random order. Participants completed baseline testing in the regular classroom, instruction on the concept and practice of the mathematical operation appropriate to their experimental condition, and then transitioned to the resource room for experimental participation; all participants were taught by the same educator. Participants in each condition rated confidence in the accuracy of their solution on a 100-point scale ranging from 1 (no confidence) to 100 (complete confidence) after each response during baseline, intervention, and maintenance. Upon the conclusion of the intervention period, each participant completed the 15-item questionnaire described in Study 1.

Results

The results of ANOVAs with feedback condition as the betweensubjects factor indicated that scores on the WISC-III and the Vineland scales, as well as on the measures of classroom performance, did not differ between four feedback conditions, all F .75.

A scatterplot of the median number of errors is presented in Figure 6, separately for each mathematical operation, as a function of feedback condition and experimental period. A visual analysis of trends in the number of errors specific to the IF AT and educator feedback conditions indicated that errors made during baseline were unaffected by carryover effects from rotating across the feedback conditions, and were typically lower when feedback was provided by an educator. A visual inspection of errors for the educator and IF AT conditions indicated synergistic effects of combining these two forms of feedback, with errors rates typically lower than those observed when feedback was provided by an educator alone. Potential differences in mean errors were analyzed using an ANOVA with source of feedback (Scantron, educator, IFAT, educator with IF AT) as the between-subjects factor and arithmetic operation (addition, subtraction, division, multiplication) and experimental period (baseline, intervention, maintenance) as within-subject factors. Significance was observed for the main effects and the two-way interaction of source of feedback and experimental period, all F> 8.91, all p .5). Scheffe comparisons during the intervention and maintenance periods indicated that fewer errors were made when feedback was (a) combined (educator feedback and the IF AT) than when feedback was provided, separately, by either an educator or the IF AT, and (b) provided, separately, by either an educator or the IF AT than when feedback was absent, all p

Confidence ratings did not differ as a function of mathematical operation and were thus aggregated across these operations in the analyses described below, F .48. Confidence ratings are presented in Figure 7 as a function of feedback condition and experimental period. Potential differences in confidence ratings were analyzed using an ANOVA with source of feedback (Scantron, educator, IF AT, educator with IF AT) as the between-subjects factor and experimental period (baseline, intervention, maintenance) as the withinsubjects factor; significance was observed for each main effect and their interaction, all F> 14.27, all p .5. Scheffe comparisons indicated that confidence ratings during the intervention and maintenance periods did not differ between feedback conditions, and that the lowest confidence ratings were observed for participants provided with Scantron forms, all p

Mean responses on the scales (see Table 3) measuring test anxiety and time requirements did not differ as a function of feedback condition, all F .5. Mean responses on the scales measuring satisfaction with response format, clarity of response requirements, the desirability of the response form, and the benefits of testing differed significantly as a function of feedback condition, all F > 5.65, all p

Study 4

Use of the IF AT as an Adjunctive Tool For Assisting Teachers to Maximize Teaching Effectiveness and Student Learning

Rationale: The results of educator debriefings during Studies 1-3 indicated unanimity for the potential of the IF AT as an adjunctive teaching tool that could reduce time spent providing individualized performance feedback, increase student learning and retention, and enable the presentation of more lessons during a regularly scheduled classroom session. Indeed, content analyses of educator activities prior to initiating Studies 1-3 indicated that 62% of educator time was spent providing performance feedback, that 28% was spent prompting students to answer until the correct solution was either offered by the student or provided by the educator, and that 10% was spent explaining task and response requirements. The provision of performance feedback and answering until correct are routine activities that could be delegated to the IF AT form, given its immediacy of feedback, affirmation of correct responding, and opportunity to answer until correct. Thus, Study 4 was undertaken to examine the adjunctive value of the IF AT to assist the educator in the teaching process.

Method

Participants. Nineteen male and five female students meeting the criteria for the diagnosis of mild mental retardation (AAMR, 2002) and enrolled in an urban public elementary school served as voluntary subjects. The representative participant was either a Caucasian or an African-American male, of lower socioeconomic status (United States Department of Health and Human Services, 2002), 12 years of age, currently enrolled in fourth grade after repeating at least one academic year, had participated in early intervention and preschool intervention services, and received the majority of instruction in regular classrooms. Two participants were being treated for minor panic attacks, two were being treated for seizure disorders, one was classified with traumatic brain injury, one was classified with Down syndrome, and one was classified with Williams syndrome. The median score on the WISC-III was 63 (range: 60 to 67) and the median developmental standing on the Vineland Adaptive Behavior Scale (Sparrow et al., 1984) was 8.55 years (range: 7.2 to 9.7).

Materials. The multiplication fact series of 5 and 6, addition fact series of 5, subtraction series of 5 and 6, and division series of 3 served as the test stimuli. The format of the materials and their means of delivery were identical to those described in Study 1.

Design and procedure. The participants were taught by the same educator in a resource room in four separate groups, each with six students, with the groups matched for scores on the WISC-III and Vineland Scales, as well as on the measures of overall classroom performance.

Each participant completed seven instructional sessions, at the rate of one session per day, for each mathematical operation described above. Baseline and maintenance sessions were typically completed within 5 minutes, both of which were completed in the regular classroom. Upon the completion of baseline testing, participants completed instruction on the concept and function of the mathematical operation that was next in the experimental series, and then transitioned to a resource room for experimental participation. Intervention sessions typically lasted for 30 minutes during which participants completed the same 10-item fact series five times, each time in a random order. The maintenance session was completed two weeks after the final intervention session for each mathematical operation, with participants completing the same 10-item fact series; performance on this assessment served as the primary measure of retention.

One group of students was randomly assigned to each of the four experimental conditions: Scantron, IF AT, IF AT / Scantron, and Scantron / IF AT. The procedures for the Scantron and IF AT groups were identical to those described in Study 1. Participants in the IF AT / Scantron group completed the first three mathematical operations with the IF AT and the final three mathematical operations with the Scantron; these procedures were reversed for the Scantron / IF AT group. Within each group, a latin-square procedure was used to counterbalance the sequence of the mathematical operations. Participants in each condition rated confidence in the accuracy of their solutions on a 100-point scale ranging from 1 (no confidence) to 100 (complete confidence) after each response during the baseline, intervention, and maintenance periods. Upon the conclusion of the first three mathematical operations, and again after completing the final three mathematical operations, participants completed the 15-item questionnaire described in Study 1. In addition to the dependent measures described above, the number of minutes of educator time spent providing performance feedback (that could be transferred to the IF AT), prompting students to answer until correct, and the percentage of time that each student was able to work independently, were recorded daily by the educator and were verified during each session by an experimental assistant.

Results

The results of ANOVAs with feedback condition as the betweensubjects factor indicated that scores on the WISC-III and the Vineland scales, as well as on the measures of classroom performance, did not differ between the feedback conditions, the mathematical operations, or their interaction, all F .5.

Mean errors, aggregated across the five sessions for each mathematical operation during intervention (top panel) and maintenance (bottom panel), are presented in Figure 8 as a function of mathematical operation and feedback condition. Potential differences in errors were analyzed using an ANOVA with feedback condition (Scantron, IFATJFAT / Scantron, Scantron / IF AT) as the between-subjects factor and experimental period (baseline, intervention, maintenance) and mathematical operation as within-subject factors. Significance was observed for each main effect and the interaction of feedback condition and experimental period, all F> 8.19, all p .82. Scheffe comparisons also indicated that errors for the (a) IF AT / Scantron condition increased significantly after transitioning from the IF AT to the Scantron and (b) Scantron / IF AT condition decreased significantly after transitioning from the Scantron to the IF AT, all p

Mean confidence ratings during intervention (top panel) and maintenance (bottom panel) are plotted in Figure 9 as a function of mathematical operation and feedback condition. Potential differences in confidence ratings were analyzed using an ANOVA with feedback condition (Scantron, IFAT, IF AT/ Scantron, Scantron / IFAT) as the between-subjects factor and experimental period (baseline, intervention, maintenance) and mathematical operation as within-subject factors. Significance was observed for the main effects and interaction of feedback condition and experimental period, all F > 33.04, all p .5. Scheffe comparisons also indicated that confidence ratings for the (a) IF AT / Scantron condition decreased significantly after transitioning from the IF AT to the Scantron and (b) Scantron / IF AT condition increased significantly after transitioning from the Scantron to the IF AT, all p

Mean minutes of educator time saved during an instructional session are presented in Figure 10 after aggregating minutes into deciles and partitioning the feedback conditions according to the presence of the IF AT (not present, present). An ANOVA with presence of the IF AT as the between-subjects factor indicated significant differences in time savings, F= 63.17, p

Mean responses on the scales (see Table 4) measuring test anxiety and time requirements did not differ significantly as a function of the feedback conditions, all F .5. An ANOVA with feedback condition as the between-subjects factor indicated that mean responses on the scales measuring satisfaction with response format, clarity of response requirements, the desirability of the response form, and the benefits of testing differed significantly, all F> 10.16, all p

Discussion

The test stimuli used in Studies 1-4 were components of each participant's regular educational programming, and thus they were neither distractions to ongoing classroom activities nor additional cognitive burdens; all lessons were taught by participants' regular educators, thus maximizing the validity and the generalizability of the intervention procedures. The samples evaluated in the present studies were substantially larger than those reported in studies employing single-subject paradigms (Parker & Schuster, 2002), and participants were evaluated in resource rooms after receiving regular classroom instruction on the concept, operation, and function of an arithmetic operation. In each study there were no between-condition differences during baseline, the presentation of experimental conditions was counterbalanced, and participants were matched for scores on the WISC III, the Vineland Adaptive Behavior Scale, and measures of general classroom performance; thus, neither individual differences nor the order in which the experimental conditions were completed are likely to be explicative factors for the results observed in Studies 1-4. The provision of immediate feedback resulted in the fewest errors, the least amount of perseverative inaccurate responding, and the most retention, especially when feedback was provided by an educator.

To our knowledge, the present results are the first report of the effects of combining different forms of immediate feedback on student learning; however, these results do not permit a determination of why the combination of feedback increased learning. Given the similarity of corrective information and opportunity to engage in iterative responding, it is likely that some combination of the redundancy of corrective information, the combination of visual and aural feedback, the presence of a familiar educator, and the opportunity to consider and to incorporate corrective information during the time between receiving feedback from the IF AT and the educator may be potential explanations. We have examined several of these potential explanations in followup studies and found that alternations of the order in which combined feedback is presented (IF AT followed by educator, educator followed by IF AT), the use of a familiar rather than an unfamiliar educator, the repetition of feedback by an educator (a procedure than cannot be duplicated with the IF AT), and prior experience with the IF AT do not produce significant differences in either retention or perseverative responding. Although causal factors have yet to be identified, the adjunctive value of the IF AT was established in Study 4. As a simple paper and pencil tool, the IF AT not only enabled students to demonstrate the most independent learning and the highest level of performance, but the IF AT also simultaneously assisted the educator through the provision of individualized and immediate feedback and response procedures that encouraged students to respond until a correct solution was attained.

The present results are consistent with observations made in our laboratory and classroom interventions, including those in which children with mild mental retardation have participated (Epstein et al., 2003). Despite substantive differences in test stimuli (adaptive living materials versus arithmetic fact series), similar outcomes were observed: Immediate feedback promoted retention. The present results are also consistent with observations made in our laboratory and classroom interventions during a number of studies, including those in which the learning of children at the elementary school (Epstein et al., 2003) and college levels (Dihoff et al., in press) were evaluated using fictional and regular classroom materials. The robustness of immediate feedback in these and other studies raises significant questions about the delayretention effect (Kulhavy & Anderson, 1972; Surber & Anderson, 1975).

One of the most common sources of external feedback in classroom settings is the professional educator. Bennett and Cavanaugh (1998) reported on the effectiveness of feedback to enhance the learning of a multiplication fact series by a junior high school student classified with a learning disability. In their first study, immediate feedback was defined as the learner's review of the correct answers upon the completion of the fact series. The provision of this type of feedback enabled the learner to make more correct responses than when the same fact series was completed without feedback. In their second study, feedback was provided either orally by an educator upon the learner's completion of each multiplication problem or, again, through the learner's review of the correct answers at the end of each fact series. As expected, the provision of (1) immediate oral feedback promoted more correct responding than the provision of feedback at the end of a fact series, and (2) feedback at the end of a fact series promoted more correct responding than when the fact series was completed in the absence of feedback. The Bennett and Cavanaugh (1998) results are similar to those observed in our studies when immediate feedback was presented, although there are substantial differences in the number of errors made to acquire fact series. For example, Bennett and Cavanaugh (1998) reported that only 22 errors were committed, 7 of which were repeated. A visual inspection of Study 1-4 figures presenting errors indicates that significantly higher and sustained numbers of errors were made; similarly, a visual inspection of the conditional analyses presented in Table 5 indicates that some errors, even in the presence of feedback, tend to be repeated. Although the test stimuli were identical, the format and means of their presentation differed, as did the learning difficulties of the participants; still, the causal factors behind this discrepancy remain unknown. Laboratory and classroom interventions are currently in progress using the procedures described in Studies 1-4 with students diagnosed with learning difficulties in the area of mathematics, and results collected to date demonstrate that the number of errors made by participants were similar to those reported in the present study.

The results of the present studies are consistent with prior reports of the effectiveness of feedback to enhance the performance of special need(s) students. Feedback has been shown to increase correct responding (Turner & Matherne, 1994), the learning of environmental words (Griffen, Schuster, & Morse, 1998), peer interactions in preschool children with disabilities (Odom, Chandler, Ostrosky, & McConnell, 1992), the acquisition of incidental information by secondary-age students with mental retardation (Gast, Doyle, Wolery, AuIt, & Farmer, 1991), the accuracy of students with learning disabilities reading consonant-vowel-consonant combinations (Perkins, 1988), the number of correctly chained steps completed by students with moderate intellectual difficulties (Mechling & Gast, 1997), the spelling performance and academic responding of students with severe disabilities (McDonnell, Thorson, Alien, & Mathot-Buckner, 2000), the observational learning of participants with moderate intellectual difficulties (Schuster, Morse, Griffen, & Worley, 1996), and the food preparation skills of elementary school students with moderate and severe disabilities (Fiscus, Schuster, Morse, & Collins, 2002). It should be noted that the studies described above predominately included students with mild to moderate intellectual difficulties, and similar improvements have not always been observed for students with severe to profound difficulties (e.g., Farmer, Klein, & Bryson, 1992). This latter observation was supported in our laboratories and classroom interventions in which feedback was used to assist students with varying levels of mental retardation in the acquisition of social skills.

Collectively, the results of Studies 1-4 demonstrate that use of the IF AT promotes the acquisition and retention of arithmetic fact series by elementary school students diagnosed with mild mental retardation, especially when feedback was provided by an educator. These results also suggest noteworthy differences and similarities between the IF AT and the more common procedures of CWPT, CTD, and CAI. The delivery of feedback by an educator restricts that educator to working with one student at a time, and thus the amount of time available for general classroom instruction is reduced. The IF AT enables the concurrent provision of feedback to any number of students in any learning environment. The delivery of feedback by an educator, on a one-on-one basis, requires either a considerable number of paraprofessionals or the distribution of short sessions of feedback across an extended period of time. The IF AT enables the provision of feedback on a one-to-one basis, and it does so at a comparatively nominal cost. The delivery of feedback by an educator can be adjusted to the visual and motor skills of the learner, and thus students with varying capacities can benefit from corrective feedback. The IF AT can be used with a wide range of intellectual and physical difficulties, but the learner must have the capacities to see a test question, to record an answer, and to recognize when an error has been made. The IF AT was designed for use with stimulus materials that can be translated into either a true-false or a multiple-choice format, and at an instructor's discretion, students can be permitted to answer until correct. The provision of feedback by an educator can be completed with a wide range of test materials and response options, including the ability to engage in follow-up questioning, instruction clarification, and branching toward alternative assessment strategies as a function of student performance and ability.

The results of our studies with special needs students complement those studies described above, support the effectiveness of immediate feedback to promote the correction of initially inaccurate assumptions, in general, and the use of an answer-until-correct procedure as an effective medium for the presentation of corrective feedback, in particular. The IF AT supports learning by providing immediate affirming and/or corrective feedback while involving the learner in the discovery process. The typical multiple-choice test may be an effective and practical assessment tool, but it does not convert mistakes into new learning, especially when used with special needs populations. A more optimal multiple-choice testing format would not only assess the learner's current level of understanding, but would also correct misunderstandings. That is, the test would teach as well as assess, and to those ends, the answer-until-correct procedure described in the present study is presented as a tool for use in educational settings where one-to-one educator oral feedback is impractical.

References

AMERICAN ASSOCIATION ON MENTAL RETARDATION. (2002). Definition of mental retardation. Printed on June 4, 2003. http://www.aamr.org/ policies/faq^mentaLretardation.shtml.

BENNETT, K., & CAVANAUGH, R. (1998). Effects of immediate self-correction, delayed self-correction, and no correction on the acquisition and maintenance of multiplication facts by a fourth-grade student with learning disabilities. Journal of Applied Behavior Analysis, 31, 303-306.

BRUNING, R., SCHRAW, G., & RUNNING, R. (1999). Cognitive psychology and instruction. Columbus, OH: Merrill Prentice Hall.

DIHOFF, R. E., BROSVIC, G. M., & EPSTEIN, M. L. (2003). The role of feedback during academic testing: The delay retention effect revisited. The Psychological Record, 53, 533-548.

DIHOFF, R. E., BROSVIC, G. M., & EPSTEIN, M. L (2004). Provision of feedback during preparation for academic testing: Learning is enhanced by immediate but not delayed feedback. The Psychological Record, 54, 207-232.

EPSTEIN, M. L., & BROSVIC, G. M. (2002). Students prefer the immediate feedback assessment technique. Perceptual and Motor Skills, 90, 1136-1138.

EPSTEIN, M. L., BROSVIC, G. M., DIHOFF, R. E., LAZARUS, A. D., & COSTNER, K. L. (2003). Effectiveness of feedback during the testing of preschool children, elementary school children, and adolescents with developmental delays. The Psychological Record, 53, 177-195.

EPSTEIN, M. L., EPSTEIN, B. B., & BROSVIC, G. M. (2001). Immediate feedback during academic testing. Psychological Reports, 88, 889-894.

EPSTEIN, M. L., LAZARUS, A. D., CALVANO, T. B., MATTHEWS, K. A., HENDEL, R. A., EPSTEIN, B. B., & BROSVIC, G. M. (2002). Immediate feedback assessment technique promotes learning and corrects inaccurate first responses. The Psychological Record, 52, 187-201.

FARMER, M. E., KLEIN, R., & BRYSON, S. E. (1992). Computer-assisted reading: Effects of whole-word feedback on fluency and comprehension in readers with severe disabilities. Remedial and Special Education, 13, 50-60.

FISCUS, R. S., SCHUSTER, J. W., MORSE, T. E., & COLLINS, B. C. (2002). Teaching elementary students with cognitive disabilities food preparation skills while embedding instructive feedback in the prompt and consequent feedback. Education & Training in Mental Retardation & Developmental Disabilities, 27, 55-69.

GAST, D. L., DOYLE, P. M., WOLERY, M., AULT, M. J., & FARMER, J. A. (1991). Assessing the acquisition of incidental information by secondary-age students with mental retardation: Comparison of response prompting strategies. American Journal of Mental Retardation, 96, 63-80.

GRIFFEN, ?., SCHUSTER, J., & MORSE, T. (1998). The acquisition of instructive feedback: A comparison of continuous presentation schedules. Education & Training in Mental Retardation & Developmental Disabilities, 33, 42-61.

HITCHCOCK, C. H., & NOONAN, M. J. (2000). Computer-assisted instruction of early academic skills. Topics in Early Childhood Special Education, 20, 145-158.

KULHAVY, R. W., & ANDERSON, R. C. (1972). Delay-retention effects with multiple choice tests. Journal of Educational Psychology, 63, 505-512.

KULIK, J. A., & KULIK, C. C. (1988). Timing of feedback and verbal learning. Review of Educational Research, 58, 79-97.

MCDONNELL, J., THORSON, N., ALLEN, C., & MATHOT-BUCKNER, C. (2000). The effects of partner learning during spelling for students with severe disabilities and their peers. Journal of Behavioral Education, 10, 107-121.

MECHLING, L., & GAST, D. L. (1997). Combination audio/visual self-prompting system for teaching chained tasks to students with intellectual disabilities. Education & Training in Mental Retardation & Developmental Disabilities, 32 138-153

NATIONAL COUNCIL OF TEACHERS OF MATHEMATICS. (2000). Principles and standards for school mathematics. Reston, VA: Author.

NOONAN, M. J., & MCCORMICK, L. (1993). Early intervention in natural environments. California: Brooks-Cole Publishers.

ODOM, S., CHANDLER, L, OSTROSKY, M., & MCCONNELL, S. (1992). Fading teacher prompts from peer-initiation intervention for young children with disabilities. Journal of Applied Behavior Analysis, 25, 307-317.

PARKER, M. A., & SCHUSTER, J. W. (2002). Effectiveness of simultaneous prompting on the acquisition of observational and instructive feedback stimuli when teaching a heterogeneous group of high school students. Education & Training in Mental Retardation & Developmental Disabilities, 37, 89-104.

PERKINS, V. L. (1998). Feedback effects on oral reading errors of children with learning disabilities. Journal of Learning Disabilities, 21, 244-248.

PRESSEY, S. L. (1950). Development and appraisal of devices providing immediate automatic sorting of objective tests and concomitant selfinstruction. The Journal of Psychology, 29, 417-447.

RICKEY, S. (1995). The seven principles in action: Improving undergraduate education. Massachusetts: Anker Publishing.

ROBIN, A. L. (1978). The timing of feedback in personalized instruction. Journal of Personalized Instruction, 3, 81-87.

SCHUSTER, J. W., MORSE, T. E., GRIFFEN, A. B., & WORLEY, T. (1996). Teaching peer reinforcement and grocery words: An investigation of observational learning and instructive feedback. Journal of Behavioral Education, 6, 511-533.

SKINNER, B. F. (1968). The technology of teaching. New York: Meredith Corporation.

SPARROW, S., BALLA, D., & CICCHETTI, D. (1984). Vineland adaptive behavior scales. Circle Pines, MN: American Guidance Service.

SURBER, J. R., & ANDERSON, A. C. (1975). Delay-retention effect in natural classroom settings. Journal of Educational Psychology, 67, 170-173.

TURNER, L. A., & MATHERNE, J. L. (1994). The effects of performance feedback on memory strategy use and recall accuracy in students with and without mental retardation. Journal of Experimental Education, 62, 303-314.

UNITED STATES DEPARTMENT OF HEALTH AND HUMAN SERVICES. (2002). Health, United States, 1998. Hyattsville, MD. Publication: PHS 98-1232. Author.

ROBERTA E. DIHOFF

Rowan University

GARY M. BROSVIC, MICHAEL L. EPSTEIN, and MICHAELJ. COOK

Rider University

Address correspondence to Gary M. Brosvic, Department of Psychology, Rider University, Lawrenceville, New Jersey 08648. (E-mail: Brosvic@Rider.edu).

Copyright Psychological Record Winter 2005
Provided by ProQuest Information and Learning Company. All rights Reserved