The Effects of Metacognitive Training on Performance

in Self-Directed Learning Situations

Conrad Gleber

Florida State University

The purpose of this pilot study was to determine the effects of teaching metacognitive strategies on performance and knowledge acquisition in a self-directed learning situation. The study was designed as a related sample of 56 university students enrolled in a beginning photography course for non-art majors. All participants were treated equally and were subject to the same conditions. The treatment was designed to introduce metacognitive strategies into the exercises used to teach photography Metacognitive awareness was measured twice by survey, prior to and after the treatment and lessons. They determined the extent to which students had developed or used new metacognigive strategies. Results indicated that the treatment had no effect on learning. Also, the differences between the subjects pre-condition and post-condition metacognitive awareness was normally distributed based on a self report survey of 55 question. The significant quantitative differences in participants’ performance and development led the researchers to conclude that instructional strategies which teach students to practice metacognitive skills contribute to an overall increase in performance in a course where students are expected to manage their own direction for learning.

The Effects of Metacognitive Training on Performance

in Self-Directed Learning Situations

Metacognition is the ability of one to reflect, control and understand, in a self-aware mode, one’s own learning and cognition (Schraw and Dennison, 1994). For some, it is the behavior that is the basis of our humanity. Philosophers for ages have been intrigued by the self-reflective nature of human thought. Today the application of metacognitive research has become useful beyond theoretical models. The result is metacognitive strategies that students learn and employ to turn their reflection and judgment on their knowledge (Metcalfe and Shimamura, 1994). It is a self-awareness of how thinking and knowing takes place through managing one’s own experience. Metacognitive strategies lead to the individual taking charge of their learning. They include connecting new information to former knowledge, deliberating on how to select what to learn and planning and evaluating what is learned. Research suggests that making students aware of these strategies may be useful, that students can develop strategies on their own and that their use increases learning and performance (Blakey, 1990).

This pilot research focused on training students in the use of metacognitive strategies and the effect on learner’s recall, comprehension and application of content. The training was designed as instructor led lessons which focused students attention on imagery and word cues. The lessons on imagery and cues relied on previous research which found that dual coding of information assists in higher levels of recall and understanding. Abstract information is much better remembered when it is imaged or there is a memory aid which helps recall the relationships (Hodes, 1992). Both strategies, imagery and word cues, use techniques that students can use to create a second context for the content to be learned.

The long range goal of this research is not only to find what effect teaching metacognitive strategies will have on performance but to identify a method of teaching metacognitive strategies that is highly effective. Indicators of effect will be the performance on a posttest which will measure for each of three levels of knowledge in Bloom’s taxonomy:

The primary expectation of this experiment was that students who have and use metacognitive strategies will perform better on recall, comprehension and application tasks than students who do not. It was also expected that regardless of the source of the metacognition, strategies which students used to learn what they know would increase the depth of their knowledge. Demands that are put on students to be self-reflective and creative also require them to be aware of their own conceptual underpinnings. It is the context of certain learning situations that puts pressure on students to develop an awareness of metacognition; therefore, it was expected that post treatment surveys would show significant differences to prior condition surveys

Research in metacognition has followed two paths, one details how it develops naturally throughout life and the second concentrates on the training potential of metacognitive strategies (Xiaodong, 1994). Past research has not looked very close at the possibility that students must develop strategies such as imagery to help them learn how to perform in tasks which require creative solutions. Most previous research on metacognition site the phenomena in the assimilation of complex information rather than tasks which are intended to produce personal or novel products. This study was situated in a studio art class and was different than most research and provided a new context for metacognitive strategies training.

Method

Eighty students in a sophomore photography class for non-art majors at Florida State University voluntarily participated in this study . There were 32 males and 48 females, all but 8 students were undergraduates under 25. There were no graduate students. The research conformed to the guidelines set forth by the human subjects committee of the university. Participants were informed that the treatment and survey instruments were voluntary and that all data collected were coded and anonymous.

A questionnaire was used to identify the entry level photographic skills of all the participants. The questionnaire had four items which participants used to self report their photographic skills. Participants were then stratified into experienced, familiar and novice skill groups before being randomly assigned to a treatment or control group.

The posttest (Appendix B) was the third item and all students took the test three weeks after the metacognitive training session was held for the treatment group. It was designed to measure students’ recall, comprehension and applicability of photographic rules and methods. Questions related to recall were a combination of multiple choice, fill in the blank and essay questions. Comprehension and application knowledge were tested with six situation specific problems. The problems were made up of a photograph and a scenario related to the image and asked if the camera settings described would yield certain results and if not, suggest the correct changes. Answers required that students have an understanding of the camera controls and how to apply their comprehension for desired results. Of the 23 answers to be given on the test, seven required recall, nine required comprehension and seven required application knowledge. The tests were graded by the instructor.

After participants were stratified into either an experienced, familiar or novice group they were randomly assigned to one of two groups, a treatment or control group. Students who were in the treatment group were led to another classroom while the control group watched a videotape in the regular classroom. The treatment, a one-hour training session, was directed by the instructor and a researcher. They explained the concept of metacognition and demonstrated four exercises to practice the strategy of mental imaging. The relevance of mental imaging to problems in photography was suggested and discussed with the participants.

The first exercise was a demonstration of chunking and imagery mnemonics. The students were shown a list of names and the subjects which made them famous on a chalkboard. After the list was erased they were asked to recall the list of names and subject matter and write them down on paper. Later after completing another exercise, the group returned to the list of names. Students and the instructor created a visual image of the list of photographers and their work. The image was erased and students were again asked to recall the names, this time using the image as a mnemonic.

Word cues to define imagery made up the second exercise. Students were asked to close their eyes and listen to sentences which suggested familiar sensory experiences. Another set of cues were questions like "If an uppercase version of the letter n were rotated 90° clockwise, would it be another letter?"

The third and fourth exercises had students refer to a handout (see Appendix C) . For the third exercise students were to look at three pairs of perspective views of three-dimensional objects in different orientations in space. Students were asked to look at the diagrams and decide which pairs were the same figure. The fourth exercise had students look a diagram of gears on a bicycle and imagine the correct ratio given three different scenarios. The exercises were followed with a discussion of how they used mental imagery to solve the problems.

After the control group finished viewing a videotape on the life of a famous photographer both groups were brought together in the same classroom. The instructor proceeded with a lecture to both groups on the optical relationship of exposure reciprocity. The lecture was followed by reading and exercise assignments and scheduled date (two weeks) for the posttest. Two weeks after the posttest students were asked to complete the metacognitive awareness survey for a second time.

The study was designed to use a 2 X 3 factorial design (group x pre-MAI x post-MAI x Posttest). Statistical procedures were designed to look at a comparison pre and post metacognitive scores within groups, the interaction of metacognitive scores and posttest results and posttest scores between groups using repeated Analysis of Variance (ANOVA) to test for differences.

Results

The foremost variable of interest was posttest performance on knowledge recall, comprehension and application. As shown in Table 1, both groups had nearly equal mean and standard deviation scores. The treatment and control groups began as two equal groups of 40 each. Due to absenteeism the treatment group was left with 13 participants and the control group 27.

Table 1.

Treatment Group (n=13)

Control Group (n=27)

The second variable of interest concerned the level of students metacognitive awareness. Both groups were given pretest and posttest surveys designed to measure participants awareness of metacognition. The scores were divided into 8 areas and the scores in Table 2 reveal modest variation within and between participants. In some cases the mean score decreased after the treatment.

Table 2

CK, conditional knowledge; DK, declarative knowledge; PK, procedural knowledge; P, planning; IMS, information management skills; M, monitoring; DS, debugging strategies; E, evaluation.

A review of a scatterplot which distributed all the posttest scores showed a normal distribution with a slight negative skew (-.06). The assumption of homogeneity of variance using Levene’s test found variance to be appropriate for each group and condition.

The first hypothesis postulated that the treatment group given a workshop in metacognitive strategies would perform better on posttest measures of performance across three knowledge categories. The results indicated that the treatment group scored lower than the control group. Since the results of the data went in the opposite direction of the hypothesis, no further statistical tests were done.

The second hypothesis stated that the group which received a metacognitive workshop would increase their awareness and use of metacognitive strategies both between and within groups. The treatment group had a metacognitive mean score that did not change. It was 6.92 before and after they took the posttest. The conditions of the directional hypothesis were not met; therefore, no additional statistical tests were warranted. The control group scored a mean of 6.48 before and increased slightly to 6.59 after the posttest. Although these results followed the hypothesized direction, the mortality of participants seriously affected the ability of these results to be generalized beyond this pilot study; therefore, no statistical tests were reported. Absenteeism cause the sample size to become very low and incomplete data had to be disregarded.

Discussion

The purpose of this pilot study was to investigate the degree to which training students to use metacognitive strategies can affect learning. The results showed that training had no significant effect (both groups finished with nearly equal scores) on the development of strategies or on learning. Even so, the results do not lead to the conclusion that there could never be a significant causal relationship. This conclusion stems from other research in the field that parallels these results. Early research by Brown (1978) attempted to teach specific metamemory strategies similar to our imagery exercises and when effects were less positive than expected she recommended to others that more general metacognitive strategies be taught (Xiadong, 1994).

Explanations of the results must begin with the fact that there was no treatment. The one-hour workshop was not effective enough to have participants develop and use metacognition to any significant degree. It is apparent from this study that we must consider the design and degree of intervention that may be required to make a significant difference in the way students will learn and use metacognitive strategies.

A second hypothesis proposed that training would familiarize the learners to metacognitive strategies and would result in the increased use of metacognition especially in pressure situations. The training was incomplete and did not transfer to the participants as methods for learning in this course. The design of future training should incorporate a demonstration of how certain strategies do assist learning. The worth of metacognitive strategies was demonstrated when most of the treatment group remembered the list of names after building a visual mnemonic together.

Considerations for the replication of this study include a number of issues that would correct some of the shortcomings of this experiment. One shortcoming was the number of absentee participants (40 students). The result was that the sample size had lowered the statistical power below an acceptable amount and challenged the external validity of the study. When one considers doing more or longer training sessions then the study might be better served by a related sample design that would span the 14 week semester. Absenteeism would not be as critical an issue especially if the instructor and the designer imbed the training into the existing course. The additional time would also give the researchers an opportunity to conduct qualitative data gathering that might yield a more thorough analysis than a single quantitative method. This would answer findings of other researchers that the relationship of metacognitive strategies and performance is too complex to be analyzed by a straightforward quantitative design (Reber, Noah, and Nolan 1998).

There is strong support for the study of metacognition and instructional interventions; consequently, this pilot study has significant implications for other researchers who feel that metacognition is a necessary part of learning that must be taught. This study demonstrates how difficult it is to promote metacognition through explicit instruction yet it represents a challenging area of research. The rewards of understanding the strategies that would enable learners to be more efficient at converting information to knowledge would be many. This research has some very timely implications for the growing number of students who are expected to be learning at a distance in a hypertext environment. The research will not be easy. The area is complex and not easily defined by one overall generic strategy. However, understanding how students interact with and control their learning is a fruitful area to research. It is very important to follow this and other research with conclusive causal descriptions of what can and will be effective training.of metacognitive strategies. The hope is to find ways that make the most efficient use of time while learning. The results of this research area will impact the instructional design and learning community directly and will help them impact their society.

References

Blakey, E., & Spence, S. (1990). Developing Metacognition. ERIC Clearinghouse on Information Resources Item: EDO-IR-90-6

Brown, A. L. (1978). Knowing when, where, and how to remember: a problem of metacognition. In R. Glaser (Ed.), Advances in Instructional Psychology (Vol. 1, ). Hillsdale, NJ: Lawrence Erlbaum Associates.

Driscoll, M. (1994). Psychology of Learning for Instruction. Needham Heights, MA: Allyn & Bacon.

Hodes, C. (1992). The Effectiveness of Mental Imagery and Visual Illustrations: A Comparison of Two Instructional Variables. Journal of Research and Development in Education, 26, (1), 46-56

Metcalfe, J. & Shimamura, A. P. (1994). Metacognition Knowing about Knowing. Cambridge, MA: The MIT Press.

Reber, L. P., Noah, D. & Nolan, M. (1998, ). Metaphors as Graphical Representations within Open-Ended Computer-Based Simulations. Paper presented at the American Educational Research Association, San Diego, CA.

Schraw, G., & Dennison, R. S. (1994). Assessing Metacognitive Awareness. Contemporary Educational Psychology, 19, 460-475

Xiaodong, L. (1994, ). Metacognition: Implications for Research in Hypermedia-Based Learning Environment. Paper presented at the Association for Educational Communications and Technology, Nashville, TN.

Appendix A

Appendix B

Appendix C

Should learners be challenged to think about how they think? How would it benefit the student? Should instructional designers include the teaching of metacognitive strategies? If so, should students be taught strategies before the delivery of the designed instruction or should the strategies be imbedded in the content of the instruction?

The postest measure questioned students on recall, comprehension and application of technical knowledge. There were a total of 23 questions to answer and the overall results (n=80, M= 11.3, SD= 5.05) were tabulated by one person.

The treatment group (n=13, M=12.23, SD=6.09) and the control group (n=27, M= 12.3, SD= 5.86)) had nearly equal results.

Given the results the descriptive statistics contribute to a decision that there was no treatment effect.