UW Oshkosh
menu
Future Students adult non-traditional Parents and Family Current Students Faculty and Staff Visitors and Community
Teaching Forum - Student Collection of Primary Data:

A Journal of the the Scholarship of Teaching and Learning: Sunday October 26, 2008 Edition

Search Teaching Forum Archives
 


 
 

Student Collection of Primary Data:
A Pedagogical Tool to Improve Comprehension

By Lisa Giddings

Key Words:  primary data, student learning, Economics, applied learning.

Introduction

     University and college courses can fail for many reasons. Students, for example, may not grasp the theory, understand difficult material, or care about the subject itself and, therefore, not invest time or energy into learning. In the case of my field, economics, W. Lee Hansen, Michael Salemi, and John J. Siegfried (2002: 464) argue that introductory courses fail "because [they do] not teach students how to apply economics to their personal, professional, and public lives… students never master the basics." I suspect this to be true, regardless of the subject, particularly for theoretically-oriented, less applied courses. One way to get students invested in the material, to master the basics, and to see how it applies to their own personal lives, is through the collection and interpretation of primary data. 

     University and college courses can fail for many reasons. Students, for example, may not grasp the theory, understand difficult material, or care about the subject itself and, therefore, not invest time or energy into learning. In the case of my field, economics, W. Lee Hansen, Michael Salemi, and John J. Siegfried (2002: 464) argue that introductory courses fail "because [they do] not teach students how to apply economics to their personal, professional, and public lives… students never master the basics." I suspect this to be true, regardless of the subject, particularly for theoretically-oriented, less applied courses. One way to get students invested in the material, to master the basics, and to see how it applies to their own personal lives, is through the collection and interpretation of primary data. 

     This research explores the effects of gathering and using primary data in the classroom as a pedagogical tool to better engage students with the course material. An experiment was conducted over the course of three semesters comparing a treatment and a control section of a course entitled "Women in the U.S. Economy," a 300-level general education economics course taught at the University of Wisconsin-La Crosse. Students in the treatment group interviewed couples using a survey adopted from Phillip Blumstein and Pepper Schwartz' American Couples (1983). Class data was then compiled and students were required to use the data to explore issues discussed in the course through homework and papers. Members of the control group analyzed the data but did not take part in collecting the data. Pre- and post-tests were employed to measure changes in student understanding of the major concepts developed in the course. 

Results of ordinary least square regression analyses show a significant difference in learning outcomes between the control and the treatment groups. This evidence indicates that gathering the primary data improves student comprehension of some of the basic concepts and ideas covered in the course. I believe that this is because the act of gathering the primary data serves to engage students in the learning process and, as a result, gain a greater understanding of the basic concepts of the course. Because the control group did not actually gather the data, they did not see the material in a personal way and apply concepts to their own lives. This evidence suggests that that actually collecting the survey data significantly improved the treatment groups' overall understanding of the basic building blocks of the course. It is believed that this, then will aid in student comprehension of the higher-level and applied concepts studied in the course. Although my field is economics, this technique would be beneficial to students in any field. The idea, simply, is to engage students with individuals in the real world through primary data that they collect through surveys or interviews, as they relate to the course material.

II. Background: The Course

            "Women in the US Economy" is a general education course in the department of economics at the University of Wisconsin-La Crosse. The course has no prerequisites, despite being a 300-level writing-intensive course, and there are many incentives for students in the college of business (which houses our economics department) to take the class. It fulfills one of their general education requirements, one of their writing-intensive upper-level course requirements, is listed in the college of business, and is an economics course (for majors and minors). Plus, I suspect, students expect it will be easy. As a result, our department offers the course often (sometimes twice per semester to as many as 80 students) and, interestingly, despite being cross-listed in the women's studies department, nearly 50 percent of the students enrolled in the course are men.

            The course covers three major aspects of issues related to women and the United States economy: the labor market, the household, and poverty. It combines historical information with statistics and economic theory. One goal of the course is to improve critical thinking skills. In this context, that means being able to recognize that few issues are black or white, to be able to use logic, as well as identify reputable sources to defend whatever position they decide is most correct. The issues presented in the course lend themselves to exercises in critical thinking. For example, while we can calculate a wage gap between men and women, the size of the gap is subject to measurement issues. The cause of the gap is likely due to many factors including differences in schooling and preferences between men and women as well as, possibly, discrimination. Furthermore, prominent scholars disagree about the issue, providing fodder for good classroom discussion.

Like Hansen, Salemi, and Siegfried's (2002: 464) description of introductory economics courses, this course failed. Despite providing students with current, accessible articles and data, they simply did not believe the information presented. Additionally, students did not seem to find the issues presented in the class relevant, important, or even interesting.

My first response was to provide them with recent newspaper articles (including local, national and international information) reinforcing the class material in addition to the textbook. When this did not work, I presented current data from the Census Bureau. Students would do the homework and interpret their graphs correctly, but still, in class discussions, their opinions remained the same: men and women are essentially equal in all areas public and private. In discussion, they would articulate that they believed there was no difference between men's and women's wages or occupations. I could not even get to the critical thinking portion of the material, for example, exploring whether any observed inequality is a result of women's choices or discrimination. Even if they conceded that inequality existed, they were convinced that it did so in the distant past and was, therefore, irrelevant.

            This problem was different than the usual run-of-the-mill undergraduate laziness, lack of time spent studying, or over-commitment that generally leads to an unproductive classroom. It was not that students in this class were not doing the work. Also, it was not necessarily a matter of learning. My hypothesis is that their disbelief in the issues presented was related to three distinct factors that all related to their level of engagement with the material. First, the students had not yet experienced most of the issues that I presented to them. In their experience women do not get paid less (they all work for less than $10/hour); women do not experience occupational segregation (they nearly all work at bars, restaurants, or at the university); there was no obvious "old boys network" working to advance men and discriminate against women (the bar and restaurant owners certainly want to hire women and they did not work in jobs with hierarchies that were apparent to them); neither men nor women did housework (either in their apartments or their dorms); none of them had children so they have not had to make choices about careers versus family or any real division of labor between spouses; none were in poverty; and their university population was 99 percent white, so affirmative action is something they read about only in Newsweek. They simply could not relate to the material.[1]

            Second, from their perspective, the information was an historical relic. Most of their mothers worked while they were young and so the phenomenon of women moving into the labor market did not represent a critical juncture in their personal histories. Third, because of reasons one and two, they did not make the connection between the course and any relevant public policy that could affect their lives. In summary, the students did not believe the material because they have not experienced the issues, were not engaged with it and could not imagine it affecting their lives in any way in the future.

III. Primary Data as a Pedagogical Tool

            Engaging students through collecting and interacting with primary data is essentially an extension of active learning or "problem based learning," and is certainly not new. In fact, getting students to perform experiments and collect primary data is such a common technique in the sciences, it has not been explicitly published as a pedagogical tool.

A problem-based learning (PBL) curriculum "provides authentic experiences that foster active learning, support knowledge construction, and naturally integrate school learning and real life" (ASCD, 2005). It is claimed to be a practical strategy for fostering deeper, critical, active-learning strategies (Ramsden, 1992). PBL also provides a stimulus for learning (Boud and Feletti, 1991). The use of primary data in the classroom can support problem based learning by providing students with hands-on evidence with which to explore complex problems. Peter Milbury and Brett Silva (1998) developed activities that drew upon "authentic, primary sources" from the Library of Congress American Memory Website. Their goal was to actively engage students in research and critical thinking. According to Milbury and Silva, the primary sources helped students develop good habits when working with a large amount of information about "fuzzy" problems with no "preconceived" or "textbook" solutions. Furthermore, students could not simply "cut and paste" information from the Internet because they were required to use the data at hand.

Carol G. Johnston, Richard H. James, Jenny N. Lye, and Ian M. McDonald (2000) evaluated a project that incorporated problem-based learning in an economics classroom. They found that the project generated a "positive student reaction . . . and evidence of increased student preparation for tutorials" (p. 13). The authors did not, however, find any measurable improvement in exam scores.

Several studies have documented the explicit effects of using primary data in the classroom. In a course titled "Early United States History," Susan Leighow Meo (2000) had undergraduate history students read a series of primary sources and analyze them in a journal. One result of this interaction with primary sources was to "allow students to see history as an ongoing process of constructing the past, rather than a fixed body of knowledge." According to Meo, the use of primary sources improved student ability to identify different authors' perspectives and evaluate the credibility of different sources. Additionally, she claims that teaching using primary sources modeled the pedagogical strategies for future teachers in the classroom.

Wynell B. Schamel (1998) used primary source documents in a secondary school history course. By using primary sources, Schamel claims that students learned that the record of historical facts reflects the "personal, social, political, or economics views of the participants who created the sources" and that their own biases come into play upon their review. A benefit of using the primary sources collection, according to Schamel, is the development of broad cognitive and analytical skills.

IV. The Experiment

This experiment follows a typical strategy of education research by first pretesting students, then applying an innovative pedagogical strategy to a treatment group, followed by posttesting the students in order to see of the treated group learned differently than the control group. Using this logic, the learning process can be thought of as similar to a production process in which the output is increased aptitude (Salemi and Tauchen, 1980: 42). In this model the inputs to the learning process are student aptitude at the beginning of the course, the time a student spends studying, and the characteristics of the learning environment including the treatment itself.

Having identified a problem in my course relating to student engagement rather than to effort, I designed a cross-sectional household survey that students could administer themselves and then use the aggregate class data in homework assignments and papers in order to foster some commitment to the material. It is hypothesized that actually gathering primary data engages the students in a way that develops an understanding of the issues more effectively. Gathering the data is a significant step toward engaging with the material, not simply analyzing data that has been handed to you. The students had already worked with much more accurate data from the Census – data with a much larger and random sample – but were still not engaged in the material or the issues. It is believed that when students go out to real people and ask questions, that the issues will become important to them.

The survey is based on the concepts presented during the course as well as questions posed by Blumstein and Schwartz (1983).[2] It includes basic demographic information such as race, income, and education level, as well as information on who performs domestic duties and a series of questions on participant opinions on various issues. This survey was ideal for exploring the issues presented in the course and, ironically, the results ended up exaggerating the information I had already provided to them, reinforcing my points. For example, the class surveys regularly result in wage gaps between men and women that are larger than the national average.

This assignment was given to the treatment group on the first day of class and students were expected to turn in their survey results within the first week of classes. They were required to choose a couple that was currently cohabitating, not as roommates, but not necessarily legally married. I compiled the survey results and made the entire data set available to the class. Students were then required to use the data set in several homework assignments.[3]

The assignment of gathering the primary data took a relatively small amount of time and was performed entirely outside of the classroom. Each survey could be conducted over the phone and took approximately 10 minutes to complete. While the members of the control group had no corresponding outside activity, it is expected that the act of gathering the data itself rather than the time spent gathering the survey data served to engage the students in the treatment group as compared to the control group.

V. Data and Methodology

The experiment was conducted over the course of three semesters. During each of the three semesters, the two sections of the course were taught back-to-back on Tuesday and Thursday afternoons. One section was randomly selected to conduct the survey. This section was designated the "treatment group." Semester 1 consisted of a "treatment section" of 25 students and a "control section" of 22 students. Semester 2 included 22 students in both the treatment and control sections. Semester 3 included 24 students in its "treatment section" and 18 students in its "control section." Pooled, the three semesters resulted in 71 students in "treatment sections" and 62 students in the "control section."

In order to evaluate the effects of gathering the primary data, pretests and posttests were administered to six sections of the course during three separate semesters.[4] Pretests were administered during the first week of classes and posttests were administered during the last week of classes. The test contained 31 questions related to the class material, including quantitative, qualitative, and open-ended components. The questions were designed to measure student understanding of the basic definitions and concepts discussed in the class. The instrument did not capture deep understanding or the ability to apply knowledge gained from the course. However, as stated, in order to get to higher learning of the material, one must first grasp the basic concepts. Furthermore, I had identified that students in the course were not getting these basic concepts. For this study, only the quantitative answers were evaluated. A statistical algorithm was employed to grade the quantitative questions of the entry and exit exams in order to assure consistent grading.[5]

            It was expected that the scores on the entry exam would not differ significantly between the treatment section and the control section, and that the scores would improve for both the treatment and the control group. I hypothesized that because members of the treatment groups were more engaged with the material, their posttest scores would be significantly higher than that of the control group on average.

            To control for student aptitude, several proxy measures were included in the regression analysis. [6],[7] These included the student's cumulative grade point average, the final grade in the course, and their English, math, and composite ACT scores. It is expected that students that took the course as an elective would be more likely to perform better than those that took the course as a requirement, so this knowledge is also included in the analysis. Information on any student input such as time spent studying was not available.

            A dummy variable for the sex of the student was included. Research suggests that males and females differ in their performances on exams. Mary L. Williams, Charles Waldauer and Vijaya G. Duggal (1992) found some evidence that males outperformed females on essay exams in introductory courses. They also found no difference in overall quantitative skills between men and women. Benjamin Greene (1997) found no evidence that females are better at more verbal evaluations of economic knowledge. Because this study was based on quantitative questions, it was not expected that the variable "Sex" will have any significant effect on either the entry or the exit scores.

            Age may have a positive effect on both the entry and the exit scores as it may proxy for year in school or capture some measure of maturity or even possibly seriousness on behalf of the student in terms of study habits. Also included were dummy variables indicating whether a student was an economics major, and if they were not enrolled in the college of business. It is expected that economics majors and those enrolled in the college of business would perform better than students who were not economics majors or who were not enrolled in the college of business. Table A1, which can be found in the Appendix, provides a list of the variables and their descriptions. Table A2, which can be found in the Appendix, presents the means and standard deviations of the variables broken down by the treatment/control groups and semester.     

            Table A3 provides the results from difference of means tests for the entry test and exit test results for the three semesters combined. Difference of means tests show that there is no significant difference between the treatment and the control groups on the pretest in any of the semesters examined.[8] There is, however, a statistically significant difference between the treatment and control groups in the entire sample in the results of the posttest. On average, members of the treatment group scored 3.725 points higher than those in the control group on the posttest. This difference was significant at the 0.10 level.

            In order to test for the effects of gathering the primary data on the student's posttest scores while controlling for the other inputs into the learning model, particularly student aptitude, ordinary least squares regressions were run with posttest scores as the dependent variable. Equation 1 represents the model:

EQN 1

Posttest = â0 + â1 Treatment + â2Fall + â3 Sex + â4 Age +  â5NonCBA + â6EconMajor + â7CumGPA + â8Grade + â9ACTEng + â10ACTMth + â11ACTComp + â12Elective + â13Required + å

            The dummy variable representing the treatment was included in the regression. It is expected that the treatment (gathering primary data) will have a significant positive effect on the posttest results. That is, the better a student performed on the pretest, the better they will perform on the posttest.

VI. Results

            The results of the ordinary least squared regression indicate the treatment had a positive and significant effect on the posttest results. Table 1 presents the results of standard ordinary least square regression.

Table 1: Ordinary Least Squares Regression Results, Dependent Variable: Posttest t-statistics are in parentheses.

Coefficient

Pooled Sample

Treatment

3.234
(2.24)

Fall

-0.986

(-0.63)

Sex

-0.539

(-0.40)

Age

0.307

(1.67)

NonCBA

-0.977

(-0.58)

Econmajor

3.648

(2.07)

CumGPA

2.511

(1.28)

Grade

-1.876

(-1.32)

ACTEng

-0.643

(-2.00)

ACTMth

-0.374

(-1.28)

ACTComp

1.030

(2.02)

Elective

-0.764

(-0.20)

Required

-0.674

(-0.44)

Constant

66.712

(10.55)

R-Squared

0.1625

N

133

Source: author's calculations.

            In addition to the treatment, age, being an economics major, and the comprehensive ACT score had a significant positive effect on the posttest outcome. Age improves the posttest score by less than one half of a point whereas having an economics major improves students' posttest scores by more than three points, and the ACTComp improves posttest scores by over one point. One could infer that students that have declared economics as a major have greater aptitude toward the subject or are inclined to enjoy the material more than non-majors and, therefore, performed better on the posttest. Interestingly, the English component of the ACT test had a significant negative effect on the posttest score.

            Although these variables did not affect the posttest results significantly, whether or not the course was taken in the fall semester, being a woman and being a non-business major resulted in lower posttest outcomes on average in the overall sample. Furthermore, one's grade in the course, while insignificant, was negatively related to the posttest results. This is surprising. While the posttest measures only basic concepts, it is expected that if students master the basic concepts, they will do better at higher-level thinking and more applied exams over the material. This evidence indicates that while those individuals in the treatment group may have significantly improved their understanding of the basic concepts, they could not extend this improvement to the more higher-level and applied concepts covered in the course and measured in the final grade.

VII. Conclusions

            This experiment showed that when students gather relevant primary data as a part of their classroom experience, their understanding of the basic course material is significantly improved. This may be an indication of greater engagement with the course material and, as a result, a better comprehension of the ideas presented in the class. Even after controlling for other factors that might affect one's ability to perform on the posttest, the treatment of actually gathering the survey data improves student scores on the posttest. This experiment, however, was limited to only measuring the effect of gathering the data. I suspect that using a survey to explore the issues of the course affected all of the students, not just the ones who gathered the data.      

            In addition to becoming more engaged with the material, however, there are several other spillover benefits from using primary data as a pedagogical tool. As stated in the review of the literature, both gathering and using survey data models the research strategies for future economists. As was shown in the literature, this would certainly be applicable to any discipline as students would be expected to use the appropriate research methods in their data collection. By conducting the survey and inputting the results, students learn about data. They learn that data is rarely "clean" and, depending on how the researcher deals with these issues, he or she may get different results. Furthermore, students learn the importance of what questions are asked, how they are asked, and how statistics can tell different stories. One student, for example, became interested in the difference between reporting the mean and the median when exploring the wage gap between men and women. In my course, students also learned to use a statistical software package, in this case Excel or SPSS, which would be helpful to them in a future career in the field. This too, could be generalized to different fields and different tools of analysis.

            Further research on this pedagogical technique could investigate student engagement in the course material using qualitative methodology. This analysis focused on measuring student comprehension of basic quantitative concepts from the course. It would be interesting to measure changes in more in-depth student understanding of the material. I suspect the engagement that this technique provokes would be more likely to motivate students to commit themselves to the higher-level material in the course. Whereas a qualitative study may work well here, the measurement tool used in this analysis was not sensitive enough to get at this change.

            Getting students engaged with the material is the one true goal for any teacher of any subject, of any age. As such, the technique could be applied to any discipline. Unfortunately, the survey created here cannot simply be cut-and-pasted into one's syllabus. If you're not an economist, that is. The idea that can be transported from one discipline to the next, however, is the idea of having students gather data themselves and use it in a meaningful way to simply get them involved in their own education.

Appendix

Table A1 Variable Definitions

Variable

Description

Posttest Score

A grade on the quantitative portions of the entry and exit instrument. Continuous variable ranging from 0 to 100.

Difference

A variable measuring the difference between the pretest and the posttest scores.

Treatment

A dummy variable for the treated section. 1 = Treatment, Control = 0.

Fall

A dummy variable for Fall/Spring. Fall = 1 Spring = 0.

Sex

A dummy variable for sex. 1= male.

Age

Age of the individual at the time of taking the course.

CumGPA

Cumulative grade point average. Range: 0 to 4.0.

Grade

Grade in the course ranging from 0 to 4.0.

NonCBA

A dummy variable indicating if the individual is not in the college of business. 1 = non business 0 = business.

EconMajor

Is the individual an economics major? Dummy variable 1 = yes, 0 = no.

ACT English

Score on the English portion of ACT, range is 1-36

ACT Math

Score on the Math portion of ACT, range is 1-36

ACT Cumulative

Cumulative score on ACT exam, range is 1-36

Elective

Dummy variable indicating whether the student reports taking the course to fill an elective requirement, 1 = yes, 0 = no.

Requirement

Dummy variable indicating whether the student reports taking the course to fill a college of business or major requirement, 1 = yes, 0 = no.

Table A2. Table of Means, Standard Deviations are in Parentheses

All

All

Semester 1

Semester 2

Semester 3

Variable

Treatment

Section

Control

Section

Treatment Section

Control Section

Treatment Section

Control Section

Treatment Section

Control Section

Pretest

64.913

(12.11)

62.554

(12.57)

60.533

(10.71)

56.970

(13.96)

63.442

(13.66)

65.422

(10.09)

70.823

(9.86)

65.873

(11.73)

Posttest

78.706

(6.08)

74.981

(9.17)

79.914

(5.87)

74.221

(9.02)

75.996

(6.19)

75.768

(8.44)

79.931

(5.60)

74.947

(10.56)

Pre minus Post

13.793

(12.52)

12.427

(14.49)

19.381

(10.68)

17.251

(15.17)

12.554

(14.32)

10.346

(14.25)

9.107

(10.61)

9.074

(13.07)

Fall

0.310
(0.47)

0.355

(0.48)

0

(0)

0

(0)

1

(0)

1

(0)

0

(0)

0

(0)

Sex

0.521

(0.50)

0.613

(0.49)

0.480

(0.51)

0.682

(0.48)

0.454

(0.51)

0.500

(0.51)

0.625

(0.49)

0.667

(0.49)

Age

21.676

(3.43)

21.306

(3.19)

21.960

(4.13)

21.091

(1.06)

21.182

(1.33)

21.318

(4.80)

21.833

(4.00)

21.556

(2.53)

CumGPA

3.169

(0.44)

2.937

(0.59)

3.110

(0.48)

2.788

(0.54)

3.070

(0.43)

2.859

(0.60)

3.323

(0.39)

3.216

(0.57)

Grade

3.106

(0.65)

2.952

(0.72)

2.780

(0.60)

2.886

(0.55)

3.159

(0.56)

2.795

(0.83)

3.400

(0.64)

3.222

(0.73)

Non-College of Business

0.254

(0.44)

0.290

(0.46)

0.160

(0.37)

0.318

(0.48)

0.273

(0.46)

0.318

(0.48)

0.333

(0.48)

0.222

(0.43)

Major

0.183

(0.39)

0.161

(0.37)

0.080

(0.28)

0.091

(0.29)

0.227

(0.43)

0.136

(0.35)

0.250

(0.44)

0.278

(0.46)

ACTEng

23.587

(3.16)

22.868

(3.59)

23.150

(4.18)

22.789

(3.31)

24.500

(2.35)

21.722

(3.94)

23.174

(2.66)

24.250

(3.21)

ACTMth

25.698

(3.31)

24.698

(3.68)

25.000

(3.31)

24.368

(3.79)

25.350

(3.57)

24.778

(3.021)

26.609

(2.99)

25.000

(4.37)

ACTComp

24.556

(2.43)

23.830

(2.21)

23.650

(2.80)

23.632

(2.19)

24.750

(2.20)

23.222

(2.02)

25.174

(2.15)

24.750

(2.27)

Elective

0.028

(0.17)

0.097

(0.30)

0.00

(0.00)

0.00

(0.00)

0.091

(0.29)

0.182

(0.39)

0.00

(0.00)

0.111

(0.32)

Requirement

0.451

(0.50)

0.274

(0.45)

0.360

(0.49)

0.273

(0.46)

0.500

(0.51)

0.273

(0.46)

0.500

(0.51)

0.278

(0.46)

N

71

62

25

22

22

22

24

18

Source: author's calculations.

Table A3. Difference of Means Tests: Treatment Versus Control, standard deviations in parentheses under means and 2-tailed significance tests in parentheses under t-tests

Pretest Mean

Independent Samples t-test

Mean Difference

Posttest

Mean

Independent Samples t-test

Mean Difference

N

Trtmt

Control

Trtmt

Control

Entire Sample

64.9128

(12.11)

62.554

(12.57)

1.101

(0.273)

2.359

78.706

(6.08)

74.981

(9.17)

2.792*

(0.006)

3.725

133

 

References

Association for Supervision and Curriculum Development (ASCD). 2005. "The Definition of Problem-Based Learning." Website last visited, May 19, 2005. http://www.ascd.org/portal/site/ascd/menuitem.a4befa0de1b8d1fddeb3ffdb62108a0c/

Blumstein, Philip and Pepper Schwartz. 1983. American Couples: Money, Work, Sex. New York: William Morrow and Company, Inc.

Boud, D., and G. Feletti. 1991. The Challenge of Problem-Based Learning. London: Kogan Page.

Bransford, John D., Ann L. Brown, and Rodney R.  Cocking. 2000. How People Learn: Brain, Mind Experience, and School. National Academy Press.

Gardner, Howard. 1991. The Unschooled Mind: How Children Think and How Schools Should Teach. New York: Basic Books.

Greene, Benjamin. 1997. "Verbal Abilities, Gender, and the Introductory Economics Course: A New Look at an Old Assumption." Journal of Economic Education. Vol. 28(1): 13-30.

Hansen, W. Lee, Michael Salemi and John J. Siegfried. 2002. "Use It or Lose It: Teaching Literacy in the Economics Principles Course." American Economic Review, Vol. 92(2): 463-72.

Harbury, C. D. and R. Szreter. 1970. "The Value of Prior Experience of Economics for University Students." Journal of Economic Education. Vol. 2(1): 56-62.

Johnston, Carol G., Richard H. James, Jenny N. Lye, and Ian M. McDonald. 2000. "An Evaluation of Collaborative Problem Solving for Learning Economics." Journal of Economic Education. Winter: 13-29.

Leighow Meo, Susan. 2000. "'In Their Own Eyes': Using Journals with Primary Sources With College Students."

Milbury, Peter and Brett Silva. 1998. "Problem-Based Learning, Primary Sources, and Information Literacy." Multimedia Schools. Vol. 5(4): 40-45.

Poole, William. 2004. "Free Trade: Why Are Economists and Noneconomists So Far Apart?" Federal Reserve Bank of St. Louis Review. Sept/Oct: 1 – 6.

Ramsden, P. 1992. Learning To Teach in Higher Education. London. Routledge.

Salemi, Michael K. and George E. Tauchen. 1980. "Guessing and the Error Structure of Learning Models." The American Economic Review, Vol. 70(2): 41-46.

Schamel, Wynell B. 1998. Teaching with Documents: Using Primary Sources from the National Archives. Volume 2.

White, Halbert L. 1980. "A Heteroscedasticity-consistent covariance matrix estimator and direct test for heteroscedasticity." Econometrica, Vol. 48(4): 817-838.

Williams, Mary L., Charles Waldauer and Vijaya G. Duggal. 1992. "Gender Differences in Economic Knowledge: An Extension of the Analysis." Journal of Economic Education, Vol. 23(3): 219-231.



[1] Of course, this is not unusual for a college course. Nor is it an excuse. William Poole (2004) notes the divergence between student understanding and support of free trade in theory versus in practice. "Questions asking about free trade in principle reveal support for free trade, albeit not as strong as economists'. However, questions asking about free trade in practice reveal strong reservations" (2004: 2). It has been noted that one of the main goals in good pedagogy is to deconstruct students' "preconceived notions" rather than simply teach students the material of the course. See Howard Gardner (1991) and John D. Bransford, Ann L. Brown and Rodney Cocking (2000).

[2] The survey is available from the author upon request.

[3] Examples of homework assignments that relied on the survey results are available from the author upon request.

[4] The pre and posttests are available from the author upon request.

[5] It should be noted that neither the pretest nor the posttest results counted toward student grades. While it is possible that not attaching a grade to the test would provide students little motivation to do well on the posttest, it is also possible that attaching a grade to the test may discourage students in the pretest. In any case, both grading and not grading the exams may bias the resulting data. Furthermore, all sections were treated in the exact same way, so any bias associated with my personal grading style was consistent across control and treatment groups as well as across semesters.

[6] In some cases, the pretest itself is used as a proxy for student aptitude. Empirical evidence suggests that this variable may contain too much measurement error (Salemi and Tauchen, 1980) to be of any value (students have an incentive to guess, for example, when taking the pretest). As a result, and because other measures of aptitude are available, it is not included in this equation. Even when it is included in the equation, however, results show that the pretest is not a significant predictor of posttest results.

[7] Using similar logic with regard to the pretest score, I did not use a variable measuring the difference between the pretest and the posttest scores as a dependent variable in the OLS regression. A variable measuring the difference would contain the same measurement error since it is based on the pretest values. Furthermore, I am less interested in how much better students did on the pre and posttest than whether or not the treatment simply affected the posttest results.

[8] Table A3, providing the results from difference of means tests for the entry and exit test results for the three semesters combined, can be found in the Appendix.

Edit this text

CopyrightUW System

Lisa Giddings, Associate Professor of Economics, University of Wisconsin-La Crosse. Author may be contacted via e-mail at Giddings.lisa@uwlax.edu.