Dr. Jay Hull of Dartmouth was kind enough to provide answers for several of the questions I posed to him about the performance of students who scored a 5 on the AP Psychology exam. I will post his answers later the week.
First, I contacted Trevor Packer, who you may know as the College Board's vice president for the AP Progream, and asked him for his thoughts. He gave me permission to post his comments in full below. I began by asking him if he could share the data that the College Board had about AP and Dartmouth:
---------------------------------------
Absolutely. Here is the link: http://bit.ly/WnOQBn
Table 1 shows the number of Dartmouth students as well as the students from the other participating institutions (Cornell, Stanford, UCLA, etc.).
Table 2 shows the outcomes for the AP Psych students in the sequent college Psych course in comparison to the non-AP students, with and without controlling for SAT scores.
Appendix A provides the titles of Dartmouth's intro and sequent psychology courses that yielded the data about AP and non-AP student performance.
There are many possible explanations for the differences between these findings and the "experiment" on AP Psych students that is being anecdotally reported. None of these possible explanations support using Dartmouth's AP Psychology experiment as evidence for the highly problematic and unsupported claim that AP Psychology course curricula and assessments are not college level:
1. Is Dartmouth's Intro Psych course so different from what is valued by other highly selective institutions that by aggregating Dartmouth students with other institutions' students, this study failed to detect differences between Dartmouth and other highly selective institutions in outcomes for AP Psych students?
2. Has Dartmouth changed its Intro Psych course so significantly since the mid-90s when this study was conducted that the results from this study would not be applicable now?
3. Did the design of Dartmouth's "experiment" fail to control for time and order effects? For Dartmouth's experiment to yield any valid claims about AP vs Dartmouth's intro course, the same "condensed Dartmouth exam" would need to be administered to Dartmouth students following the same time elapse between when the AP students completed the AP course and then took the Dartmouth exam: 4 months, 16 months, and 28 months. Only by controlling for these effects would it be possible to claim that AP students had not achieved the same level of learning as students taking Dartmouth's Intro Psych course.
4. Did the sampling of questions in the "condensed Dartmouth exam" focus too narrowly on a recent unit, or did it cover the full range of knowledge and skills taught in the course?
5. It is not valid to infer from this AP Psychology experiment that a unilateral elimination of AP opportunities for students in the 30 other AP subject areas is warranted by data and evidence.
So if Dartmouth wants to achieve a goal of keeping students on campus for a minimum of 4 years, it is ostensibly their right to eliminate AP credit policies. But we object firmly to any attempt by individuals there to deflect the ire of students and parents over this proposed policy by blaming AP Psychology teachers for not being able to teach a college-level course. I feel deep obligation to remind Dartmouth and the public that 5,800 college professors -- including Dartmouth's own faculty who have worked in particular on AP Computer Science, AP Environmental Science, and AP Physics -- have staked their professional reputation, skill, and credibility on the claim that they have designed and scored AP Exams to the same standards as their own institutions meet.
Many of these faculty are speaking out against the claims a Dartmouth professor has made that the work so many professors have done together to define college-level standards, approve college-level syllabi, and assign scores representing college-level performance is not actually college level. They've made the following statement: http://bit.ly/VS29wV
The design process for AP Psychology as well as the audit ensure that the curriculum design is college level. But this does not guarantee quality of instruction. So the exam design and scoring standards, set by college professors, ensure that the learning outcomes are attained before credit and placement are granted. This is what the Morgan/Klaric study shows.
More recent studies continue to affirm the validity of AP scores for credit and placement:
http://bit.ly/WbKMU1
http://bit.ly/13MGkl1
http://bit.ly/W52lbG
Trevor Packer
---------------------------------------
-- posted by Steve
Posts
3 comments:
I think Trevor asks the most important question when he says: "Is Dartmouth's Intro Psych course so different from what is valued by other highly selective institutions?" Dartmouth may have some really interesting, unique goals/skills for their intro psych class, but THAT'S the information that needs to be discussed in order to really understand the meaning of their study. This is actually a good example of how operational definitions are crucial to an accurate interpretation of a study.
Can Dr. Hull provide any more details about the test they used?
Thanks for your attention to this interesting bit of controversy from Dartmouth. I, too, would be very interested in the content of the test used by Dartmouth.
Really, Trevor, really? These citations are propaganda, not peer-reviewed studies.
I have resisted being forced into teaching the AP Psychology curriculum for years, and Dartmouth's decision feels like vindication. I'm more interested in whetting my students' appetite so they're excited to take a real college course than force-feeding them factoids that they'll forget before they arrive at the ivy-covered halls.
Post a Comment