The Dartmouth psychology chair replies

The following is a statement from Dr. Jay Hull, chair of the psychology department at Dartmouth. (If you're just catching up, see posts one and two about this issue.)

I asked Dr. Hull to reply to some specific questions I had about the views of the psychology department at Dartmouth. I am not interested in whether Dartmouth should stop giving credit for AP scores or any other issue - I think that's entirely that school's decision to make. But since all of the articles I've read have specifically pointed to students who made a 5 on the AP Psychology exam not performing to the standards of the psychology department, I wanted to hear directly from someone there. Dr. Hull was gracious enough to send the response below in reply to my questions.

-----------------------------------
From Dr. Hull:

First, note that this was not a scientific study.  Nor did we intend it for public consumption.  Indeed, we did not even intend these data to inform decision making at the college regarding AP credit in general.  We simply collected these data to inform our own departmental decision making regarding whether to give Advanced Placement credit for Introductory Psychology. 

We took three years of introductory psychology tests taken by Dartmouth students.

Each year has three terms of exams (we offer Introductory Psychology in the fall, winter, and spring terms).  Each term has 4 exams (each exam over about one quarter of the material) plus a comprehensive final exam.  Each exam has 50 questions.  So, we ask 250 questions per course and 750 questions per year (although many of the latter are redundant – we don’t give the exams back to students and we do re-use questions). 

Exams are over both lecture and book material.  We combined the exams in a single year,  discarded lecture based questions, discarded redundant textbook questions, discarded (a very few) questions flagged as bad questions by the faculty, and then randomly selected questions from the resultant pool.  A score of 68% was required to pass.

This procedure was then repeated for three years.

Year 1:
    97 questions
    89 students with AP Psych scores of 5
    7 passed (8%)
    Mean score = 57%

Year 2:
    97 questions
    93 students with AP Psych scores of 5
    8 passed (9%)
    Mean score = 56%

Year 3:
    80 questions
    26 students
    5 passed (19%)
    Mean score = 65%

Overall:
208 students took it
20 passed (9.6%)

Keep in mind:
1.    These tests were based on tests given to Dartmouth students in a Dartmouth course. 
a.    We use a high level textbook (Gleitman; Gazzaniga & Heatherton) whereas high school AP courses typically use a general level textbook (Myers).
b.    We are a Department of Psychological and Brain Sciences and our textbook selection reflects this.  This is appropriate for our majors, but probably not true of AP courses.
c.    The difficulty of the questions used in our test reflects the difficulty of the questions chosen by our faculty for testing knowledge of students in our courses – i.e., we did not randomly select from a test bank pool provided by the publisher, but rather from a pool of items used by our professors in our courses.
d.    It is out impression that students often take AP Psych courses earlier than other AP courses (Calculus) and may have forgotten the material after 2-3 years.

Finally, note that students who got an AP Psych score of 5, who fail our exam, and who subsequently take our introductory course do not do better than those without an AP course.  Although  a null effect, this is at least consistent with the notion that our exam is not unduly strict.  In other words, it does not seem to screen out students who have mastered the material in a way that makes our introductory psychology course easier for them.
-----------------------------------

This is Steve again. If you have thoughts about this reply, I'd encourage you to post them in the comments below.


--posted by Steve