d

 

 

How to Avoid Getting Statistically Snookered

 

 

The Journal of Educational Research presents the following interview conducted with Dr. Gerald Bracey

 

 

Interview


JER: In your book, On the Death of Childhood and the
Destruction of Public Schools, you call the 20-year old education
report, “A Nation At Risk,” a “veritable golden
treasury of slanted, spun, and distorted statistics.” You say
that education special interest groups “need another treatise
to rally round”—that being the No Child Left
Behind Act, which you call a “weapon of mass destruction”
for our public schools. NCLB is now marking its 5-
year anniversary. Please explain your opposing position
on NCLB.


Bracey: I have seen nothing to suggest that NCLB has
improved schools. [Margaret] Spellings [U.S. Secretary of
Education] touts gains on NAEP [National Assessment of
Educational Progress]. She implies that the 5-year period
she speaks of is 2001–2006, but it was really 1999–2004 and
most of the gain could have occurred while [President]
Clinton was in office. Given the incredibly rocky start of
NCLB, it would only have had the period from Sept 03 to
Feb 04 to work its miracles.
We are now hearing “reports” but no firm data that
Reading First [federal education program mandated under
NCLB requiring schools to use scientifically based reading
instruction] is improving reading. Given what gains have
been reported and given that teachers are spending more
time teaching reading, it could readily be the time, not the
program that is important. In addition the principal indicator
is fluency. Kids can be fluent as hell and not understand
a word they’ve just read.


JER: Why are researchers and laypersons alike snookered
by statistics? (That is, why do they only see what they
want to see?)


Bracey: I don’t think researchers are snookered by statistics
so much as they are by the overwhelming negativity
surrounding public schools. People lie in wait for chances
to prove the schools are terrible. When I returned from
abroad and landed a job as Director of Research, Evaluation
and Testing for the Virginia Department of Education,
I learned this: In 1974–75 the testing service at the Department
had tried one year to get test results back to schools
earlier than usual.
To do this, they had given the tests in September rather
than the usual mid-October mid-November period. The
norms for the tests, though, presumed that the tests had
been given in October–November. The combination of giving
the tests before kids were really settled in and the mismatch
with the norms caused the scores to plummet. The
next year, the tests were given when they should have been
and the scores jumped back to where they had been. But it
was too late. The legislature had already enacted a Basic
Skills law and it was my job to develop tests for the Basic
Skills that were constructed by our curriculum people.


JER: Last month, the Fairfax County (Virginia) School
Board defied the U.S. Department of Education and challenged
NCLB by deciding not to make thousands of immigrant
students take a federally mandated test that they
believe is unfair. The state of Virginia swiftly responded by
threatening to punish the county school system if they do
not comply with the NCLB reading tests for English-as-asecond-
language students. What is your opinion of the
county’s action?


Bracey: I’m with Fairfax. It is beyond idiocy to test kids
on material they can’t cope with linguistically. You can be
a competent student and still not cope with the test for several
reasons. First, oral language as often seen in the classroom
is much more forgiving than written language. Second,
on a report you can take your time and seek help.
Third, while your teacher is not trying to make a fool of
you, the test is. Given a five-choice multiple-choice test,
the test must fool a lot of people into picking the four
wrong answers. Otherwise, the items don’t “behave” properly.
Separating out the right answer from the “distractors”
is a very subtle task.


JER: How should Advanced Placement (AP) and International
Baccalaureate courses be administered in high
schools? Should teachers and principals encourage or pressure
students to take these courses to keep up with the best
students in other countries?


Bracey: Some schools treat AP as something to encourage
kids to take in order to see what a tough course actually
looks like and to claim a high percentage of enrollment.
Others discourage kids from the course or discourage them
Others discourage kids from the course or discourage them
from the tests because they want a high test score average. I
tend to be with the former-you can learn alot by doing so-so on a challenging test.



JER: Time magazine, in its December 18, 2006, issue on
“How To Build a Student For the 21st Century,” begins
with the “little dark joke” that Rip Van Winkle awakens
after a 100-year snooze and is totally baffled by all he sees
until he walks into a schoolroom. He exclaims, “We used to
have these back in 1906. Only now the blackboards are
green.” Time reported that educators are aiming too low—
reading and math competency is “the meager minimum,”
and scientific and technical skills are “insufficient.”
Also, in On the Death of Childhood and the Destruction of
Public Schools, you mention that mathematics, and maybe
science too, are not well presented in public school curricula,
nor are they well taught. What is your opinion on this
issue?


Bracey: Reports like the ones in Time are a farce. The
Rip Van Winkle tale is at least 25 years old. The classrooms
I saw in a school district I worked in were quite different
from those I experience in that the kids were much more
active in their learning, much freer to move about the
room as long as they didn’t disturb people, and had much
more time working on their own or in small groups.
My comment on math and science pertains to elementary
schools. I don’t think our elementary teachers for the
most part get a good grounding in those subjects. By the
way, elementary teachers were the only group who had
SATs and GPAs lower than other majors in a big gov’t
study [“Out of the Lecture Hall and Into the Classroom”
from the National Center for Education Statistics].
If other countries took NAEP tests, Singapore would be
the only nation where a majority of students are proficient
in science. A handful would be proficient in math. No
nation would have more than one third of its students proficient
in reading. These conclusions come from a procedure
devised by Bob Linn [co-director of the National Center
for Research on Evaluation, Standards, and Student
Testing at UCLA] and used recently by Richard Rothstein
[research associate at the Economic Policy Institute]. I have
been screaming for years that the NAEP levels—basic, proficient,
and advanced—are far too difficult. American
fourth graders finished 3rd among 26 nations in TIMSS
[Third International Mathematics and Science Study] but
NAEP said only 29% were proficient or better. That’s nuts.


JER: Do you envision a time when public schools in
impoverished areas will receive funds and staff to provide as
good an education as public schools in middle- and upperincome
areas? What could bring this event to fruition?


Bracey: I don’t think we’ll see much improvement in
poor areas until we can provide for the needs of these kids
outside of school. The kid who died recently from an
infected tooth is just an extreme example. Many poor children
have medical conditions in their teeth, ears and eyes
that impair schooling. Many poor mothers-to-be get inadequate
prenatal care and their kids suffer cognitively because
of that. Too many poor kids have lead in their bodies.
Nerves don’t function well under that condition. Etc.
Professional development of teachers would help, too,
but the above conditions need attending to as well. I would
like to think we could find incentives to lure better teachers
into poor areas, but it’s hard to imagine even though
TFA [Teach for America] claims that about 1/4 of their
recruits continue after their 2-year contract is up.


JER: National certification for teachers, which can take
up to 400 hours to complete, is said to push teachers to
adapt lessons for each student, to analyze their work, and to
reach out to families of colleagues and students. Conflicting
research exists on whether students score better on
standardized achievement tests with a board-certified
teacher (presently over 55,000 nationwide). Should teachers
be nationally certified?


Bracey: I don’t know that there needs to be a national
certification, but there needs to be better reciprocity that
would allow teachers to move from state to state without
having to jump through an unreasonable number of hoops
to teach in her new state of residence.


JER: Schools of Education and teacher-education programs
have been portrayed as “intellectual wastelands,”
called “impractical and irrelevant,” and cited as the root
cause of bad teaching and inadequate learning (Labaree,
2004; Cochran-Smith & Zeichner, 2005). The views of
opponents and proponents of traditional teacher-preparation
programs and state-certification requirements appear
to be at opposite ends of the spectrum. Labaree concluded
that “Balance, it seems, is unwelcome on both sides of this
debate” (p. 171). How do you feel about teacher-training
programs? How can a balance be achieved?


Bracey: I think there’s something to be said for the
Labaree position and NCATE [National Council for
Accreditation of Teacher Education]. Right now I don’t see
agreement on what greater professionalization would mean.
Some people worship NCATE; some hate it. When I was at
Mason [George Mason University, Virginia], it was seen
largely as something to get through.
But I also think different kinds of schools of education
get lumped together unfairly. Mason, for instance, doesn’t
have an undergraduate teacher prep program; many of the
well-known schools of education don’t. They’re more oriented
to research and evaluation. On the other hand,
places like Clarion or Edinboro [both universities in Pennsylvania]
work hard with undergraduates to produce good
teachers. But they’re not well known outside of the schools
that hire their grads. They are not the universities you see
written up in Ed. Week [Education Week], nor do they have
the professors you see quoted a lot.


JER: How do you see your book on educational statistics
being used, for instance, in a teacher-education program?
324-328 Bracey M_J 07 5/8/07 11:01 AM Page 325
326 The Journal of Educational Research


Bracey: As an awareness program. You can’t be cognizant
of every slanted statistic, but you can be aware that
statistics are gathered by people with agendas. They might
be benign, but someone has decided that the statistics in
question are important for some purpose—to make people
stop smoking, to make women seek care for bulimia, etc.
People who have some programmatic goal are likely to collect
the stats they think will advance them towards that
goal.
It will also help teachers—and others—to be in a position
not to be bullied by someone pushing an agenda with stats.


JER: What are your thoughts on extending the public
school day in the United States to either (1) keep up with
students in countries such as China, whose students were
reported to spend 30% more time each day in school than
U.S. students or (2) simply improve student performance?


Bracey: Chinese kids spend more time in school, but
most get 2 hours off to go home for lunch. Only 40% of
Chinese kids get past 9th grade. Jim Fallows of the Atlantic
Monthly is now stationed in Shanghai and in a recent email
dropped a line that the schools there are awful. He didn’t
elaborate. Naturally, the value of the extended day depends
on what you do with it. If it were used, as it is in some
places, to insure art, music and other areas currently
neglected because of NCLB, then it would be worthwhile.


JER: Do you believe that year-round schooling in this
country would improve student achievement and test scores?


Bracey: Year-round schooling OUGHT to improve
achievement, but so far the data have not shown any great
impact at all. I don’t know why.


JER: You have spent a large portion of your life explaining
and highlighting the pitfalls of statistics and research.
Why do you feel that people are suspect of you and not the
research or statistical presentations? In essence, why don’t
people listen to you?


Bracey: I think people are suspicious of me in large part
because my history is atypical, because I have no institutional
affiliation, and because my message runs so strong
against the “believe the worst” syndrome. It is also true that
research universities are heavily invested in public school
failure and problems—they use that as leverage to pry
money out of state and federal government. To admit
things are better would be to turn off the money spigot.
And, of course, they don’t like it when I say this.


REFERENCES:


Cochran-Smith, M., & Zeichner, K. (2005). Studying teacher education:
The report of the AERA panel on research and teacher education. Mahweh,
NJ: Erlbaum.
Labaree, D. F. (2004). The trouble with ed schools. New Haven, CT: Yale
University Press.

 

kg f