Friday, November 16, 2007

State rankings by NAEP improvement, whites only

++Addition++Steve Sailer weighs in. Commenters make speculations and suggestions (there are those who are both intrigued and who are incredulous) that add lots of extra value beyond what I have here. If interested in the data, it's definitely worth jumping over there.


As a follow-up to the post looking at scholastic improvement by state as measured by the NAEP, the same follows, for whites exclusively. Improvement is measured by looking at how the 2003 4th grade cohort compares to the 2007 8th grade cohort in each individual state. The better the latter performs relative to the former, the greater the improvement:

RankStateImprovement (SD)
2.North Dakota.84
9.South Dakota.50
23.New Mexico.13


33.New Jersey(.06)
36.Rhode Island(.17)
41.South Carolina(.40)
43.New Hampshire(.56)
47.New York(.89)
48.West Virginia(1.10)
49.North Carolina(1.18)

Unfortunately, the NAEP site does not give an average 2007 score for whites in the District of Columbia, for lack of sample size. Algebraically figuring the white score based on its proportional representation puts DC at (.21), between Rhode Island and Virginia.

The actual number could vary widely from that estimate. If, instead of 3% (the NAEP only gives these percentages in whole numbers), only 2.5% of 8th graders in DC's public school system are white, it is easily propelled into the top spot. Whatever the case, DC's whites represent a tiny contingent--there are less than 3,500 whites in the entire district, K-12.

While the white variance by state seems, in sum, wider than is the case for states when all races are considered, that's a consequence of how the data are presented. The standard deviations in each case are derived from the totality of scores for all races, and for whites only, respectively. Because race figures significantly in determining absolute scores, including all races creates more overall variability, and hence in terms of actual scores, 'larger' standard deviations (by about 50%) than is the case when only whites are considered.

Steve and other commenters raise questions about the influence of a brain drain phenomenon (DC attracting upper-echelon immigrants and pushing out impoverished blacks, Hispanics moving to the South and especially North Carolina, the upwardly mobile leaving the backcountry culture of West Virginia, and I also wonder what the consequences are for states that have taken in a substantial number of New Orleans' refugees, etc).

Around three percent of the population moves its primary residence into a new state each year. But over a four year span, the cumulative effect might be substantial. To the extent that this explains the improvement (or deterioration), it takes away from the measure's utility as a proxy for teaching effectiveness by state.

An influx or outflow will also presumably have some effect on the students who have stayed put. However, Nevada's whites have done well in the face of one of the greatest foreign migration rates in the country. On the other hand, California's whites, who experience an even higher rate of foreign migration into their state as their neighbors to the east do, haven't fared as happily.

Although North Carolina shows a lot of deterioration when both all races or just whites are looked at, it's conceivable that in the Old North state along with the rest of the country, migration patterns are changing the classroom atmosphere for the students who remain, for better or worse.

Of course, immigrants from outside the US are not the only migration force at the state level, as most of the people who plant themselves somewhere new have come from another American state.

Also, the proportion of kids in private schools varies by state. Children who split their pre-collegiate schooling between the private and public spheres tend to start in the former and end in the latter (adjusted for total population in primary and secondary schools across the US, kids in their primary years are about 30% more likely to be enrolled in private school than those in their secondary years are). Since, on average, kids in private schools tend to be a notch above those in public schools, a higher rate of children in private schools should be beneficial in terms of improvement.

Louisiana has the highest percentage of children enrolled in private schools in the country, and it does poorly. But New Hampshire, with the second smallest percentage of private school kids doesn't do well, either.

I do not mean to equivocate on the demographic angles, so as to make the method appear a viable way of approximating the general effectiveness of public pedagogical strategies. I will look at all of these factors (and any other potentially viable suggestions) in light of improvement by state.


Jason Malloy said...

Interesting stuff as always AE.

It's conceivable that states that see more improvement between the 4th and 8th grade level are actually the ones with worse school systems.

The heritability of academic achievement rises rapidly between pre and post-adolescence, so a student (A) hobbled by a poor primary school system would make more gains than a genetically identical child (B) in a good school system.

Time I
Bad Grade 4 (A)
IQ 92

Good Grade 4 (B)
IQ 98

Time II
Bad Grade 8 (A)
IQ 100

Good Grade 8 (B)
IQ 100

Bad (A) gain = 8 points
Good (B) gain = 2 points

I suppose this could be tested by controlling for initial ability.

Audacious Epigone said...



Do you have estimates on heritability from the ages of nine to fourteen?

If that is the case, primary schooling, and by extension the effectiveness of teaching, really is of little value.

Any ideas on how to get at initial ability on a national scale?

What would be most helpful would be some fairly clear-cut, quantitative proxy of teaching effectiveness or a way to compare the rigor of certification requirements by state. Something along those lines would probably provide a pretty good idea of which way the trend runs.

Steve Sailer said...

Wow, these are big differences: a couple of standard deviations from top to bottom.

It's interesting to see that among the two biggest states, Texas comes out a standard deviation ahead of California, just like the stereotype says. Maybe George W. Bush really did fix education in Texas!

Audacious Epigone said...


I was surprised by the variation, too. Although the SDs are narrower than is the case when all races are considered, in terms of absolute scores on a 500-scale test, the difference in improvement is still more than eleven points between Montana and Connecticut. That is substantial.

Jason Malloy said...



See this paper (PDF, pp 10-14). It's not exactly the same in every study, but based on the chart on p 14 let's go with .45-.75 for ages 9-14.

Jeff Williams said...

Hi AE...

Congratulations on your good work on this table. Explaining the differences among states is now the task at hand.

I think you are off the track if you are looking for some kind of difference in the white student populations. I believe your table actually shows the relative effectiveness of each state's corps of teachers, which is exactly what you were trying to show. I believe you can assume that the student populations are nearly the same.

What can explain the differences, then, in the effectiveness of teachers? One determinant could be the toxic effect of union power. Under a certain kind of powerful-union regime, teachers no longer have any incentive to care about the quality of work they do. They can't be fired or disciplined.

To examine this relationship, one must develop a rating system for teacher union power. One indicator could be average teacher pay, and a table of that can be found at this URL:

Go to page 32 in the Adobe reader.

Connecticut is always known for having the highest-paid teachers. Michigan is known for having a powerful teachers union. Adjusting for relative income levels and cost of living, Michigan's teachers' pay is higher than Connecticut's.

What will correlate best with your data, though, in my guess, is relative numbers of teachers fired. I believe that states where very few teachers are ever fired will rank near the bottom of your chart.

That data on firings, which I don't know where to locate, would be an indicator, in my mind, of a "toxic" union presence that would undermine teacher performance.

(I have also sent email to Steve Sailer expressing these ideas.)

You are doing important work here. Keep up the good work. It may be a source of enlightenment to some beclouded minds.

Anonymous said...

The method here is really bad.

First, it is inappropriate to use NAEP scores to rank order the states. See the reason/proof on

Second, ETS (the private contractor that analyzes and reports NAEP results) recommends against making 2003 4th grade to 2007 8th grade comparisons. The characteristics of the NAEP scale don't support such comparisons.

Audacious Epigone said...


Thanks much for that!

It seems the author misstates the effect of childrearing (.26 correlation between adoptive siblings explaining 26% of the variance...)

Incidentally, I didn't think human brain's consumed up to 25% of resting metabolism. I thought it was more in the area of 15%-20%.


Thanks, but it was Steve's idea. I just did what he suggested. Architects deserve the credit. Builders are a dime a dozen (heh, that sounds so demeaning to builders, especially coming from someone as mechanically declined as myself).

Good suggestions. I looked at pay for all races, and found nothing. But I'll try with whites only. I agree that some measure of teaching accountability/merit (or lack of union power, which is probably pretty much the same thing) would be optimal. I will keep looking for it.


Of course the average scores are estimates, using representative samples of a state's population. What's your point? Exit polls rely on estimates. So does marketing research. And on and on.

I looked at fourth and eighth grades over a period that spanned eight years. The gaps correlated quite strongly for that entire time frame. Secondly, I converted score variances into standard deviations.

Steve Sailer said...

Perhaps CN is down at the bottom of public school value added because so many of the better public school 4th graders have enrolled at Olde Grottlewick Prep School by 8th grade?