One-percent of a Standard Deviation

... is quite a bit smaller than a one-percent increase. In fact, 1% of a standard deviation is pretty damned small. It seems somebody needs a Statistics course:

Public schools located near private schools increased reading and math scores more than public schools that had little competition.

Huge, I tells ya.

For every 1.1 miles closer to the nearest private school, public school math and reading performance increases by 1.5 percent of a standard deviation in the first year following the announcement of the scholarship program. Likewise, having 12 additional private schools nearby boosts public school test scores by almost 3 percent of a standard deviation. The presence of two additional types of private schools nearby raises test scores by about 2 percent of a standard deviation. Finally, an increase of one standard deviation in the concentration of private schools nearby is associated with an increase of about 1 percent of a standard deviation in test scores.

Test scores rose more for elementary and middle schools than for high schools, perhaps because the scholarship made K-8 private schools affordable but didn’t cover as much of the tuition at private high schools.

Hummmm ...

Did the scores of the private schools drop at the same time as the public school rose?  If the public school scores rose, was it because the parents of weaker kids took the money and ran? Was it because Florida is investing heavily in on-line learning and told certain kids that their behavior was unacceptable IN school so they had to switch to the private school or take courses online? We'll never know but this is an equally valid interpretation of the facts as presented.

I find the "1.5 percent of SD per mile" statistic interesting but pretty meaningless. That's not a standard deviation, it's one-one hundredth of a standard deviation. That's the equivalent of SAT scores rising 1 point. Read the collegeboard's take on significance.

Just because you can see something in your educational microscope doesn't mean there's anything worth looking at.
The quote also says that this happens only in the first year following the announcement. So the average SAT scores went up about 1 pt.
Once.

Here are Florida's average SAT scores for the last couple years. Notice the yearly fluctuations larger than that touted by the article. Note, the standard deviation for SAT scores is typically 100 - 110 points. So 1.5% of a standard deviation would be 1.5 points.

The timeline is also interesting. The idea that the mere announcement of a private school makes a difference in the first year (but only in the first year) indicates that it's got nothing to do with the education provided since it takes some time for a kid to get an education. Statistically insignificant.

I'd be looking for information on who paid for this study and who has the most to gain by falsely trumpeting miniscule gains and falsely attributing them to the glorious private schools.