Selection Bias at KIPP again

It's getting rather repetitive. Lots of people have said it, Jim Horn, for one. It seems as though the journalists aren't listening, or at least are swallowing the bait hook, line and sinker when it comes to the press releases from that grandfather of charter schools, KIPP.

Education News put out this blurb committing that journalistic sin: copy without critical thinking.

As of Fall 2010, 33 percent of students who completed 8th grade at a KIPP middle school ten or more years ago have graduated from a four-year-college. This rate is above the national average (30.6%) and four times the rate for students from low-income families.

It's never been about the money and it certainly isn't about the KIPP program. Socio-economic status is an indicator of educational success not because money is the key factor but because selection is.

What kids are likely to apply to KIPP, suffer through the selection process, stay in KIPP throughout the 8th grade, have the ability to maintain focus under pressure, put up with the incredible stresses and challenges demanded of them, and attend the mandatory summer and Saturday classes? These same traits are a damn good indicator of college success, too.

KIPP isn't the key to their success - it's merely the spotlight that shines on them, taking all the credit for their success while ruthlessly weeding out any child who isn't up to the standard, in much the same way that the average height would increase if you shot all the short people.

Education through academic eugenics.

Testing, Scoring and Trusting the Data

A fascinating case study is the NY Regents, scoring, and the unintended (or purposeful) consequences of a line in the instructions to the scorers.

First, here's the graph of the number of kids getting each score (from WSJ).  The issue can be seen quite clearly. The graph overall is a typical left-skewed distribution, as you'd expect from this type of test. The trouble comes when you explore that jump in the middle at the passing mark.

Of course, the nattering class is all up in arms over this, claiming fraud and misconduct.  You can almost hear jeers of "Union Bastards trying to save their jobs by lying on the tests."  Unfortunately for those people, the reason comes down to one sentence:

"[State officials] note that the state actually requires teachers to regrade certain Regents tests where the student barely fails in order to check for grading errors."

When you are singling out tests for special consideration, and the stated focus is to look for "scoring errors" on "barely failing tests", the only way for the scores to change is up.  Since it's easiest to give one or two points to a 64 or 63, it's logical that those would be the most effected.


Looking at the graphs, you can see that some teachers (probably a school at a time) set their cut off at 50 while the majority set the cutoff at 55.

Why the negative slope in that region? When looking for ambiguous answers which could be scored higher, you have to "rescore" each problem until you get an appropriate number of points. It's easier to find one such than 10 such.

Similarly, since there was no reason to check more answers than just enough to get the kid to pass, the uptick was only to 65, though it seems that many teachers weren't keeping close track of the extra points and brought the kid up to a 66 or 67.

Conversely, if the teachers had been instructed to rescore all students within ten points of the cutoff instead of just those who were below it, then you would have seen some students adjusted downward, balancing out much of the upward movement.  If there were a similar uptick at the 65 mark in this hypothetical, then and only then can you claim that teachers are deliberately mis-scoring to jigger their VA measures. (and even then, I'd put the reason as teachers wanting to help students rather than being so coldly self-interested.)

The place where I found the link to this article had this comment: "Teachers don't want to flunk kids that just barely miss the passing score. Until all responsibility for creating and scoring state exams is given to an independent body with no interest in the results of the tests, the results reported should be viewed skeptically."

Ummm, no. Scoring tests is really complicated.  Pearson, the biggest company, uses part-time, barely out of college, minimum wage people to do the scoring. Getting the "right" score is more a matter of whether or not the scorer speaks English and actually knows the material.

Frankly, given the mess that the testing industry is in when it comes to scoring, I have a feeling that the teachers are doing a more conscientious job. If you want a nasty introduction to the follies of testing company scoring sessions, check out Todd Farley's "Making the Grades." It's well-written but damn depressing if counting on accurate test scores because you're stuck in the hell of value-added and merit pay.

It's a Rosy Picture, Rotten at the Core.

I shouldn't need to add much to this:

New York City’s top-ranked school is under investigation for cooking the books, reports the New York Times. Theater Arts Production Company School, a middle and high school located in a low-income Bronx neighborhood,  graduated 94 percent of seniors, more than 30 points above the citywide average. The school earned a near-perfect score in “student progress,” based partly on course credits earned by students.  The school’s no-failure policy requires teachers to pass all students who attend class, regardless of their performance; no more than 5 percent of students can get D’s.

In practice, some teachers said, even students who missed most of the school days earned credits. They also said students were promoted with over 100 absences a year; the principal, rather than a teacher, granted class credits needed for graduation; and credit was awarded for classes the school does not even offer.

When you tie teacher and admin pay to "performance", tie school survival to "attendance" and "graduation rates," it's not a stretch to imagine why people would lie. Should we blame Klein, Black, Bloomberg, Duncan, Obama, GWBII?  Some combination of all of them? Or just go firing school officials?

Ray Allen wins again.

Funny guys, those Boston Celtics. Winning on an awesome Ray Allen shot and immense Shaquille O'Neal shoulders.

And yes, that tweet makes total sense.

BTW,
Reggie Miller: 2560 three-pointers.
Ray Allen: 2533 three-pointers. 13-21 shooting threes for the last five games. When's the record gonna be?
Data below the jump.
Celtics are 32-9 in this 82 game season so far. When, if he keeps up this pace, should he be buying lots of tickets for friends and family to help him celebrate the record? If you're going to go, you'll have to ask for time off but you'll need to ask well in advance or they'll just refuse you. Place your bets ... which game will it be?

Fri, Jan 21 vs Utah
Sat, Jan 22 @ Washington
Tue, Jan 25 vs Cleveland
Thu, Jan 27 @ Portland
Fri, Jan 28 @ Phoenix
Sun, Jan 30 @ Los Angeles
Tue, Feb 1 @ Sacramento
Fri, Feb 4 vs Dallas
Sun, Feb 6 vs Orlando
Mon, Feb 7 @ Charlotte
Thu, Feb 10 vs Los Angeles Lakers
Sun, Feb 13 vs Miami
Wed, Feb 16 vs New Jersey
Tue, Feb 22 @ Golden State
Thu, Feb 24 @ Denver
Sat, Feb 26 @ Los Angeles Clippers
Mon, Feb 28 @ Utah
Wed, Mar 2 vs Phoenix
Fri, Mar 4 vs Golden State
Sun, Mar 6 @ Milwaukee
Wed, Mar 9 vs Los Angeles
Fri, Mar 11 @ Philadelphia
Sun, Mar 13 vs Milwaukee
Mon, Mar 14 @ New Jersey
Wed, Mar 16 vs Indiana
Fri, Mar 18 @ Houston
Sat, Mar 19 @ New Orleans

PISA scores - another look.

What do you see? When you look solely at schools with fewer than 10% of students on FRL (i.e., poor), US schools would be better than those of the top of the chart, Korea. When you include the whole spectrum of US schools and their students, the US is much lower.

The US has a much bigger spread than any other country (the largest standard deviation of wealth in the developed world).

Our overall scores are unspectacular because we have a high percentage of children living in poverty, over 20%. This is the highest among all industrialized countries. In contrast, child poverty in high-scoring Finland is less than 4%. For Cleveland and for the US as a whole, the major problem is poverty. Before we worry about teacher quality, institute longer school days, and increase testing, we need to make sure that all children are protected from the effects of poverty: This means adequate health care and nutrition, and access to books. When we do this, American test scores will be at the top of the world.-- Stephen Krashen

On jumping to conclusions and historical data.

In discussions that refer to data about historical topics, the non-math folk often fall for the correlation - causation error, as well as misinterpreting their evidence.

It is generally accepted that height actually declined somewhat in late period and early modern times (see, for example, preserved sites like Williamsburg or Jamestown, where door frames and hallways are significantly smaller than nowadays), the reason having to do with more widespread malnutrition and chronic health problems emerging in the 14th century and continuing on until the 18th century. (from a mailing list focusing on medieval history.)

Average is a really tough thing to deal with here and there are a bunch of problems with this statement.

Let's deal with averages first. It's pretty easy to find the average height of modern people. They're alive and measurable. You call together twenty folks at the mall and ask them their height. Convenience sample leads to selection bias, so you retry with a cluster sampling. That's better, but did you happen upon the community of immigrants who are shorter? SO you expand your search and try a more systematic approach. In the end, you have a sense of the average height.

Now try that for the 14th century. Your sources are whichever bones you've been able to find and measure and a few written claims. Not much for numbers there.

Second issue is that pre-19th century doors, hallways and buildings weren't built for standard-sized people. They were built for the occupants and those occupants had different needs. Stating that door frames and hallways are smaller than currently because of a lesser average height is a conclusion that doesn't necessarily follow.

Your average tells you about the group but not much about individuals, so your building codes are set for the 99.8% who are shorter than +3 standard deviations from the mean so that no one is inconvenienced. But what is the average height? That's easier to tell now, but not so much for earlier demographics.

Room size does not correlate to occupant height either. My classroom ceiling, for instance, is nearly twelve feet high. Can we really conclude anything from that? On the other hand, all of the occupants for the last twenty years have been shorter than 6'5" and half of them were averaging 5'4" (the girls). All classrooms built in the last twenty years have 8' or 9' ceilings, becuse of changes in the building code, not because people kept hitting their heads.

The windows are huge, nearly 3' by 8', but that speaks to the need for free light and air rather than a race of giants.  Need and purpose are better indicators of door size in pre-code days.


The typical pre-1850 house has low ceilings and doors I want to crouch for (but that are still taller than I am), for a couple of reasons.

A room with a low ceiling is easier to heat, less difficult to build, and uses less wood in the building process. A 7' ceiling is not a problem for anyone who was going to use it - the builders were making houses for themselves and they didn't feel the need to give themselves 2' of headroom.

Sure, it's shorter than today, but only relatively. We notice because the standards have changed and it FEELS lower, not because we need to duck.

The same is true for the door. It's shorter. It was custom-made and of varying heights. Often it wasn't "square" but rather built to fit the space rather than the other way around. It might have a corner cut off to fit under the eaves or clear a stair. In any case, the door was built for the prospective occupants, not the rare fraction who needed a 6'8" door. Andre the Giant could bloody well duck.

Likewise hallways. What's the need for a wide hallway? It's just wasted space. Your big furniture was on the main floor or was carried up in pieces and assembled in place. There were no ADA-type requirements. Again, we feel constricted, but the person of that time would consider our modern requirements to be ostentatious displays of wealth. The typical house of the 15th century was maybe half the square footage of 20th century houses, and that is being dwarfed by 21st century ones. This correlation is with wealth, not height.

Other factors chime in: the cost or taxes assessed on road (or canal in this case) frontage might dictate your building style. For an extreme example of this, check out houses in Amsterdam: narrow, almost useless staircases and a hoist mechanism outside for lifting pianos and furniture to the second or third floor.


The takeaway for math teachers is to stress that we need more than correlations to justify cause. I might state the original paragraph and have the kids try to punch holes in it. Critical thinking and all that. I'd be looking for points like those above and:

• If the claim is that people were taller before the Templars' suppression at the hands of Philip the Fair, became shorter during the 14-17th centuries and then started getting taller again in the 18th, then your doors should show the same trends for the same time periods, in the same types of houses and at the same socio-economic levels. If the doors to churches showed the same decrease and increases, that would be a good sign.
• How could the builder know the average height of 15th century mankind? Communications were difficult and slow, the size of the foot varied by region among other factors. Besides, even if he did know, why would he care? He's building for the man in front of him, not some mythical beast. Is there evidence that builders knew the average?
• Is the rural house different from that in the city?  See the Amsterdam note above.
• What are other reasons for the differences do you see? 

• War.  Make your attackers duck as they enter. Some Indian adobes have four foot doors for this reason. This picture is from Doune castle.
• Security. A smaller door is easier to bolt closed.
• Religion. Make your visitors bow in humility as they enter.
• Practical. They wanted to reach the ceiling beams for storage hooks.
• Sexism.  Only women used this door so who cared about it being more than 6'? 
• This was where the children slept.  There was plenty of room under the eaves.
• Mathematical considerations:
• Can the average height be measured?  Is that number meaningful here? Should the anthropologist use median or mode instead?  What was the builder using?
• What kind of sampling would work best for ascertaining that measure?
• Historical considerations:
• How do we know that people did get shorter? Did they or is this a myth?
• If they were, was it due to nutritional deficiencies? Were chronic health problems all that widespread in just those four centuries or merely the Black Death taking all the headlines? Was the 9th century healthier?  How about NYCity in the 1800s tenements?  London's Soho at the time of cholera epidemics?
• Were the doors shorter during the time period in question? What was he looking at when he measured?
• What would the students do as step two to confirm or deny these theories? Why haven't they started?

I love math.

What happened to "Education"?

Joanne Jacobs has a piece on evaluation programs that are looking to videotape teachers as they give lessons and then "Go Look at Tape." The issue seems to be that

More than 99 percent of teachers are rated satisfactory by their principals, reports a study on “the widget effect” by the New Teacher Project.

and this is somehow "bad," hence the need for overhaul. While I would argue that you need to improve the system so that middle-managers (Admin) can appropriately rate, measure and evaluate their assembly-line workers (teachers), I'm not sure that going to videotape is much of an improvement. If the Principal can't see the "errors" while sitting in the class, how is he supposed to make anything out of the 17th such review, even if he did have the time. Well, they have an answer for that ... Gates figures we'll pay outside evaluators who will somehow be able to see what the highly paid admin can't or won't.

The Gates Foundation is developing a new model, with the help of social scientists and teachers, reports the New York Times. Outside evaluators analyze videotapes to determine whether teachers are teaching well.

We've got money for outside evaluators? Pretty cool. I'll sign up. What gives you the idea that I'll be any more or biased/petty/superficial than the current system? The fact that I'll be able to rewind the tape? More likely I'll fast forward through it. The fact that I don't know the teacher so I'll be more honest? More likely, I'll make a snap judgment and move on.

You know we'll pay for training out of our RTTT money or something, hire bunches of consultants to teach teachers to evaluate other teachers who are trying to teach students. Pretty amazing, ain't it?

Hundreds of teachers will be trained to review 64,000 hours of classroom video. They will look “for possible correlations between certain teaching practices and high student achievement, measured by value-added scores.”

Let's say it's 800 teachers - that's 80 hours of tape to watch, per person. Asking a lot? I can't even spend that much time watching TV or movies in a month while sitting mindless and half-comatose. I'd need to be unemployed to look at 80 hours of tape multiple times and evaluate someone.

But here's the crux of my complaint ... those correlations. I've had students who completely and utterly bombed in class but did well on testing. I've also had plenty who did well in my class, well in the next couple of classes, had a great high school career, and then a great college career, and are now cranking out 6 figure salaries ... but didn't do well on testing. How do I know they did well? They told me.

Since when did value-added scores on NEAP mean an education? Since when could anyone figure out what I do that "works" by watching 10, 20 hours of videotape? Really? Do they know me? Since when did someone watching a few hours of video actually have a clue as to what went on and who and how the students were effected? Ten hours of video? That's two days. Does anyone here think that I couldn't fake it for two days and mess around the rest of the year if I was of a mind to?

I didn't think so.

The Further You Extrapolate, the Sillier You Look

I'm always bemused by graphs that try to extrapolate too far.  Case in point at right (found by Darren).  The numbers previous to today are obviously known well but the future is a little cloudier.  Nevertheless, this graph claims to see a trend going rather upward, even though the trend for the next few years is downward.

Like the "population bomb" scare stories of the recent past, this one is just silly.

Why silly? Because it assumes that "given no change in policy, the numbers will do this."

Given no change? This is America.  We can't stop changing. The one thing that will never happen is steady state.  Somebody will introduce a bill to change this or that and your whole projection becomes worthless ... as if it meant much to begin with. We adapt in this country.  When the deficits were spiraling out of control under Reagan/Bush I, the country adapted, we nearly balanced the budget and we were starting to work on the debt.  Clinton's policies were less of a driver than the tech boom but, c'est la guerre. I'm frankly surprised that the National Review published it because it shows the sharpest drop under Obama's second term.


I had to tweak Darren, since he's a staunch conservative:

"Come on. Extrapolating a possibly exponential curve out to 2082? You really shouldn't, as a math teacher, have let that go without some comment. 

But I'll play along ... let's read the graph. The percentage surges upward in 1980-1984 when Reagan was doing those enormous deficit budgets -- to drive the Russians to bankruptcy, yes, but it still increases.

Then the economy recovered and those percentages dropped under Clinton (not that he was totally responsible for that, but it does make a good way to tweak conservatives!).  Likewise, the percentages rises under GWB, hits a peak in 2010 under Obama, and then shows a tremendous downward slope under the remaining years of Obama's term (and into his next?). Then, in 2016 (when the Republican presumably gets elected), the percentage starts to climb again.

I'm not sure that's exactly what you had in mind."

Darren responded later ...

"It's when the health care costs really start to kick in. Of course extrapolations that far out are silly, which is one of the reasons why I love the global warmers so much. Still, I don't see anyone arguing that our interest payments are going to go *down* any time soon."

No. But why is that graph any less ridiculous? The spending by Government is totally under the control of the Legislature and can be adjusted yearly as the political winds blow. If they wanted to, the Afgan war could cease in days, the bailouts could stop, the contracts could be ended and penalties paid.

Global warming isn't quite up to a vote by the House Ways and Means Committee. Natural processes don't stop on a whim.

I don't particularly care about the gloom and doom part of the debate anyway, so I'm probably not a good person to argue this with. I don't care about global climate as such, but I do care about MY air. If NYCity goes under water, it'd be turned into a modern Venice pretty quickly and Mankind would adapt. Florida would build dikes like Holland and Mankind would adapt. Ideal croplands would be found further north and, well, you get the idea.

One-percent of a Standard Deviation

... is quite a bit smaller than a one-percent increase. In fact, 1% of a standard deviation is pretty damned small. It seems somebody needs a Statistics course:

Public schools located near private schools increased reading and math scores more than public schools that had little competition.

Huge, I tells ya.

For every 1.1 miles closer to the nearest private school, public school math and reading performance increases by 1.5 percent of a standard deviation in the first year following the announcement of the scholarship program. Likewise, having 12 additional private schools nearby boosts public school test scores by almost 3 percent of a standard deviation. The presence of two additional types of private schools nearby raises test scores by about 2 percent of a standard deviation. Finally, an increase of one standard deviation in the concentration of private schools nearby is associated with an increase of about 1 percent of a standard deviation in test scores.

Test scores rose more for elementary and middle schools than for high schools, perhaps because the scholarship made K-8 private schools affordable but didn’t cover as much of the tuition at private high schools.

Hummmm ...

Did the scores of the private schools drop at the same time as the public school rose?  If the public school scores rose, was it because the parents of weaker kids took the money and ran? Was it because Florida is investing heavily in on-line learning and told certain kids that their behavior was unacceptable IN school so they had to switch to the private school or take courses online? We'll never know but this is an equally valid interpretation of the facts as presented.

I find the "1.5 percent of SD per mile" statistic interesting but pretty meaningless. That's not a standard deviation, it's one-one hundredth of a standard deviation. That's the equivalent of SAT scores rising 1 point. Read the collegeboard's take on significance.

Just because you can see something in your educational microscope doesn't mean there's anything worth looking at.
The quote also says that this happens only in the first year following the announcement. So the average SAT scores went up about 1 pt.
Once.

Here are Florida's average SAT scores for the last couple years. Notice the yearly fluctuations larger than that touted by the article. Note, the standard deviation for SAT scores is typically 100 - 110 points. So 1.5% of a standard deviation would be 1.5 points.

The timeline is also interesting. The idea that the mere announcement of a private school makes a difference in the first year (but only in the first year) indicates that it's got nothing to do with the education provided since it takes some time for a kid to get an education. Statistically insignificant.

I'd be looking for information on who paid for this study and who has the most to gain by falsely trumpeting miniscule gains and falsely attributing them to the glorious private schools.

Polls may be hitting a wall.

Slashdot has a thought:

"The 'cellphone effect.' In 2003, just 3.2% of households were cell-only, while in the 2010 election one-quarter of American adults have ditched their landlines and rely exclusively on their mobile phones, and a lot of pollsters don't call mobile phones. Cellphone-only voters tend to be younger, more urban, and less white — all Democratic demographics — and a study by Pew Research suggests that the failure to include them might bias the polls by about 4 points against Democrats, even after demographic weighting is applied."

This will make the "science" of polling even more suspect.  It's a factor I hadn't really considered until now, but everyone that I know who has dropped their landline for a cell-only life is definitely in the Democratic profile. 

On the other hand, those people most likely to skip voting entirely are also in the exact same demographic.

Vouchers again

Voucher results mixed
Overall, public-school students did better on state tests
By Jennifer Smith Richards / Columbus (OH) Dispatch

On the whole, Ohio students who used tax-funded vouchers to attend private schools last school year did no better on state tests than public-school students.

One commenter wrote:
"That's not the issue, is it? What we want to know is whether private school students did better than they would have in a public school. That's the issue. The only way you can come close to answering that question is with random samples and control groups--neither of which were used here."

Very true. In fact, random samples and control groups would tend to eliminate the selection bias inherent in the voucher system and make the comparisons even worse for the voucher schools. Add to that a size bias - the small voucher school that randomly does well is used as a club over the heads of the public school.  At the same time, the small voucher school that randomly does poorly is ignored .... until next year's random increase means that you can trumpet "Its huge improvement is the result of vouchers."

When the one with all the advantages doesn't win,
there is usually a reason.

Fundamentally, public schools should be the recipients of public school money. They are run by the town for the benefit of its citizens, are controlled by a School Board elected by the citizens, use a budget that is voted on by its citizens, and by and large are staffed by its citizens. Accountability is to the town.  At least in Vermont.

Private schools are not accountable to the town, do not have to make their finances public, do not have to answer to the taxpayers, and are run for the benefit of the school. Private and charter schools do not have to follow federal mandates for the education of all students, can dismiss or expel students for disciplinary or educational reasons without refunding the town, and can send kids back to the public school if their academic performance isn't up to par. Private schools can skim the best students off the top, "recruit" from other districts, include I20 students rather than IEPs or 504s, offer "scholarships," and do many other things to "win."

What's the real issue? If the charter and private schools cannot out-do the public schools despite their very real head start, we shouldn't be blaming the public schools.

Common Sense on Drug Stats

Every once in a while we get hit with the workshop that exists only to justify its own existence, while keeping the presenter employed. The drug-scare workshop is the prime candidate.

1. Funny About Money went to one. They said 50% of CC students are coming to class stoned or high on multiple drugs.
2. Our Drug Awareness Dude from the community center showed up with two suitcases of things to look for and police-approved fake samples of paraphernalia. 
3. The ex-cop came in to show us those brain scans of druggies vs non-druggies. "Look at the "holes" in this one and at the nice smooth brain over here."
4. The ex-druggie claimed to have taken this entire list of drugs and abused his kid, and here she is to confirm that he was an asshole. The poor girl stands there looking embarrassed as hell as he lists all the signs of alcoholism and drug use in your students and describes how he used to mistreat his family while under the influence.

All this time, I'm thinking, "Calm down. Scare tactics only work for a few minutes until people start thinking again." It burns me up. Here we are paying this clown a couple thousand to preach bullshit and take up a lot of time. Isn't there a better use of our resources?

The ex-druggie is spouting utter crap, trying to make himself look better in his own eyes. His daughter is still being abused and I have no sympathy for him. He's telling us the signs of alcoholism - slanty eyes in kids - and the admin are eating it up. No one is surprised when the email comes by the next day ... "Please stop asking kids about their parents' alcoholism." He's telling the kids the dangers of drugs and how they'll ruin your life. Most of the kids noticed he's getting paid a couple grand to speak to a gynasium full of kids for an hour. A few also noticed that he "wants to come back a lot because this is the worst school in the whole state and I'd really like to come back often to help you guys." Gee, thanks.

The ex-cop has gotten hold of some pictures that scare the bejeezus out of him, except that he doesn't understand them. "Look at the holes in this kid's brain." then, the best line, "See the awful colors? Those are caused by drugs."

Ahhh, it's a computer generated scan of activity levels, Dude. It's a graph. The colors were chosen for contrast. There are no physical holes, just activity levels below an arbitrary cutoff.

Then, there were the line graphs. No vertical or horizontal scale, no mention of what was actually being measured. "This is a graph of memory for a kid on drugs." Me: "Time in days, months? Number of items, type of items, first 800 digits of pi? What memories, what kind of mental tasks?" Him: "I don't know. I just got these graphs from their website." Me: "...."

But wait, there's more. There's the community anti-drug crusader.

Apparently, according to him, our students are taking any or all of over a hundred different types of drugs. He had displays in the two suitcases so therefore he must be telling the truth. We had problems: meth, heroin, speed, downers, pot, booze, cocaine, peyote, mushrooms, wine coolers and hard lemonade, injectables, glue and other inhalants -- even (this is no shit) "Some girls are soaking their hair scrunchies in gasoline so they can sit in class and sniff the fumes all period." Really? Don't you think someone might notice that? Doesn't this seem a bit far fetched?

So I asked him, "According to what you have seen and found, according to what the Police Chief standing next to you has seen and found, according to what the kids themselves tell you about what they've heard other kids are doing, what is really happening in this town?" "Alcohol, mostly. Marijuana."

What about the other things you're showing us? "Well, there was that one kid three years ago." Here's a little math for you: 100 - 2 = 98% bullshit.

Funny with Money was told by the college people responsible for policing and preventing it, that about fifty percent of students in a community college classroom, at any given time, are likely to be abusing some kind of drug, legally purchased or not, often more than one something.

"Substances" range from meth to over-the-counter cold pills and nostrums. So, the homeopathic "cold medicine" that contains less of anything useful or harmful than does the average placebo is also lumped into the statistic. And probably Hall's and Ricola.

Anyone else wondering why these geniuses are telling us all this? Maybe to justify their own salaries through irony? If this is such a problem, why haven't they done anything about it?

And seriously? Most community college kids are there on borrowed money and many are working fulltime, desperate for an education. They aren't going to class wasted, they're going to class tired. If they were wasted, they wouldn't bother going to class.

Why do I complain? Because overloading the classroom teacher with trivial bullshit that has no bearing on anything in their classroom is a lose-lose situation.


If you have us looking for wraiths that do not exist, then we will find those wraiths anyway and waste time, effort, soul, and most importantly, trust, barking up the wrong trees. Everyone loses.


If you tell us that slanty eyes in kids are a sure sign of parental alcoholism, then that is what the faculty will see. Don't be surprised if parents thus accused do not take it well.

If you tell us that 95% of our students are on drugs right now (via the ex-druggie, with the principal standing right next to him, nodding wisely), then we will misinterpret tired or worried or frustrated as "drugged." If you tell us that we should report everything, then an awful lot of nonsense will get thrown into the system -- parents will be called, kids will be called in, DYS will have to be notified -- because a kid had a fight with his girlfriend and didn't sleep well for the last two weeks.

That's how you lose when you are dealing with students.

You lose. YOU. LOSE.

Kids won't open up when they really need help. They will fall into the patterns you accuse them of. The parents will learn to NEVER trust the faculty or the school. The kids will withdraw from all of the possible support. Your school will take on a whole new feel and no one will like it.

If you try and tell students that the brain doesn't stop growing until the age of 25, so "Students shouldn't drink until then," you will come off as loony and hypocritical. Kids watch TV on Sunday for the football. We shouldn't expect the other messages to magically disappear: Alcohol is cool and very desirable and the perfect home has hundreds of cans in the fridge, strength and power are all-important (steroids and other PEDS), and women are supposed to dress like that.

If your presentation is mostly hyperbole and loaded with graphs you don't understand, your rhetoric will pass right on by. Kids are good at ignoring adult exaggeration and they hate hypocrisy.

Kids laugh at the lameness of the "Eggs on Drugs" commercials. They don't listen to teachers who bullshit them.

Is there a problem in my school? Yep.
Should the school discuss it? Yep.
Should we do something? Absolutely.

If you have information for us in the school, we will gladly listen to it and do our best to act on it.  Just follow one simple rule:

"Tell me the truth without exaggerating and stop wasting my time."

What Teachers Need from Administrators.

Dangerously Irrelevant has the beginning post of a series on what teachers want from administrators. Though I agree with some of this gentleman's points, there is some contradiction in what Brian Crosby says.

I agree with him when he wants teachers to have more say in professional development. No wasted meetings. This should be obvious, but isn't. Admin who haven't taught in years should temper their advice. Schools should hire teachers and then expect them to be teachers - stop second guessing all the time. Accountability is good if you allow the teachers to teach. If you require they follow a script, then hold the script accountable.

Absolutely: Change for its own sake is stupid. Adopting the gee-whiz Fad of the Month from a summer admin workshop while instituting a new math curriculum, new science curriculum, and new Common Core Standards only ruins any chance of knowing whether any reforms were successful. Maybe it's a science mindset, but you should only change one variable at a time if you want to isolate cause.

"Research" in education is often inapplicable to the school. Apply with care. Read the damn paper before you try to implement the hype in the newspaper report of the University press release of the heavily hedged Executive Summary. Don't expect the 150 college students in the study to behave in the same way as your high-schoolers.

When you make changes, you should be "experimenting" properly. Measure before and after. Have a control group. Add your data to that of the study. Isolate variables. Either track the proposed change scientifically or accept the fact that you are playing Russian Roulette with kids who won't get another chance at a high school education.

I'm not so thrilled with the idea of 2 weeks in the beginning of the year for planning - seems like too much and wouldn't be well used by the faculty. It's a nice idea in theory, but details go out the window at first contact with students. Besides, I like having the summer.

A big sticking point for me is the reform contradiction and I felt that his message lost its way.

Saying "Changing course constantly is very bad," he wants to stay away from the Summertime Fad and avoid jumping on the research-based fad. Then he wants to explore non-research-based pedagogies (i.e., fads), change to 21st century tech and wants his admin to "BEG for creativity and innovation." Which is it?

"Creativity and innovation" are euphemisms for "summer workshop fad" except for frame of reference. Is it always a fad if your admin proposes it but realistic and wonderful if it came from your own 6-hour workshop? Brian is tripping down this staircase.

The Charter Pendulum does a double-take.

One way from Pittsburgh, via Schools Matter:

Among charter school students, about 20 percent didn't meet basic academic standards in reading and math, compared with about 12 percent of district students, according to 2009 Pennsylvania System of Student Assessment test results ....
About 75 percent of Pennsylvania public school students scored advanced or proficient in reading and math, compared with about 59 percent of charter school students ...

and then back the other way from Reason TV, via Joanne Jacobs, :

Hurricane Katrina destroyed one of the worst public school systems in the U.S., says Reason TV. New Orleans started fresh with a system based on choice. Now, “60% of New Orleans students currently attend charter schools, test scores are up, and talented and passionate educators from around the country are flocking to New Orleans to be a part of the education revolution.”

all in the same day.

Makes you wonder how long it will take for them to notice that the gains by KIPP and other charters have more to do with selected students than with superior funding methods? That charter schools don't have the same SpecEd requirements? That charter and private schools are relieved of many regulations and have the flexibility of getting rid of problem students? (defined as students who cause trouble, or in the case of KIPP, don't want to work too hard or for 60 hours a week.)

And yes, it is still despicable to say that Katrina had a silver lining in that it "improved the public school system." Even if that caveat were true, and it's not, the statement assumes that death and massive destruction has a good side.

Boredom

via Joanne Jacobs, Bored in school and summarizing.

High school is boring, say two thirds of students in the 2009 High School Survey of Student Engagement.
# 41% went to school because of what they learn in classes.
# 23% went because of their teachers.
# 30% went because they enjoy being in school.

In related news, 79% of those who were interviewed at home last Saturday afternoon said they were bored, leading the Society for the Prevention of Childhood Boredom to recommend that home life be changed ("Reformed") so that kids could be stimulated constantly with creativity and intellectually stimulating projects.

Mothers who participated by purchasing the Home Boredom Reform Toolkit reported that they had taken the kids out for ice cream and had a "wonderful, fulfilling time." After further interviews, the teenagers' boredom response rate changed to 93.2%, most saying "Going for ice cream was SOOO boring."

Mothers who participated by buying a lawnmower, lining up a few potential customers and then letting the teenagers create a lawn-mowing/general yard work business with absolutely no further input from either parent reported that the teenagers didn't seem so bored anymore. The teenagers themselves couldn't be interviewed because they were out earning money or out spending that money on their car, their date or going for ice cream with their friends.

Students who are thinking about about dropping out say they don’t like the school, don’t like the teachers or don’t see any value in the work they’re asked to do. Students said they wanted chances to be creative at school. Most dislike lectures and like “discussions in which there are no clear answers.”

In other news, adults who don't like their jobs or who were just laid off in a nasty recession were encouraged to go back to school. There, they could take courses that improved their job prospects or trained them for a new career. Poofy courses and touchy-feely courses that discussed the feelings of the students are to be avoided. Writing, grammar and basic math courses are the recommendation, followed by computer courses and science. History and languages are also good. Avoid classes that are geared toward sparing the feelings of the students. Look instead for those that offer lots of useful information, delivered in a compact and efficient style ... like a lecture. Not a boring and monotonous one of course, but one delivered by a teacher who knows his stuff.

The report looks at schools that are trying to engage disaffected students.
Kealakehe High School in Kailua-Kona, Hawaii, has focused on building relationships between school staff and students. . . . According to the principal of Kealakehe, Wilfred Murakami, there is a perception that people love the school and that most students participate in school activities; HSSSE data revealed that actually just about half of the students say they love the school and, similarly, about half participate in school activities. The principal uses the data to raise important questions with his staff: “What about the rest of the kids? What are those kids doing?”

In a final bit of related news, The American Center for Conservative Liberal Studies in Education and Life in General has just released a study saying that teenagers who answer surveys in class report that answering surveys is really boring. Comments added at the bottom include, "I really wish we could just learn things instead of taking these ridiculous surveys." and "Great, another f****ng survey."

And that's why the Curmudgeon isn't a big fan of this type of "data-driven" school reform.

Teacher Quality Study

Via Joanne Jacobs, via Reuters, from Science comes a twin study on teacher effectiveness.

I’d be interested in the criteria for “strong” teacher and “weak” teacher. If the “strong” one is defined (as the linked article seems to imply) as someone whose 20 students did well on a single reading test, then perhaps it isn’t surprising that one of those students did better on a test.

At least this study seems to be comparing teachers within the same building, using twins. That’ll help control for income and such. I’m not as convinced that a single measure of just one skill is a particularly good way to differentiate, however, though my expertise is certainly not at the k-3 level. Assuming that you can even separate the discipline at this level I have to wonder: if one teacher is better at science or math while the first is only good at teaching the reading skills needed for that one test, the differences could be enough to label one or the other as “deficient” and that’s not a good way to do this.

I’d also like to know how the kids were placed in the various classes. If it’s random, then this study has more credibility, but if the placements are done for reasons of ability or behavior, then that test is measuring too many variables to definitively say one teacher is better than the other.

I'm sure the researchers tried to control for this, but schools do tend to separate kids on purpose and that will skew the study. If a weaker student is intentionally placed with teacher #1, then it shouldn't be surprising if the teachers show a difference.

All in all, a good study. Hopefully it's not taken directly to the Congress as the article suggests - it's way too early for that and the Congress would probably try to apply its lessons to high or middle schools and it's way to specific for that.

BTW, the photo that accompanied the article showed a child using an abacus during a national math test in India. If one of our teachers used an abacus during teaching and the other used a calculator, that would definitely change the results.

Article below the jump

Poor teachers may hamper good students: U.S. study
Thu Apr 22, 2010 4:45pm EDT
WASHINGTON (Reuters) - An unusual genetic study supports the argument that good teachers make a difference and shows that poor teachers may do damage, even to gifted students, researchers said on Thursday.
The study, published in the journal Science, showed that effective teachers help kids with the best genes read better, while poor teachers brought down all the children in a classroom to the same mediocre level.
The findings by behavioral geneticist Jeanette Taylor at Florida State University and colleagues could influence the debate in Congress, the White House and school districts across the United States about measuring the quality in schools.
"In circumstances where the teachers are all excellent, the variability in student reading achievement may appear to be largely due to genetics. However, poor teaching impedes the ability of children to reach their potential," Taylor and colleagues concluded.
To tease out the effects of genes and environment, the researchers turned to the time-tested model of twins. Identical twins share all their DNA, while fraternal twins share about half, or as much as any brother or sister.
Their theory: if one identical twin does better than his or her sibling in a different classroom, much of the difference must be due to the teacher.
They studied 280 identical twin pairs and 526 fraternal twin pairs in the first and second grades from a diverse selection of Florida schools.
To determine teacher quality, they used Oral Reading Fluency test scored for the entire classroom of each twin.
"(It's) a timed measure of how many words children can read in a passage," Taylor said in a Science podcast.
They checked to see how many more words children could read at the end of the year compared to the beginning.
"We felt that was a reasonable estimate of teacher effectiveness," Taylor said.
When teachers were good, the genes really mattered. If one identical twin excelled with a good teacher, the other did too. But if one twin had a strong teacher and the other twin had a weak teacher, twins with strong genetic potential did just so-so in the poor teacher's class.
"Better teachers provide an environment that allows children to reach their potential," Taylor said.
"As state and national policy increasingly focuses on teacher quality, the effect that teachers have on the strong documented genetic foundation of reading is an important question," the researchers wrote.
While great teachers do not guarantee success, they said, policymakers need to realize that good teaching is important, even for gifted children.
One weakness of the study -- the researchers threw out data from identical twins who happened to be in the same classroom who scored differently from one another on the reading test. Taylor said she did not know how many such cases there were.
(Reporting by Maggie Fox, editing by Stacey Joyce)

Those pesky nongraduating basketball players

I'm sure you've all read that Arne Duncan wants to prevent schools from joining the NCAA Basketball tournament if they don't graduate enough of their players.

I had a long rant thing going about data selection and narrow-minded politicians, coupled with a healthy dose of "Why hold black players to different standards than black non-players" and "What would you do about the players who don't want to graduate from college. What if they really want to be in the NBA but the NBA won't allow them to join, which has to be discrimination of some kind because white and Asian tennis players and golfers join their professional leagues at 16 and no one says a word."

But I decided that I'd just show these two graphs and ask Arne to explain the obvious problems:

Statewide College Graduation Rates from IPEDS Graduation Rate Survey

Humm, those seem to be aligning with red-state, blue-state demographics, too. Is that the real goal here? Or is Arne just a DUKE / Vermont fan like me? He wants to see all their competition eliminated by fiat instead of by a last-second three? You think Murray State, Northern Iowa and St. Mary's wanted anything less than the best competition, Arne? Get out of the way, Dude. Your mouth and that hole-in-the ground are getting confused again.

Yes, thank you, my bracket is getting CREAMED!
ESPN: 340 points * PCT(%)42.0% * RNK 2,773,415
AAAAAHHHHHHHHHHH!

Monday Morning Update:
ESPN: 380 points * PCT(%)55.4% * RNK 2,132,140

Colleges makes kids Liberal apparently.

The Chronicle quotes a study that college makes students more liberal but not smarter about civics.

I have so many questions, starting with basic methodology, below the fold:

1. Does A cause B, does B cause A or is it C causing both, or nothing?
2. Does the change in liberalism follow the change in civics knowledge or vice versa?
3. Is there a confounding factor, like the changing male-female ratio? or the changing attitude that "All students must go to college" - are we seeing the results of increased perception of need for a college education to make a good living?
4. Are younger people more likely to be liberal anyway, regardless of college?
5. Is civics something colleges deal with, or is this a K-12 thing?
6. Did the researchers control for age and SES? A PhD might have taken HS civics in 1975 and the HS graduate two years ago. Education has changed in the HS setting. (no shit).
7. Does ALL higher education make graduates more liberal or is the choice of college or major an issue?
8. Did the researchers bias their own study by excluding the military academies (like yours truly) and the religious schools like Brigham Young? (by asking 2500 random people that's what they essentially did)
9. Wealthier people take more college - was wealth a factor rather than the years in school?
10. The institute has a "tradition-minded view of issues" - was this the only study they did?& Or did they throw away data that would have changed their results?

I don't like this study. Results seem too pat, too simplistic. Too Glenn Beck-ian.

The survey contained 118 questions: 33 on civic knowledge, 39 on public philosophy, 29 on civic behavior, 16 on demographics, and 1 on popular culture.

Many questions are worded strangely or have answer choices that "guide" the respondent, while some are just odd. "Socrates, Plato, Aristotle, and Aquinas would concur that: ... " -- Maybe it's that I'm not a civics teacher. They were chosen from previous ISI tests, Immigration, and NAEP tests. This makes them perfect survey instruments?

I did find the 33 questions pretty easy - but I was able to read them and consider the answers. A typical telephone survey doesn't allow for consideration or for nuance. How would you respond if you were asked this one over the phone?

Free markets typically secure more economic prosperity than government’s centralized planning because:
a. the price system utilizes more local knowledge of means and ends
b. markets rely upon coercion, whereas government relies upon voluntary compliance with the law
c. more tax revenue can be generated from free enterprise
d. property rights and contracts are best enforced by the market system
e. government planners are too cautious in spending taxpayers’ money

The numbers are weird for a random survey, too: this "random telephone survey" of 2508 managed to include 240 people who self-identified as college professors. Really? 10% is very high. And a random telephone survey with all the quits, drops, and lying?

And then, the conclusions. They used regression to determine the effect of more education instead of just measuring it. More civics education meant more disagreement with phrases such as "The Bible is the Word of God," "The Ten Commandments are irrelevant today." "America corrupts otherwise good people." "Public school teachers should be allowed to lead prayers in school." "Homeschooling families neglect their community obligations." "Religion and science typically conflict." None of these correlates with conservative or liberal views and all of them are of the "We'll tell you the good answer" variety - bias built right in.

Oh well. Gotta get back to work.

Here's the article if it gets hidden in archives:

---

College Makes Students More Liberal, but Not Smarter About Civics, Study Finds

By Jill Laster
While many graduates of American colleges cannot answer basic civics questions, a higher education does make their opinions more liberal on controversial social issues, according to a new report issued on Friday by an academic think tank.

The Intercollegiate Studies Institute, an independent group with a tradition-minded view of issues, asked about 2,500 randomly selected people more than 100 questions to gauge their civic knowledge, public philosophy, civic behavior, and demographics.

"The Shaping of the American Mind," the fourth report from the institute on civic literacy, will be formally released on Wednesday.

Richard A. Brake, a co-author of the report, said he and his colleagues had sought to see what civic or social lessons students were learning in college.
The institute found that people who had attained at least a bachelor's degree were more likely than Americans whose formal education ended with a high-school diploma to take a liberal stance on certain controversial social issues. For example, 39 percent of people whose highest level of education was a bachelor's degree supported same-sex marriage, compared with 25 percent with a high-school diploma. The trend continued with advanced degrees: About 46 percent of people with master's degrees supported same-sex marriage, as did 43 percent of people with Ph.D.'s.

Previous surveys have found that, in general, college does not bring students up to a high level of civics knowledge. According to the Institute's 2008 report, based on a survey of 2,500, people whose highest level of educational attainment was a bachelor's degree correctly answered 57 percent of the questions, on average. That is three percentage points lower than a passing grade, according to the survey's authors. (This needed to be said? and quoted? and the reporter scored what? - Mr. C.)

Even earlier surveys showed that years in college were only slightly correlated to civics expertise. For a 2006 report the institute surveyed 14,000 college freshmen and seniors on basic civics questions. It found seniors answered an average of 53 percent of the questions correctly, just 1.5 percent higher than freshmen. (After the 2006 report was released, some experts questioned the study's methodology and focus on a small range of facts.) (And they will again.- Mr. C.)

Mr. Brake said results of the studies in the last four years showed that many universities do not place enough emphasis on civics or the basics of American history. He also called for universities to adopt better-balanced curricula.

"College graduates, whether it be current or graduated in the past, seem to have difficulty knowing basic things about our government and our history," Mr. Brake said. "Does college share all the blame? Of course not — this is a systemic problem, from K through 12 and all the way up. But universities train our teachers and train our leaders, so they play a role." (And newspapers play a role, too. As well as demographics. - Mr.C.)

Civics curricula have drawn concern recently from other critics, such as Bob Graham, the former U.S. senator from Florida who is now based at the University of Florida. He suggested, in an interview last summer with The Chronicle, that colleges be measured based on the number of their current students or graduates who participate in community-service or civic organizations.

Juxtaposition, Graphics, Lying Like a Rug

It's amazing what you can do with statistics. ("Lies, Damned Lies, and Statistics.") Here's a sample:



Funny how the addition of a graph makes a point stronger, while not necessarily making anything clearer or more correct.

The problem with this graph is its simplicity - those are not the only pertinent bits of information at play.

Teachers often can be incredibly one-sided and have too much effect on students by asking questions that require agreement with a false premise before answering. I have heard variations of all of these questions as I have passed by open classroom doors ...

• Did Clinton cause that improvement on his own or did the computer age have more of an effect on the economy?
• Did Bush really destroy the economy or were these events out of his control?
• Did Bush know that his failures, including "Mission Accomplished", would come back to haunt him and his legacy?
• What should Bush have done differently in the last six months of his Presidency?
• Why didn't Bush do something while the economy imploded?
  
• Is it a coincidence that Democrats have grown the economy while Republicans have not?
• Since Democrats grow the economy,as seen in the graph above, why shouldn't we pass Health Care reform and save the lives of those less fortunate?

Why is this okay? Is it just because too many people are innumerate (to use Paulos's word for it) that this kind of thing is promoted and passed on?

Just sayin'.

Not just for Global Warming

EUGENE VOLOKH ON DATA SHARING AND CLIMATE RESEARCH, though this sentiment applies to all research. This is how peer review works. It is why we know that fusion power is not yet commercially viable, that perpetual motion machines don't and that certain medicines are dangerous or effective.

My inclination would be to say that data should nearly always be shared. If you share your data, this lets others check the conclusions you draw from the data, as well as verifying the accuracy of the data against other available sources. They might disprove your arguments, or lead you to improve your arguments, or, if they reproduce your results, they might help prove the validity of your arguments. But in either case, science progresses better, and the decisions made based on the science are more reliable, than if you keep the data secret.
I can see some possible exceptions, for instance where there are concerns about the privacy of research subjects, or where the data was gathered as part of a commercial endeavor that requires that the data be kept proprietary for the commercial project to be viable, or some such. Even there I stress that the exceptions are merely possible; perhaps on balance the data should be shared even then, and in any event, even if there’s a good reason for the data not to be shared, people should view the research skeptically because of the lack of sharing.