Popular Post
Showing posts with label Statistics. Show all posts
Showing posts with label Statistics. Show all posts

Tuesday, March 29, 2011

Tuesday Child is full of grace ... 20, 21, and 22.

Problem 20

Difficulty: Easy, but fun. A new way to look at averages and the effect of an addition on the average, depending on n.

Problem 21

Difficulty: Linear programming.Any bets this will somehow work out to an integer?

Problem 22


Difficulty: Easy. Once you've got the details right.

Standard instructions for this series: No calculator allowed. Express answers in reduced form. Rationalize denominators. Radicals must be reduced. All numbers are base ten unless otherwise specified. Do not approximate radicals or π. Leave such answers as 1025π or √39, for example. Source: UVM Math Contest

answer 20, answer 21 and answer 21.

Thursday, January 20, 2011

Ray Allen wins again.

Funny guys, those Boston Celtics. Winning on an awesome Ray Allen shot and immense Shaquille O'Neal shoulders.

And yes, that tweet makes total sense.

BTW,
Reggie Miller: 2560 three-pointers.
Ray Allen: 2533 three-pointers. 13-21 shooting threes for the last five games. When's the record gonna be?
Data below the jump.
Celtics are 32-9 in this 82 game season so far. When, if he keeps up this pace, should he be buying lots of tickets for friends and family to help him celebrate the record? If you're going to go, you'll have to ask for time off but you'll need to ask well in advance or they'll just refuse you. Place your bets ... which game will it be?

Fri, Jan 21 vs Utah
Sat, Jan 22 @ Washington
Tue, Jan 25 vs Cleveland
Thu, Jan 27 @ Portland
Fri, Jan 28 @ Phoenix
Sun, Jan 30 @ Los Angeles
Tue, Feb 1 @ Sacramento
Fri, Feb 4 vs Dallas
Sun, Feb 6 vs Orlando
Mon, Feb 7 @ Charlotte
Thu, Feb 10 vs Los Angeles Lakers
Sun, Feb 13 vs Miami
Wed, Feb 16 vs New Jersey
Tue, Feb 22 @ Golden State
Thu, Feb 24 @ Denver
Sat, Feb 26 @ Los Angeles Clippers
Mon, Feb 28 @ Utah
Wed, Mar 2 vs Phoenix
Fri, Mar 4 vs Golden State
Sun, Mar 6 @ Milwaukee
Wed, Mar 9 vs Los Angeles
Fri, Mar 11 @ Philadelphia
Sun, Mar 13 vs Milwaukee
Mon, Mar 14 @ New Jersey
Wed, Mar 16 vs Indiana
Fri, Mar 18 @ Houston
Sat, Mar 19 @ New Orleans

Sunday, January 2, 2011

Driving While Impaired.

Not to mention how ridiculous you look.
It's funny (meh) what the reality is:

Several studies, such as a 2005 paper in the British Medical Journal, have found that talking on a cell phone, even with a hands-free device, causes more driver impairment than a 0.08 BAC. A 2001 American Automobile Association study found several other in-car distractions that also caused more impairment, including eating, adjusting a radio or CD player, and having kids in the backseat.
h/t - Reason magazine

And then there is having a baby in the front seat, safely protecting you from a collision with the steering wheel and the airbag. Of course the airbag will kill your kid when it impacts his face at >100 mph. (There's the equivalent of three shotgun shell loads inside it to make it inflate fast enough.)  If it doesn't inflate fast enough, then your body will crush your child through the steering wheel. Of course, that's not something that would EVER happen to our BrittBritt.


You can't fix stupid.

Wednesday, November 24, 2010

The Further You Extrapolate, the Sillier You Look

I'm always bemused by graphs that try to extrapolate too far.  Case in point at right (found by Darren).  The numbers previous to today are obviously known well but the future is a little cloudier.  Nevertheless, this graph claims to see a trend going rather upward, even though the trend for the next few years is downward.

Like the "population bomb" scare stories of the recent past, this one is just silly.

Why silly? Because it assumes that "given no change in policy, the numbers will do this."

Given no change? This is America.  We can't stop changing. The one thing that will never happen is steady state.  Somebody will introduce a bill to change this or that and your whole projection becomes worthless ... as if it meant much to begin with. We adapt in this country.  When the deficits were spiraling out of control under Reagan/Bush I, the country adapted, we nearly balanced the budget and we were starting to work on the debt.  Clinton's policies were less of a driver than the tech boom but, c'est la guerre. I'm frankly surprised that the National Review published it because it shows the sharpest drop under Obama's second term.


I had to tweak Darren, since he's a staunch conservative:

"Come on. Extrapolating a possibly exponential curve out to 2082? You really shouldn't, as a math teacher, have let that go without some comment. 

But I'll play along ... let's read the graph. The percentage surges upward in 1980-1984 when Reagan was doing those enormous deficit budgets -- to drive the Russians to bankruptcy, yes, but it still increases.

Then the economy recovered and those percentages dropped under Clinton (not that he was totally responsible for that, but it does make a good way to tweak conservatives!).  Likewise, the percentages rises under GWB, hits a peak in 2010 under Obama, and then shows a tremendous downward slope under the remaining years of Obama's term (and into his next?). Then, in 2016 (when the Republican presumably gets elected), the percentage starts to climb again.

I'm not sure that's exactly what you had in mind."

Darren responded later ...
"It's when the health care costs really start to kick in. Of course extrapolations that far out are silly, which is one of the reasons why I love the global warmers so much. Still, I don't see anyone arguing that our interest payments are going to go *down* any time soon."

No. But why is that graph any less ridiculous? The spending by Government is totally under the control of the Legislature and can be adjusted yearly as the political winds blow. If they wanted to, the Afgan war could cease in days, the bailouts could stop, the contracts could be ended and penalties paid.

Global warming isn't quite up to a vote by the House Ways and Means Committee. Natural processes don't stop on a whim.

I don't particularly care about the gloom and doom part of the debate anyway, so I'm probably not a good person to argue this with. I don't care about global climate as such, but I do care about MY air. If NYCity goes under water, it'd be turned into a modern Venice pretty quickly and Mankind would adapt. Florida would build dikes like Holland and Mankind would adapt. Ideal croplands would be found further north and, well, you get the idea.

Saturday, November 20, 2010

One-percent of a Standard Deviation

... is quite a bit smaller than a one-percent increase. In fact, 1% of a standard deviation is pretty damned small. It seems somebody needs a Statistics course:
Public schools located near private schools increased reading and math scores more than public schools that had little competition.
Huge, I tells ya.
For every 1.1 miles closer to the nearest private school, public school math and reading performance increases by 1.5 percent of a standard deviation in the first year following the announcement of the scholarship program. Likewise, having 12 additional private schools nearby boosts public school test scores by almost 3 percent of a standard deviation. The presence of two additional types of private schools nearby raises test scores by about 2 percent of a standard deviation. Finally, an increase of one standard deviation in the concentration of private schools nearby is associated with an increase of about 1 percent of a standard deviation in test scores.
Test scores rose more for elementary and middle schools than for high schools, perhaps because the scholarship made K-8 private schools affordable but didn’t cover as much of the tuition at private high schools.
Hummmm ...

Did the scores of the private schools drop at the same time as the public school rose?  If the public school scores rose, was it because the parents of weaker kids took the money and ran? Was it because Florida is investing heavily in on-line learning and told certain kids that their behavior was unacceptable IN school so they had to switch to the private school or take courses online? We'll never know but this is an equally valid interpretation of the facts as presented.

I find the "1.5 percent of SD per mile" statistic interesting but pretty meaningless. That's not a standard deviation, it's one-one hundredth of a standard deviation. That's the equivalent of SAT scores rising 1 point. Read the collegeboard's take on significance.

Just because you can see something in your educational microscope doesn't mean there's anything worth looking at.
The quote also says that this happens only in the first year following the announcement. So the average SAT scores went up about 1 pt.
Once.

Here are Florida's average SAT scores for the last couple years. Notice the yearly fluctuations larger than that touted by the article. Note, the standard deviation for SAT scores is typically 100 - 110 points. So 1.5% of a standard deviation would be 1.5 points.

The timeline is also interesting. The idea that the mere announcement of a private school makes a difference in the first year (but only in the first year) indicates that it's got nothing to do with the education provided since it takes some time for a kid to get an education. Statistically insignificant.

I'd be looking for information on who paid for this study and who has the most to gain by falsely trumpeting miniscule gains and falsely attributing them to the glorious private schools.

Tuesday, November 2, 2010

Polls may be hitting a wall.

Slashdot has a thought:
"The 'cellphone effect.' In 2003, just 3.2% of households were cell-only, while in the 2010 election one-quarter of American adults have ditched their landlines and rely exclusively on their mobile phones, and a lot of pollsters don't call mobile phones. Cellphone-only voters tend to be younger, more urban, and less white — all Democratic demographics — and a study by Pew Research suggests that the failure to include them might bias the polls by about 4 points against Democrats, even after demographic weighting is applied."
This will make the "science" of polling even more suspect.  It's a factor I hadn't really considered until now, but everyone that I know who has dropped their landline for a cell-only life is definitely in the Democratic profile. 

On the other hand, those people most likely to skip voting entirely are also in the exact same demographic.

Tuesday, July 27, 2010

TFA Studies Shouldn't Lie or Obfuscate

Joanne has a short piece on ‘Study laundering’ on TFA and I'll quote a few things here. The gist is that Studies on TFA are cherry-picking data to promote a predetermined viewpoint ... ya think?

I'm no fan of TFA in that I don't like the overall attitude of "I'm so incredible because I went to HA-VAAD and therefore everyone at one of those poor, low class schools - public schools, mind you - should welcome me in my magnificence and install me in a classroom of my choosing. The five-week summer training session is all I need to become a teacher, whether math or science or social studies - it doesn't matter because I'm so wonderful. I can even change from a math teacher to a social studies teacher at will."

I'm sure that there are some good teachers in the making at TFA. But many don't want to be teachers in the long run. They just want a temporary job and to feel good about themselves for a while before they take a real job in their chosen profession. What's not good about that is that an experienced veteran who is planning on staying, or someone who wants to be a teacher long-term, is moved aside for the temp. That's not good for the long-term health of a system.

Anyway, back to the study.
“Weaponized” education research and “study laundering” are illustrated by a Great Lakes Center study knocking Teach for America for high turnover and “mixed” performance, writes Eduwonk. Half of TFA teachers leave after two years and 80 percent leave after three, the study says. However, the researchers use data from studies that conflate TFA teachers who leave their original school placement after two years with those who leave the teaching profession, Eduwonk charges. A 2008 Harvard study (pdf), found that 61 percent of TFA teachers stay in teaching beyond the two-year commitment.
Definitions of terms are so important, aren't they? That conflation might be appropriate if the study noted it. Of course, TFA itself says: "These teachers, called corps members, commit to teach for two years in one of 39 urban and rural regions across the country." It's not like the study hasn't taken TFA at its word but it should have been more specific.
Teach For America surveys its alumni regularly and the most recent survey found that 65 percent of Teacher For America’s 20,000 alumni remain in education, with 32 percent continuing as teachers. And remember, that’s a survey of alums going back almost two decades now so that one in three figure should be viewed in that context as well as the larger context of TFA’s mission.
So 65 percent remain in teaching after two years but only half of those as teachers? Sounds like the original press release had the information correct and TFA is blustering its own spin. This is not a point in their favor.

Teacher churn is bad for a school, despite the supposed "wonderfulness of a TFA teacher." Teachers don't "Go bad" in the last few minutes of the school year. Those 70 percent of TFA people who didn't continue after 2 years were probably very clear about their desires soon after starting year 2 and the students knew it. This is a bad situation all around as the TFA are just filling out their time - I've never seen a lame duck teacher who was successful.

Then we look at the 32% - how long did they last? According to the study, a third of this later group left after the third year - who are these? Maybe only those TFAs who were unable to get a real job and just hung on for another year - bad news. I can't think those schools are well served by these long-term subs. It takes at least two years to get your feet under you and get your classes "right."
On the performance issue, studies that use rigorous methodology find that “Teach For America teachers perform as well or better than other teachers, not only emergency certified teachers but traditionally trained ones and veterans,” Eduwonk writes, including lots of link to research studies. The results are not mixed.
Debatable. I'd want to delve more deeply before I took a different set of studies as gospel. Wouldn't you? Which performance measures are they talking about? The ones that nobody can find any merit for? Probably just a test score comparison between a temp teacher or unlicensed one and the TFA. Hardly telling.

How about this little graphic. Talk about playing with statistics and implied information. Is it 10% of the 4510 are Black (450) and therefore 12% of the group is Asian, or is it that Blacks are 10% of the 33% (150)? Is a small percentage of blacks a problem because TFA is staffing black schools?  I figure it's written this way so that we infer that 33% are people of color and then add 10% Black and 8.5% Latino to get 52% minority?  Who's fudging the definitions now?

Sunday, March 21, 2010

Those pesky nongraduating basketball players

I'm sure you've all read that Arne Duncan wants to prevent schools from joining the NCAA Basketball tournament if they don't graduate enough of their players.

I had a long rant thing going about data selection and narrow-minded politicians, coupled with a healthy dose of "Why hold black players to different standards than black non-players" and "What would you do about the players who don't want to graduate from college. What if they really want to be in the NBA but the NBA won't allow them to join, which has to be discrimination of some kind because white and Asian tennis players and golfers join their professional leagues at 16 and no one says a word."

But I decided that I'd just show these two graphs and ask Arne to explain the obvious problems:

Statewide College Graduation Rates from IPEDS Graduation Rate Survey


Humm, those seem to be aligning with red-state, blue-state demographics, too. Is that the real goal here? Or is Arne just a DUKE / Vermont fan like me? He wants to see all their competition eliminated by fiat instead of by a last-second three? You think Murray State, Northern Iowa and St. Mary's wanted anything less than the best competition, Arne? Get out of the way, Dude. Your mouth and that hole-in-the ground are getting confused again.



Yes, thank you, my bracket is getting CREAMED!
ESPN: 340 points * PCT(%)42.0% * RNK 2,773,415
AAAAAHHHHHHHHHHH!

Monday Morning Update:
ESPN: 380 points * PCT(%)55.4% * RNK 2,132,140

Sunday, February 7, 2010

Colleges makes kids Liberal apparently.

The Chronicle quotes a study that college makes students more liberal but not smarter about civics.

I have so many questions, starting with basic methodology, below the fold:

  1. Does A cause B, does B cause A or is it C causing both, or nothing?
  2. Does the change in liberalism follow the change in civics knowledge or vice versa?
  3. Is there a confounding factor, like the changing male-female ratio? or the changing attitude that "All students must go to college" - are we seeing the results of increased perception of need for a college education to make a good living?
  4. Are younger people more likely to be liberal anyway, regardless of college?
  5. Is civics something colleges deal with, or is this a K-12 thing?
  6. Did the researchers control for age and SES? A PhD might have taken HS civics in 1975 and the HS graduate two years ago. Education has changed in the HS setting. (no shit).
  7. Does ALL higher education make graduates more liberal or is the choice of college or major an issue?
  8. Did the researchers bias their own study by excluding the military academies (like yours truly) and the religious schools like Brigham Young? (by asking 2500 random people that's what they essentially did)
  9. Wealthier people take more college - was wealth a factor rather than the years in school?
  10. The institute has a "tradition-minded view of issues" - was this the only study they did?& Or did they throw away data that would have changed their results?
I don't like this study. Results seem too pat, too simplistic. Too Glenn Beck-ian.

The survey contained 118 questions: 33 on civic knowledge, 39 on public philosophy, 29 on civic behavior, 16 on demographics, and 1 on popular culture.

Many questions are worded strangely or have answer choices that "guide" the respondent, while some are just odd. "Socrates, Plato, Aristotle, and Aquinas would concur that: ... " -- Maybe it's that I'm not a civics teacher. They were chosen from previous ISI tests, Immigration, and NAEP tests. This makes them perfect survey instruments?

I did find the 33 questions pretty easy - but I was able to read them and consider the answers. A typical telephone survey doesn't allow for consideration or for nuance. How would you respond if you were asked this one over the phone?
Free markets typically secure more economic prosperity than government’s centralized planning because:
a. the price system utilizes more local knowledge of means and ends
b. markets rely upon coercion, whereas government relies upon voluntary compliance with the law
c. more tax revenue can be generated from free enterprise
d. property rights and contracts are best enforced by the market system
e. government planners are too cautious in spending taxpayers’ money
The numbers are weird for a random survey, too: this "random telephone survey" of 2508 managed to include 240 people who self-identified as college professors. Really? 10% is very high. And a random telephone survey with all the quits, drops, and lying?

And then, the conclusions. They used regression to determine the effect of more education instead of just measuring it. More civics education meant more disagreement with phrases such as "The Bible is the Word of God," "The Ten Commandments are irrelevant today." "America corrupts otherwise good people." "Public school teachers should be allowed to lead prayers in school." "Homeschooling families neglect their community obligations." "Religion and science typically conflict." None of these correlates with conservative or liberal views and all of them are of the "We'll tell you the good answer" variety - bias built right in.

Oh well. Gotta get back to work.

Here's the article if it gets hidden in archives:

College Makes Students More Liberal, but Not Smarter About Civics, Study Finds

By Jill Laster
While many graduates of American colleges cannot answer basic civics questions, a higher education does make their opinions more liberal on controversial social issues, according to a new report issued on Friday by an academic think tank.

The Intercollegiate Studies Institute, an independent group with a tradition-minded view of issues, asked about 2,500 randomly selected people more than 100 questions to gauge their civic knowledge, public philosophy, civic behavior, and demographics.

"The Shaping of the American Mind," the fourth report from the institute on civic literacy, will be formally released on Wednesday.

Richard A. Brake, a co-author of the report, said he and his colleagues had sought to see what civic or social lessons students were learning in college.
The institute found that people who had attained at least a bachelor's degree were more likely than Americans whose formal education ended with a high-school diploma to take a liberal stance on certain controversial social issues. For example, 39 percent of people whose highest level of education was a bachelor's degree supported same-sex marriage, compared with 25 percent with a high-school diploma. The trend continued with advanced degrees: About 46 percent of people with master's degrees supported same-sex marriage, as did 43 percent of people with Ph.D.'s.

Previous surveys have found that, in general, college does not bring students up to a high level of civics knowledge. According to the Institute's 2008 report, based on a survey of 2,500, people whose highest level of educational attainment was a bachelor's degree correctly answered 57 percent of the questions, on average. That is three percentage points lower than a passing grade, according to the survey's authors. (This needed to be said? and quoted? and the reporter scored what? - Mr. C.)

Even earlier surveys showed that years in college were only slightly correlated to civics expertise. For a 2006 report the institute surveyed 14,000 college freshmen and seniors on basic civics questions. It found seniors answered an average of 53 percent of the questions correctly, just 1.5 percent higher than freshmen. (After the 2006 report was released, some experts questioned the study's methodology and focus on a small range of facts.) (And they will again.- Mr. C.)

Mr. Brake said results of the studies in the last four years showed that many universities do not place enough emphasis on civics or the basics of American history. He also called for universities to adopt better-balanced curricula.

"College graduates, whether it be current or graduated in the past, seem to have difficulty knowing basic things about our government and our history," Mr. Brake said. "Does college share all the blame? Of course not — this is a systemic problem, from K through 12 and all the way up. But universities train our teachers and train our leaders, so they play a role." (And newspapers play a role, too. As well as demographics. - Mr.C.)

Civics curricula have drawn concern recently from other critics, such as Bob Graham, the former U.S. senator from Florida who is now based at the University of Florida. He suggested, in an interview last summer with The Chronicle, that colleges be measured based on the number of their current students or graduates who participate in community-service or civic organizations.

Saturday, January 23, 2010

Polls, Damned Polls and Statistics.

From NYCeducator, who got it from Miss Cellania, who got it from PHDComics.com.

which brings to mind this gem from The Register

Since I'm in that sort of mood, I note that Pols (politicians) and Polls (the surveys) always seem to be liars of convenience. Go figure.

Monday, December 28, 2009

Juxtaposition, Graphics, Lying Like a Rug

It's amazing what you can do with statistics. ("Lies, Damned Lies, and Statistics.") Here's a sample:



Funny how the addition of a graph makes a point stronger, while not necessarily making anything clearer or more correct.

The problem with this graph is its simplicity - those are not the only pertinent bits of information at play.

Teachers often can be incredibly one-sided and have too much effect on students by asking questions that require agreement with a false premise before answering. I have heard variations of all of these questions as I have passed by open classroom doors ...
  • Did Clinton cause that improvement on his own or did the computer age have more of an effect on the economy?
  • Did Bush really destroy the economy or were these events out of his control?
  • Did Bush know that his failures, including "Mission Accomplished", would come back to haunt him and his legacy?
  • What should Bush have done differently in the last six months of his Presidency?
  • Why didn't Bush do something while the economy imploded?
  • Is it a coincidence that Democrats have grown the economy while Republicans have not?
  • Since Democrats grow the economy,as seen in the graph above, why shouldn't we pass Health Care reform and save the lives of those less fortunate?
Why is this okay? Is it just because too many people are innumerate (to use Paulos's word for it) that this kind of thing is promoted and passed on?

Just sayin'.

Tuesday, December 8, 2009

Percentage or Points?

This may seem like a minor point, but the implications are not. I have to give the people credit for using the phrase "12% drop" correctly but it doesn't excuse the conclusion.

Sandra Stotsky and Ze'ev Wurman say
"Yet, is it really the case that low-performing high school students would drop out if high school diploma requirements were ratcheted up? That doesn't seem to be the case in Massachusetts, which in 2008 reduced its dropout rate by 12% from the previous year."
You figure this to be an amazing drop worthy of Gates money. Then you follow the link and find that the dropout rate in Massachusetts went from 3.8% to 3.4%. Not only that, but this "proof" of their theory is further shaken by the clauses supplied -- a new way of counting who is which class, new definitions, new SIS system. This is another case of the random lucky bounce being taken as an incontrovertible law of nature that happens to be in your favor ... until the next unlucky bounce demands a new law of nature in your favor.

Did the ratcheting up cause the drop? Can't say. Correlation with two data points is not quite good enough to show cause.

Friday, November 27, 2009

Vaccines do not cause autism.

I'm not a fan of those such as Jenny McCarthy who run around trying to scare people into forsaking vaccines and known cures in favor of holistic medicine and other proven-to-be-pointless non-working treatments. The polio vaccine has saved countless children but Jenny is certain that it caused autism in her child so she campaigns against it. Not that she has any scientific evidence for that, of course. Not that she doesn't constantly rail against "scientific evidence" as if repeating the words will somehow turn them into blasphemy.
“I do believe sadly it's going to take some diseases coming back to realize that we need to change and develop vaccines that are safe. If the vaccine companies are not listening to us, it's their fucking fault that the diseases are coming back. They're making a product that's shit. If you give us a safe vaccine, we'll use it. It shouldn't be polio versus autism.”
Jenny McCarthy in Time Magazine, April 2009
Learn science, people. Do proper research. Trust logic.

I have had students tell me that the government is just trying to make money off the swine flu vaccine, that it'll just make you sick, that it'll cause all kinds of things. Then, of course, they get in a car and drive like idiots so I'm not terribly surprised that they can't take this seriously. At least the vast majority of the students in our school have gotten the shot a couple weeks ago.

Preventable IllnessesPreventable Deaths

You can keep up to date with the Jenny McCarthy Body Count.

You can check this data at the CDC's Morbidity and Mortality Reports, if you trust those **scientists**