blueollie

Statistical stupidity from NPR

I am no longer surprised when NPR commits a statistical howler.

Today, they aired a touching story about a black hockey player and the racism that he endured. So far, so good.

But then, they go on to say this:

Despite stars like P.K. Subban and Evander Kane, hockey remains a stubbornly white sport. Only about five percent of NHL players are black 28 years after James.

Uh, exactly why is this a problem?

1. Backs are about 12.6 percent of the US population.

2, Blacks are about 2.9 percent of the Canadian population.

3. These days, the NHL gets about 50 percent (just over) of its players from Canada, and just under 25 percent from the United States.

So making the assumption that one would expect zero black players from other than the US or Canada (think of the hockey powers of this world), IF the representation were exactly proportional, we’d expect:

(.5)(2.9) + (.25)(12.6) = 4.6 percent, which is about what they have.

So why is 5 percent a problem? My guess: they assume that blacks are grossly over represented in football and basketball, they should be in hockey as well? If so, why?

Of course, this isn’t as bad as some idiot academic administrator with a Ph. D. insisting that everyone needs to be above the department average, but …well…let’s just say that I am cranky and that NPR comes across as smug.

February 27, 2015 Posted by | media, social/political, statistics | , | Leave a comment

stats, oz effects, and observant football players….

In the discussions about poverty and racism, I’ve been very vocal about parents being the ones responsible for feeding their kids. (here and here) Don’t have kids that you can’t afford to raise properly! Yes, this attitude draws the ire of many, including those who vote the same way that I do.

But when discussing irresponsible parenting, poverty, social pathologies and the like, we need data and we need to analyze it honestly. So, the headlines go: “unwed motherhood is up” and you read:

Census demographers said that single motherhood, while on a steady uptick since the 1940s, has accelerated in recent years. The birth rate for unmarried women in 2007 was up 80 percent in the almost three decades since 1980, the report said. But in the previous five years alone, between 2002 and 2007, it was up 20 percent.

Echoing the findings of many academic studies, the Census Bureau report said women with college degrees and higher household incomes are far less likely to be single mothers than are women who have lower household incomes and less education. […]

Overall, 36 percent of all births in the United States were to unmarried mothers in 2011, the year that the census analyzed from answers provided in the American Community Survey.

In the Washington region, 28 percent of births are to unmarried women. In the District, more than half of all births, 51 percent, were to unwed mothers. Maryland also had a higher rate than the national average, with 39 percent of all births out of wedlock. Virginia, in contrast, had a lower rate than the national average, with 31 percent of births to women who are not married.

The census also found that Asian mothers were the least likely to be unmarried, with just 11 percent of new Asian mothers being single. White single mothers also were below the national average, at 29 percent. Among Hispanics, 43 percent of all new mothers were unmarried, as were 68 percent of all African American women who had recently given birth.

Yep….the percent of births to unwed mothers is up! So, it follows that unwed women (especially black women) are having more kids than before? Uh…no.

Remember: “percent” is a type of fraction and it is: \frac{unwed mom births}{total births} So if the numerator (the top) goes up, the percent goes up. But…if the bottom goes down by more than the top goes down, then the fraction, and hence the percentage, goes up! And we see:

(via the sometimes preachy, finger wagging Tim Wise)

Looking first at the broader issue of so-called “illegitimate children” in the black community, those who forward this argument simply do not understand how to read or interpret basic statistical information. They claim, for instance that the “out-of-wedlock birth rate” for black females has skyrocketed; but in fact, nothing could be further from the truth. In reality, actual birth rates for unmarried black women (which means the number of live births per 1000 such women) has dropped dramatically. From 1970-2010, the birth rate for unmarried black women fell by nearly a third, from 95.5 births per 1000 unmarried black women to only 65.3 births per 1000 such women. In other words, unmarried black women are already doing exactly what conservatives would have them do: namely, having fewer children. This means that even if we were to accept the absurd argument that out-of-wedlock childbearing is evidence of cultural pathology, black culture must then be steadily getting healthier and less pathological, rather than more so. In a given year, for every 100 single black females, between ninety-three and ninety four of them will not have a baby—hardly evidence that out-of-wedlock childbearing is a normative experience for black women.
The common confusion on this issue seems to stem from the fact that although unmarried birth rates have fallen considerably, the share of children born in the black community who are born out of wedlock has indeed doubled since the early 1970s. It sounds like a big deal perhaps, but what does that statistic really signify? If unmarried black women are cutting back on childbearing — and remember, that’s what the data says — the increase in the percentage of black births that are births to single moms can’t possibly be the result of those moms’ increasing “irresponsibility.” Rather, this statistical phenomenon must be due to an entirely different factor, and indeed it is: namely, married black couples have cut back even further on childbearing than single moms have. If married black couples are having far fewer children than before, and are cutting back even faster than single women, the overall percentage of births that are out-of-wedlock will rise, owing nothing to the supposedly irresponsible behaviors of single black folks. If black married couples suddenly reverted to their family size norms of fifty years ago, the share of black births to unmarried moms would plummet, even if there were no further drop in the birth rates for single black women at all.

Moral: when talking about “percentage of”, remember that you are dealing with a ratio, which has both a numerator and a denominator.

Now of course, this requires actually knowing some mathematics (albeit at an elementary level) and while this makes you smarter and more likely to engage in disciplined thinking, it is unlikely to make you popular. Paul Krugman (speaking about Dr. Oz) explains:

Simon Wren-Lewis had an interesting piece on why the financial sector buys into really bad macroeconomics; he suggested that financial firms aren’t really interested in anything but very short-term forecasting, and that

economists working for financial institutions spend rather more time talking to their institution’s clients than to market traders. They earn their money by telling stories that interest and impress their clients. To do that it helps if they have the same worldview as their clients.

Thinking about Dr. Oz also, I’d suggest, helps explain a related puzzle: even if you grant that the right wants alleged experts who toe the ideological line, why can’t it get guys who are at least competent? Why do they recruit and continue to employ people who can’t do basic job calculations, or read their own tables and notice that they’re making ridiculous unemployment projections, and so on?

My answer has been that anyone competent enough to avoid these mistakes would also be unreliable — he or she might at some point actually take a stand on principle, or at least balk at completely abandoning professional ethics. And I still think that’s part of the story.

But I now also suspect that the personality traits you need to be an effective entertainer on inherently not-so-much-fun subjects like health or monetary policy are inherently at odds with the traits you need to be even halfway competent. If Dr. Oz were the kind of guy who pores over medical evidence to be sure he knows what he’s talking about, he probably couldn’t project the persona that wins him such a large audience. Similarly, a hired-gun economist who actually knows how to download charts from FRED probably wouldn’t have the kind of blithe certainty in right-wing dogma his employers want.

So how do those of us who aren’t so glib respond? With ridicule, obviously. It’s not cruelty; it’s strategy.

Oh, how I see this. Krugman wrote about a famous incident in which a popular trader was confronted with the fact that every bit of advice he gave was completely wrong, and how anyone who listened to him would have lost money. But hey, he really knows how to yell and draw applause:

So, there was a fun moment on CNBC: Rick Santelli went on a rant about inflation and the Fed, and CNBC analyst Steve Liesman went medieval on him:

It’s impossible for you to have been more wrong, Rick. Your call for inflation, the destruction of the dollar, the failure of the US economy to rebound. Rick, it’s impossible for you to have been more wrong. Every single bit of advice you gave would have lost people money, Rick. Lost people money, Rick. Every single bit of advice. There is no piece of advice that you’ve given that’s worked, Rick. There is no piece of advice that you’ve given that’s worked, Rick. Not a single one. Not a single one, Rick. The higher interest rates never came, the inability of the U.S. to sell bonds never happened, the dollar never crashed, Rick. There isn’t a single one that’s worked for you.

Of course, he got applause because he shares the same world view of those applauding him.

And my goodness, I think that I’ve seen some of this locally. When one looks at the leaders of some local institutions, it is easy to tell from watching what moves they make that they really don’t know what they are doing. But they are sure good at getting the “right” type of people to like them. I’ve seen this in the Navy as well. Remember when the US Submarine Greenville sank a Japanese ship because it did a risky surfacing exercise to impress some civilians and didn’t follow proper procedures?

The commander of the submarine was a classmate of mine at Annapolis and I went to Nuclear Power school with him. Even then, he was an expert at cutting corners when no one was looking, but telling the superior officers what they wanted to hear when they were around; he convinced them that he “was one of them”. It was a type of “affinity fraud”.

Now of course, Paul Krumgan is an economist and he talked about losing weight. He never looked fat to me; in fact he looks like many mathematicians in the sense that most of us appear to be normal sized. You notice that at conferences, though my mind’s eye detects that, as a group, we are starting to get fatter.

Well, as far as us being more slender than normal:

obesityratebyoccupation

Now this spread surprises me; I’d guess that firefighters and police officers would be required to stay physically fit. I’d guess wrong, unless this figure is “inflated” by things like private security guards.

Note: I can recommend the article, as it is about the employer’s interest in helping employees with their weight problems.

Football players
I can recommend this Jon Stewart video; it is a short clip that attacks the attack on the “don’t shoot” and “I can’t breathe” protests. If nothing else, listen to the last minute in which a pro football player explains that “a call for justice should threaten no one”.

December 21, 2014 Posted by | mathematics, NFL, politics, poverty, social/political, statistics | , , , , , , , , , | Leave a comment

I am not slowing down with age…and the globe isn’t warming either.

No, I am not slowing down with age.

Evidence:

1. Tracy Trot 4 miler: 2013: 33:07. 2014: 32:59.
2. Turkey Trot 3 miler: 2013: 24:56. 2014: 24:53
3. River Run: (Certified course) 2011: 26:56. 2012: 25:03 2013: 24:56 2014: 24:42
4. Race for the Cure: 2012: 25:13. 2013: 25:48. 2014: 25:27
5. Run to Remember: 2012: 24:34. 2014: 24:17
6. Best 4 5K runs: 2011: 25:43. 2012: 25:01 2013: 25:04 2014: 24:56

So, by all but one metric (Race for the Cure), I am getting faster with age!

Is that absurd?

Of course it is; in 1998 I ran a few 5Ks under 20 minutes; from 1997 to 1999 I was typically under 21 and my last sub 21 was in 2001; last 7 minutes a mile pace or faster was 2002.

What the mild “local improvement” represents is my improving after knee surgery in 2010; in 2009 I was running mostly 24:00 to 24:20 or so.

Why I bring this up: you see similar absurdities in climate change denial.

rodbell-graph

November 27, 2014 Posted by | climate change, environment, science, statistics | , , | Leave a comment

Limits of “Common Sense”…

Workout notes It was 23 F and sunny when I started a run (shuffle, really) and I had intended to go 10K. I ended up doing my double corn stalk course; I didn’t start my stop watch but estimate that I ran the 8.1 miles in roughly 1:31-1:32. It wasn’t much of an effort; very slow and steady.

Ferguson shooting
Yes, African Americans have a reason to worry about encounters with police. But the risk of being killed by police is low compared to other risks. It is just that mundane murders and accidents, while FAR, FAR more numerous, don’t make the news nearly as often.

In fact, people often struggle to understand how statistics affects their lives. For example, much of the time, early detection isn’t of much value for many types of cancers. Many cancers aren’t dangerous, others multiply so rapidly that early detection doesn’t help much; there is only a few types of cancers that early detection helps with. This shows up statistically.

Statistics can be brought to bear on “miracle stories” as well:

A bit more about science : evidently some science magazines are trying to increase sales by manufacturing a “crisis” in evolutionary theory that just isn’t there. Sure, there IS debate as to the mechanisms of evolution and research continues to be done. But that is really more “status quo”.

And no: just because evolution doesn’t make sense TO YOU doesn’t mean that it isn’t true, no matter what your religion says.

Speaking of religion Jerry Coyne points us to this humorous cartoon:

rabbitgodvsduckgod

November 25, 2014 Posted by | evolution, running, science, social/political, statistics | , , , , | Leave a comment

Misleading statistical correlations

I’ve talked about something like this before.

In Peoria: the “flagship” running race is the Steamboat races: there is a flat 4 mile race which attracts elite runners as well as most of the “casual” runners, and a very hilly 15k race (9.3 mile) which does NOT attract elite competition, but does attract many “serious” amateur runners.

Hence you get the following (I used what was my age group (50-54) as a comparison):

4 mile men: 138 finishers, median time 37:05, median pace: 9:16
15K men: 45 finishers, median time 1:19:50, median pace: 8:34

4 mile women: 128 finishers median time 46:10, median pace: 11:32
15K women: 27 finishers, median time: 1:28:41, median pace: 9:32

So, if one were to blindly use statistical correlation, one might conclude that one speeds up when one switches from running a flat 4 mile (6.4 km) course to a far hillier, 15K (9.3 mile) one.

Of course, what is going on is that the shorter race tends to attract the more casual runners; few people (especially 50 year old people) decided on a whim to see how fast they can do 15K, especially if they aren’t an experienced runner.

This is an example of “regression to the mean”; note how much larger the 4 mile groups are.

But a look at statistics reveals something else.

At this weekend’s 4 miler, I noticed that I placed “better” among “all males” (42/82) than I did among the 50-59 old males (6/8); that is .53 to .75. I figured: “ok, small race” and decided to take a look at the Steamboat 15K statistics.

So, in my age group, I was 38/43 or .88; but MY SAME TIME would have gotten me 40/60 (.67) in the 35-39 and 46/60 (.76) in 30/34. Once again, my performance placed me better in YOUNGER age groups than in older ones.

So, does this mean that “getting older makes you faster?” Uh, no. My times as a 38 and 39 year old were 1:08 and 1:07 respectively; lately I’ve been pushing 1:30 in this race.

So, just what is going on?

Here is my conjecture: perhaps a “fit but not that serious about running workout bro” might decide to take a crack at the 15K without doing a lot of specific running training, but when one gets into their 50’s, only the more serious runners even attempt the distance?

In other words, the folks I used to beat in my age group so many years ago are no longer running, and only those who used to beat me are still out there.

Then one might also point out that while my age group placing is suffering, I am actually improving next to ALL 50 year olds, since, at races, I am comparing myself to the 50 year olds who regularly race.

That is, I am too stupid to admit that I suck at running and to just give it up. :-)

November 3, 2014 Posted by | running, statistics | , | Leave a comment

Class, Gaussians and Islam in the world…

Islam in the world I wrote about this earlier. Evidently many conflate criticism of Islam (and its practices) with a justification for discrimination against Muslims; those are very different things. A Pakistani woman wrote an open letter to explain why the points brought up by Bill Maher and Sam Harris are worthy ones:

And in yesterday’s issue of Pakistan Today, you’ll find her piece: “An open letter to Ben Affleck“. As a few quotes below will show, she goes after Affleck for trying, as she argues, to minimize the plight of Muslim women like her. Just a bit to give you the flavor:

Noble liberals like yourself always stand up for the misrepresented Muslims and stand against the Islamophobes, which is great but who stands in my corner and for the others who feel oppressed by the religion? Every time we raise our voices, one of us is killed or threatened. I am a blogger and illustrator, no threat to anyone, Ben, except for those afraid of words and drawings. I want the freedom to express myself without the very real fear that I might be killed for it. Is that too much to ask?

When I wrote a children’s book that carried a message of diversity and inclusivity for everyone, my life changed. My book, ‘My Chacha (uncle) is Gay’ has the innocent anti-homophobia message, ‘Love belongs to everyone’. This was not palatable to many of my Muslim brothers and sisters.

Since that project I have been declared an ‘enemy of God’ and deemed worthy of death. All because I want to help create a world where South Asian children too can have their stories told, so they too can know that love comes in all forms, and that that’s okay. My Muslim brothers and sisters were hit hard by this work because it addresses the issue of homophobia within our own community. It is not something they can pass off as ‘Western’ immorality. Just like they deny that any issues exist within the doctrine of Islam, many deny that homosexuality exists amongst good, ‘moral’ Muslims. Just like that, millions of people’s existence is denied. Please do not defend people who think this way, and let me tell you Ben, many ‘good’ Muslims do think this way.

What you did by screaming ‘racist!’ was shut down a conversation that many of us have been waiting to have. . . You became an instant hero, a defender of Islam.

Here is something to remember: the most radical right wing American Christians would be …flaming liberals if they were Muslims:

In your culture you have the luxury of calling such literalists “crazies”, like the Westboro Baptist Church, for example. In my culture, such values are upheld by more people than we realise. Many will try to deny it, but please hear me when I say that these are not fringe values. It is apparent in the lacking numbers of Muslims willing to speak out against the archaic Shariah law. The punishment for blasphemy and apostasy, etc, are tools of oppression. Why are they not addressed even by the peaceful folk who “aren’t fanatical, who just want to have some sandwiches and pray five times a day? Where are the Muslim protestors against blasphemy laws/apostasy? Where are the Muslims who take a stand against harsh interpretation of Shariah? These sandwich-eating peaceful folk do not defend those suffering in the name of Islam, Ben, and therein lies our problem.

Accuracy in media
Here is an example of Fox News reporting something that is blatantly false, but not issuing a correction. The claim was that Colorado law allowed for people to print out their own ballots; that is only true for Military personnel living overseas.

Epic Class Warfare rant
Marc Randazza is a famous First Amendment lawyer. He can’t stand some attitudes exhibited by some rich people, even though he is well off. Read his “candy woman” rant.

Ok, I have a few hang ups about Halloween Trick or Treaters but these tend to be “Larry Davidish”.

Mathematics and statistics Some statistical distributions appear over and over again. This one appears to be half Gaussian, half exponential.

October 27, 2014 Posted by | Fox News Lies Again, mathematics, political/social, politics/social, poverty, religion, rich, science, social/political, statistics | , , , , | Leave a comment

My take on Birthdays

mybirthdaycard

Now, if there is some omnipotent deity that knows when you will die, this is correct. But as someone else points out: we die via a “bathtub curve” function.

bathtubcurve

That is, given our current age, say x , we have an “expected time left to live l(x) and as we age, the quantity x + l(x) increases, even though, after the risk of early death is over, l(x) is a decreasing function of x .

Of course, l(x) is changed if there is a war, famine, pestilence, etc.

September 16, 2014 Posted by | humor, mathematics, statistics | , , , | Leave a comment

Golden Rule, Statistics, etc.

The Golden Rule
The Golden Rule states: “do onto others as you would have done onto you” or something like that.

But that sometimes fails. Here is a quick example:

When I go out and run or walk, I like to “get away from it all”; I usually go alone and just get it in. I might take in sights of various kinds. But for me, it is “just go”.

Personally, I prefer what happens on popular running paths in Chicago or Austin: no one greets you. You just go. That doesn’t mean that there isn’t interaction; on occasion I’ve gotten in an impromptu race with someone who wouldn’t let me pass. That is fine.

But in small towns, people like to greet. If someone greets me, I wave and that is about it. Those who can give a prolonged Good Moooorning! aren’t working hard enough, IMHO. Some want to stop and chat.

So, what would I like to be “done onto me”???? I want to be left alone. But around here, most don’t want that.

Statistics
In football, most of the time (with the exception of a North Dakota State here or there), FCS teams lose to FBS teams, and non-BCS teams lose to BCS teams. The main reason is that the bigger time programs, well, recruit better players. Now they don’t get them all; on occasion a gem gets passed over only to eventually end up on an NFL team. It does happen. But, statistically speaking, the better players end up in the “power programs”, and the win-loss records demonstrate this. Statistically speaking, the scouts usually get it right, though there are always exceptions.

The same holds true in academics. For example, here and there you might see a person with a 24 ACT do well in mathematics or engineering. But the vast majority of the time, they do not. In the long run, the tests do a decent job of predicting, though they do miss a few people.

August 31, 2014 Posted by | social/political, statistics | , , , | Leave a comment

Bias of many types…and a walk

Today’s workout: end of “leisure” workout. I did my 8.1 cornstalk course in 2 hours (some rain…I didn’t get that wet) and then 2 more miles on the treadmill: 12:00/11:20 to get 23:20. I wanted to do at least a little faster than marathon pace.

RIP: BKS Lyengar, famous yogi and author of Light on Yoga.

Here he is in 1977 when he was in his late 50’s. What flexibility, strength, and body control!

Bias
Survivorship bias: this is the annoying tendency to see, say, a dozen successful companies, see what they have in common, and then conclude that what they have in common is what made them successful. Nope; you have to see how many companies did those same things and WERE NOT successful, among other things. From the article:

This is what Pomona College economist Gary Smith calls the “survivor bias,” which he highlights as one of many statistically related cognitive biases in his deeply insightful book Standard Deviations (Overlook, 2014). Smith illustrates the effect with a playing card hand of three of clubs, eight of clubs, eight of diamonds, queen of hearts and ace of spades. The odds of that particular configuration are about three million to one, but Smith says, “After I look at the cards, the probability of having these five cards is 1, not 1 in 3 million.” […]

Smith found a similar problem with the 1982 book In Search of Excellence (more than three million copies sold), in which Tom Peters and Robert Waterman identified eight common attributes of 43 “excellent” companies. Since then, Smith points out, of the 35 companies with publicly traded stocks, 20 have done worse than the market average.

Depression I talked about depression in an earlier post. Here is some of what science knows about it right now:

Racism

See the subtle racism here? The idea is that this black Attorney General who has spoken out about race relations is somehow too “emotionally invested” or biased to be even handed. Why would a black Attorney General be any less evenhanded than a white one? And shouldn’t we be far more concerned with an Attorney General who did NOT see race relations as a problem?

Here: Kansas City police officer posts a snarky post about Michael Brown’s character (the dead teenager in Ferguson) and shows a photo of a young black man with a gun and money in his mouth. But this black man is some guy in Oregon…not Michael Brown. It is amusing that police officers everywhere are telling us to not to rush to judgement but… :-)

I suppose that given that we have 300+ million people in this country and a lot of police officers, a few are bound to be crackpots.

Racism in sports
Sadly, some African American athletes have racist stuff directed at them. Here is an example (Eddie Chambers, an elite boxer)

August 20, 2014 Posted by | boxing, racism, science, social/political, statistics, walking, yoga | , | Leave a comment

Politics: emotional issues robs us of abstract reasoning ability…

Good Vox article here. Moral (for me): mathematical and statistical reasoning really disciplines our thinking, BUT does not convince non-technical people.

This is one reason discussing issues with people outside of math, science and engineering departments is so difficult for me.

July 29, 2014 Posted by | mathematics, politics, politics/social, social/political, statistics | | Leave a comment

Follow

Get every new post delivered to your Inbox.

Join 665 other followers