succumbing to peer pressure

Tuesday, March 20, 2007

When Scientists Argue

When 'experts' disagree, or rather, various talking heads can point to various experts (either accurately or inaccurately) to support their diametrically opposed opinions, who should the public believe? Typically, the public believes whichever side they've always believed, and moves on to washing dishes or giving the kid a bath or whatever else occupies the majority of their brain space. But a suggestion was made at my conference last week that has coincidentally come up again and again for me this week. The suggestion, during a panel on the role of biostatisticians in policy development, was that it's time for the American Statistical Association to start weighing in on some of today's major controversies. Perhaps not officially endorsing a 'side' but writing a few user-friendly articles parsing the available data. As data analysts, we are, allegedly, good at spotting faulty conclusions, poor interpretations, weaknesses in data collection, etc. etc. We could, at the very least, plow through any false media claims to 'scientists disagree' when that is clearly not the case (*cough*global warming!*cough*). We are also supposed to be (at least, those of us worth our salt) capable of explaining results to non-technical audiences. Here again perhaps we could lend a hand, elaborating on the sort of equivocating scientists naturally do yet which inevitably leads the lay person to conclude that there is a lack of agreement within a given community of scholars. (a scientist's careful treading around words like proof, theory, and evidence is enough to convince the average American that he or she doesn't know what the hell he or she is talking about)

This came up at that dinner party last week, then I was arguing with someone about what swath of the population made up the majority of minimum wage earners. I contend it's single women (from an albeit biased source - Ms. Magazine) while my opponent argued that it's predominantly teenagers and young adults, none of whom are primary breadwinners. The problem is, we may both be (somewhat) correct. The Bureau of Labor Statistics reports minimum wage stats with a denominator of the general-hourly-wage-earning public. So their stats tell you what percent of hourly-wage-earning women earn the minimum (4%) NOT what percent of minimum wage earners are women (I haven't sufficiently googled to try to find a source for Ms's numbers). Do you see the difference? Furthermore, the BLS says that about half of minimum wage earners are under 25, and a little more than a fourth are 16-19. Now, those may be high school and college students working holidays and weekends. But without further information we can't conclude (as the Heritage Foundation linked earlier does) that those under 25 minimum wage earners are in school, living at home, not the sole breadwinner. So there are two problems with the data here - different denominators, and potentially inaccurate (and implicit) assumptions.

Example number 2 - Recently a friend asked me to use my quantitative wherewithal to wade through an ongoing tiff between Michael Spagat and Amnesty International and Human Rights Watch. What Spagat is claiming, given the people who listen to him, could be tremendously damaging. But I want to give the problem due thoughtfulness (my friend's request is both flattering and daunting).

It would definitely be nice if my professional association did more of this kind of thing.


Anonymous Anonymous said...

I'm not sure how feasible it'll be to apolitically/nonideologically assess the relative merits of competing data on an already highly contentious issue. Economics spends a lot of time thinking about design of empirical studies, and necessary statistical controls. And it seems that (at least partly) we tend to be more troubled by the possible caveats/criticisms of the studies we want to dislike (unless they are completely ironclad), and we dismiss as implausible the alternate stories in studies we want to like (as long as they are objectively pretty good).

While there are some issues (global warming, evolution, etc) where most of the agenda-influence dissenters are outside the community, I'd think that the vast majority of contentious issues where an impartial evidence referee would be useful would have both sides within the community.

That or you'd get something like the split between the American Bar Association and the Federalist Society: the main official body is seen as too biased and so a splinter group is formed to try to be the "referee for the other side".


8:26 PM  

Post a Comment

<< Home