Ben Mathew Economics
  • Home
  • Bio
  • Blog
  • Recommended
  • Mailing List
  • Contact

The Biggest Problem Facing Science

1/22/2014

0 Comments

 
New York Times article on publication bias:
Fears that this is resulting in some questionable findings began to emerge in 2005, when Dr. John P. A. Ioannidis, a kind of meta-scientist who researches research, wrote a paper pointedly titled “Why Most Published Research Findings Are False.”

Given the desire for ambitious scientists to break from the pack with a striking new finding, Dr. Ioannidis reasoned, many hypotheses already start with a high chance of being wrong. Otherwise proving them right would not be so difficult and surprising — and supportive of a scientist’s career. Taking into account the human tendency to see what we want to see, unconscious bias is inevitable. Without any ill intent, a scientist may be nudged toward interpreting the data so it supports the hypothesis, even if just barely.

The effect is amplified by competition for a shrinking pool of grant money and also by the design of so many experiments — with small sample sizes (cells in a lab dish or people in an epidemiological pool) and weak standards for what passes as statistically significant. That makes it all the easier to fool oneself.

That's the theory. Here's some data:
... C. Glenn Begley, who is chief scientific officer at TetraLogic Pharmaceuticals, described an experience he had while at Amgen, another drug company. He and his colleagues could not replicate 47 of 53 landmark papers about cancer. Some of the results could not be reproduced even with the help of the original scientists working in their own labs.
Now is that an artifact of publication bias too? Who knows?!

I feel that a big leap forward in science right now would be a category of journals that only commissions studies replicating important findings (or asking important new questions) and will publish the results, regardless of the answer. That will beat publication bias.
Determine what you want to study ex-ante and publish the results regardless of how it turns out. Only then can we really know what's going on.
0 Comments



Leave a Reply.

    Ben Mathew

    Author of Economics: The Remarkable Story of How the Economy Works

    Archives

    October 2016
    September 2016
    February 2016
    November 2015
    September 2015
    July 2015
    June 2015
    May 2015
    March 2015
    February 2015
    December 2014
    November 2014
    October 2014
    September 2014
    July 2014
    June 2014
    May 2014
    April 2014
    March 2014
    February 2014
    January 2014
    December 2013
    November 2013
    October 2013
    September 2013
    August 2013
    July 2013
    June 2013
    May 2013
    April 2013
    March 2013
    February 2013

    Categories

    All

    RSS Feed

Powered by Create your own unique website with customizable templates.