Tutorial 3: Science is Hard

How Hard is Science?

Science is hard. We hope this tutorial helps you to develop your intuition for some of the fundamental difficulties with scientific research.

But Science works, right?

We're 100% in favor of doing science. But we should be very careful about how we interpret scientific research. Don't put too much stock into novel results, especially if they're highly counterintuitive. Many initial findings, if they don't have a firm theoretical backing, are probably wrong. Support efforts to replicate, and try to look at all the evidence1.

Better Living Through Modeling

One of the best ways to develop your intuitions about how a system works is to play around in that system as much as possible. Sometimes it's even better to play around with a model version of the system, where you know how all the parts work individually, so you can see how those parts connect to the behavior of the whole system. This is the case even when the model represents a highly simplified version of the real world system. To borrow a line from philosopher William Wimsatt, we can use false models as a means to developing truer theories2. That's the broad purpose of the interactive games in this tutorial. As a final offering, you will be able to play with the game's parameters, and change the base rate, power, and false positive rate. We have no specific exercises for you in this next game. Just play around and possibly discover something new.


1. For some pretty reasonable suggestions about improving the practice and consumption of scientific research, see:
JPA Ioannidis (2014) How to make more published research true. PLoS Medicine 11(10): e1001747.

2. Wimsatt WC (1987) False models as means to truer theories. In M Nitecki and A Hoffman (eds), Neutral models in biology. Oxford University Press.