- [Voiceover] Starting in 1919, Ronald Fisher worked
at Rothamsted Experimental Station for 14 years.
He was quite young.
He was in his late 20s when he arrived.
It was there that he developed
and refined many of the techniques that he's famous
for and that are still in use.
So what was he up to?
He was using statistics to study how to best
apply artificial fertilizer to optimally grow winter wheat
and other crops.
While he was at it, Fisher laid the groundwork
of statistical inference, invented experimental design
featuring randomization and Inova.
The first edition of Statistical Methods
for Research Workers appeared in 1925.
What's important to underscore is how his work
here is a great example
of the kind of environment where these techniques work
especially well.
After all, it was during this period that he invented them.
These were highly controlled experiments
using randomization.
Everything was planned in advance, often called prospective.
He famously said, "To consult the statistician
"after an experiment is finished
"is merely to ask him to conduct a post mortem examination.
"He can perhaps say what the experiment died of."
The statistician has to be involved
before the experiment is conducted.
I love this passage in The Book of Why, they write
"At the time, the idea of randomly designed experiments
"horrified Fisher's statistical colleagues.
"Fisher literally drawing from a deck of cards
"to assign subplots
"to each fertilizer may have contributed to their dismay."
Why was this so important?
He was measuring the effectiveness of each fertilizer.
So he had to control for the influence of other factors.
For instance, there's water,
and hours of sunlight and soil fertility.
Anyone with plants at home soon discovers
that moving a plant
from one window to another can make a difference.
Fisher had acres of outdoor plantings
as part of his experiment.
There were a lot of variables in addition to the fertilizer.
As we shall see in the course, there are two ways of dealing
with this randomization and control variables.
When you can do it, randomization is better
because you don't have to itemize all
of those other factors, add them to your data set,
and statistically remove their influence.
As we shall see, there's lots of complications
with control variables.
So it was Fisher's job to carefully perform experiments
with different conditions under his control, specifically
the variable under study; fertilizer.
Then he used randomization to control for the influence
of all of the other factors that could potentially interfere
with his analysis of fertilizer.
Like the sunlight, water, and soil fertility.
Not only did Fisher innovate these techniques,
his book became the definitive source
for statistical research.
After Rothamsted, he was also a very influential professor
and academic.
I think it's powerful to pause for a moment
and reflect on just how influential Fisher and his book was.
It's all discussed in a book, Bernoulli's Fallacy.
In the book, "The tests are supplemented
"by many numerical examples and tables
"of the relevant distributions.
"For an experimental scientist
"without advanced mathematical training
"the book was a godsend.
"All such a person had to do was find the procedure
"corresponding to their problem and follow the instructions.
"As a result, Statistical Methods
"for Research Workers was enormously successful.
"It went through fourteens between 1925 and 1970
"and it became such the industry standard
"that anyone not following one of Fisher's recipes
"would have a hard time getting results published."
These techniques were powerful.
They still are powerful.
They represent how modern experiments are conducted.
But the reason that Fisher's influence is important
to mention is that we won't always be able to
apply these techniques like the winter wheat research
at Rothamsted.
And when you're doing anything other
than a randomized trial, you have the weight
of all of that statistical history on your shoulders,
so to speak.
In the companion course, we experienced this
as the debate between stats and machine learning.
And in this course we'll experience that
as the debate between traditional statistics represented
by Fisher and Beigian statistics.
But not to worry,
by the end of the course we'll have options
and an expanded toolkit at our disposal.