- [Instructor] We're going to stay in the waste dataset,
and now we're going to go up to analyze, regression,
linear OLS alternatives, and our second choice, lasso.
So when you look at this, it looks so similar to ridge
that you really can't tell the difference.
Let's populate the variables here.
Everything's identical so far.
And go to options,
and this looks very, very similar as well.
So what's the difference between the two?
Well, there's 2 differences that are noteworthy.
It's a different kind of penalty that's being applied,
and then the second difference that's worth mentioning
is that while Ridge has the coefficients approach 0
when they're not a good predictor,
they can actually achieve 0 in lasso.
So some folks favor it as a data reduction technique,
but I'm actually going to click on cancel and not proceed,
because we're about to practice the grid search.
And we're going to do the grid search in elastic net,
and it will make sense in a moment
why we want to jump to elastic net.
You may find, and here are the variables already populated,
and here are the options already populated.
The reason that I've preloaded this
is that we're going to do such a thorough search
that it takes quite a few seconds,
and on some machines, maybe longer than that,
for this to run.
So I've preloaded it and it's already in the output.
Here's why we're jumping to elastic net.
You certainly could try all 3, ridge, lasso,
and elastic net on your data.
But what makes elastic net unique
is it's a combination of ridge and lasso.
As we've seen, ridge and lasso look very, very similar.
They just have a different type of penalty under the hood.
Well, what elastic net does is both penalties,
and you can search all the possible values
for the ridge regression penalty,
then you can search the possible values
for the lasso penalty
and find the best combination of the two.
So that's why I've chosen elastic net
to practice our grid search.
So what I've done is the L1 ratio
is the one associated with lasso,
and that has to go from 0.1 to 1,
and I've chosen an increment of 1 to do it.
It can't be 0.
SPSS won't let you do that,
and it has a maximum value of 1.
Now, if we go down here to the alpha,
in elastic net, this is sometimes called the L2 penalty
to contrast the two.
L1, which we've just talked about, is associated with lasso,
and the alpha here is associated with ridge.
I'm starting it with a minimum of 1,
which you might recall is the default.
Then I'm taking it all the way up to 100 in increments of 1.
I'm going to click on cancel, because I've already run this.
We're going to go over to the output, and here it is.
So you can see that we've got a set of coefficients.
We'd want to be looking
at the unstandardized coefficients here.
These are the coefficients that have been calculated for us.
Over here on the left,
we can see the L1 ratio and the alpha.
So that means that, as far as lasso is concerned,
it's searched from 0.1 to 1 and chose 0.9,
meaning it's the high end of that,
it's applying a fairly large lasso-style penalty.
Perhaps because it went with a higher L1 ratio,
the alpha didn't go up.
Now, when you try this, you're going to get different numbers,
because remember, there's a holdout here,
and sometimes it won't be very successful,
because even ridge, lasso, and elastic net
are going to have a hard time with a dataset
that has this much multilinearity.
But on this particular run,
you can see that the training R squared
is 70% of variance explained, and the holdout is about 52.
That's actually not bad.
If you were to try this multiple times,
you would find that that's actually a pretty good run.
So there you have it.
There's much, much more to say about this, but now you know
that there's some really powerful additional algorithms
available to you powered by Python,
and they look just like SPSS
in the menus, in the dialogs, and in the output.