WEBVTT
1
00:00:00.005 --> 00:00:03.005
- [Instructor] Broadcasting is a way to perform an operation
2
00:00:03.005 --> 00:00:07.000
between tensors that have similarities in their shapes.
3
00:00:07.000 --> 00:00:09.009
This is an important operation in deep learning.
4
00:00:09.009 --> 00:00:11.001
The common example
5
00:00:11.001 --> 00:00:13.009
is multiplying a tensor of learning weights
6
00:00:13.009 --> 00:00:16.002
by a batch of input tensors,
7
00:00:16.002 --> 00:00:19.003
applying the operation to each instance
8
00:00:19.003 --> 00:00:21.000
in the batch separately
9
00:00:21.000 --> 00:00:24.004
and running a tester of identical shape.
10
00:00:24.004 --> 00:00:27.004
Here's an example with two by four
11
00:00:27.004 --> 00:00:32.000
multiplying one by four returns a tester of shape two,
12
00:00:32.000 --> 00:00:37.006
with dimension of one, with values of two and 16.
13
00:00:37.006 --> 00:00:40.005
If you're familiar with broadcasting semantics
14
00:00:40.005 --> 00:00:42.005
and NumPy ndarrays,
15
00:00:42.005 --> 00:00:45.005
you'll find the same rules apply with PyTorch.
16
00:00:45.005 --> 00:00:48.001
The exception to the same shape rule
17
00:00:48.001 --> 00:00:50.001
is tensor broadcasting.
18
00:00:50.001 --> 00:00:52.008
The rules for broadcastings are,
19
00:00:52.008 --> 00:00:57.001
one, each tensors must have at least one dimension,
20
00:00:57.001 --> 00:00:59.001
no empty tensors.
21
00:00:59.001 --> 00:01:02.001
Comparing the dimension sizes of two tensors,
22
00:01:02.001 --> 00:01:07.003
going from last to first, each dimension must be equal
23
00:01:07.003 --> 00:01:11.001
or one of the dimensions must be of size one
24
00:01:11.001 --> 00:01:15.008
or the dimension does not exist in one of the tensors.
25
00:01:15.008 --> 00:01:17.007
Tensors of identical shape,
26
00:01:17.007 --> 00:01:20.009
of course, are trivially broadcastable,
27
00:01:20.009 --> 00:01:22.009
as you saw earlier.
28
00:01:22.009 --> 00:01:25.008
Here are some examples of situations
29
00:01:25.008 --> 00:01:29.005
that honor the rules and allow broadcasting.
30
00:01:29.005 --> 00:01:33.002
We first have tensor A
31
00:01:33.002 --> 00:01:37.001
and tensor B, here, has the third and second dimensions
32
00:01:37.001 --> 00:01:43.005
identical to tensor A, with dimension one absent.
33
00:01:43.005 --> 00:01:46.009
Tensor C has third dimension equals to one,
34
00:01:46.009 --> 00:01:50.002
second dimension identical to A
35
00:01:50.002 --> 00:01:53.008
and first dimension empty.
36
00:01:53.008 --> 00:01:57.002
Tensor D has third dimension identical to A,
37
00:01:57.002 --> 00:01:59.006
second dimension equals to one
38
00:01:59.006 --> 00:02:06.005
and first dimension empty, and it is broadcastable as well.
39
00:02:06.005 --> 00:02:10.004
PyTorch's broadcast semantics are compatible with NumPy's,
40
00:02:10.004 --> 00:02:13.000
but the connection between PyTorch and NumPy
41
00:02:13.000 --> 00:02:15.002
goes even deeper than that.
42
00:02:15.002 --> 00:02:18.005
If you have existing ML or scientific code
43
00:02:18.005 --> 00:02:22.002
with data stored in NumPy ndarrays,
44
00:02:22.002 --> 00:02:26.003
you may wish to express the same data as PyTorch tensors.
45
00:02:26.003 --> 00:02:30.003
Whether to take advantage of PyTorch's GPU acceleration
46
00:02:30.003 --> 00:02:34.007
or its efficient abstractions for building ML models.
47
00:02:34.007 --> 00:02:39.003
It's easy to switch between ndarrays and PyTorch tensors.
48
00:02:39.003 --> 00:02:42.009
Let's take a look at some examples.
49
00:02:42.009 --> 00:02:48.003
Here's a conversion and here is the expected output.
50
00:02:48.003 --> 00:02:51.001
PyTorch creates a tensor of the same shape
51
00:02:51.001 --> 00:02:55.002
and containing the same data as NumPy array, going as far as
52
00:02:55.002 --> 00:03:00.003
to keep NumPy's default 64 bit float data type.
53
00:03:00.003 --> 00:03:03.008
The conversion can just as easily go the other way, here,
54
00:03:03.008 --> 00:03:07.001
as the example of a version.
55
00:03:07.001 --> 00:03:10.005
And the expected outcomes are here.
56
00:03:10.005 --> 00:03:14.001
It's also important to know that these converted objects
57
00:03:14.001 --> 00:03:16.007
are using the same underlying memory
58
00:03:16.007 --> 00:03:18.006
as their source objects,
59
00:03:18.006 --> 00:03:22.007
meaning that changes to one are reflected in the other.
60
00:03:22.007 --> 00:03:25.001
And here's an example,
61
00:03:25.001 --> 00:03:28.007
and the expected output is shown here.
62
00:03:28.007 --> 00:03:32.007
And this concludes the Intro to PyTorch NumPy Bridge.