Is .data still useful in pytorch?
I'm new to pytorch. I read much pytorch code which heavily uses tensor's .data
member. But I search .data
in the official document and Google, finding little. I guess .data
contains the data in the tensor, but I don't know when we need it and when not?
.data
was an attribute of Variable
(object representing Tensor
with history tracking e.g. for automatic update), not Tensor
. Actually, .data
was giving access to the Variable
's underlying Tensor
.
However, since PyTorch version 0.4.0
, Variable
and Tensor
have been merged (into an updated Tensor
structure), so .data
disappeared along the previous Variable
object (well Variable
is still there for backward-compatibility, but is deprecated).
Paragraph from Release Notes for version 0.4.0
(I recommend reading the whole section about Variable
/Tensor
updates):
What about
.data
?
.data
was the primary way to get the underlyingTensor
from aVariable
. After this merge, callingy = x.data
still has similar semantics. Soy
will be aTensor
that shares the same data withx
, is unrelated with the computation history ofx
, and hasrequires_grad=False
.However,
.data
can be unsafe in some cases. Any changes onx.data
wouldn't be tracked byautograd
, and the computed gradients would be incorrect ifx
is needed in a backward pass. A safer alternative is to usex.detach()
, which also returns aTensor
that shares data withrequires_grad=False
, but will have its in-place changes reported byautograd
ifx
is needed in backward.
Aside from @benjaminplanche 's answer, I'd use it to manually change values of parameters.
For instance, I have the following model:
model = nn.Sequential(nn.Linear(10, 1))
and for some reason, I'd like to manually update the values of its parameters. Then, I can do:
for param in model.parameters():
param.data = 10 * param.data # multiply the parameter values by 10.
Note that if we remove .data
behind param
, the parameter values won't be updated.
Such use can be found in BYOL (Bootstrap your own latent) and this Github webpage for BYOL pytorch implementation.