Generate Python docs from pytorch/pytorch@46a5e12

Author: pytorchbot

main/.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: 415b8e915ec5776ff33d82339ead392c
+config: 2cd854d2e4565aea0d120a636e3d768b
 tags: 645f666f9bcd5a90fca523b33c5a78b7

main/_images/RReLU.png

@@ -1414,7 +1414,7 @@ list_contains
 
 .. note::
 
-    Tags: :doc:`python.assert <python.assert>`, :doc:`torch.dynamic-shape <torch.dynamic-shape>`, :doc:`python.data-structure <python.data-structure>`
+    Tags: :doc:`torch.dynamic-shape <torch.dynamic-shape>`, :doc:`python.assert <python.assert>`, :doc:`python.data-structure <python.data-structure>`
 
     Support Level: SUPPORTED
 
@@ -1469,7 +1469,7 @@ list_unpack
 
 .. note::
 
-    Tags: :doc:`python.control-flow <python.control-flow>`, :doc:`python.data-structure <python.data-structure>`
+    Tags: :doc:`python.data-structure <python.data-structure>`, :doc:`python.control-flow <python.control-flow>`
 
     Support Level: SUPPORTED
 

main/_images/ReduceLROnPlateau.png

@@ -60,7 +60,7 @@ list_contains
 
 .. note::
 
-    Tags: :doc:`python.assert <python.assert>`, :doc:`torch.dynamic-shape <torch.dynamic-shape>`, :doc:`python.data-structure <python.data-structure>`
+    Tags: :doc:`torch.dynamic-shape <torch.dynamic-shape>`, :doc:`python.assert <python.assert>`, :doc:`python.data-structure <python.data-structure>`
 
     Support Level: SUPPORTED
 

main/_sources/generated/exportdb/index.rst.txt

@@ -62,7 +62,7 @@ list_unpack
 
 .. note::
 
-    Tags: :doc:`python.control-flow <python.control-flow>`, :doc:`python.data-structure <python.data-structure>`
+    Tags: :doc:`python.data-structure <python.data-structure>`, :doc:`python.control-flow <python.control-flow>`
 
     Support Level: SUPPORTED
 

main/_sources/generated/exportdb/python.assert.rst.txt

@@ -147,7 +147,7 @@ list_contains
 
 .. note::
 
-    Tags: :doc:`python.assert <python.assert>`, :doc:`torch.dynamic-shape <torch.dynamic-shape>`, :doc:`python.data-structure <python.data-structure>`
+    Tags: :doc:`torch.dynamic-shape <torch.dynamic-shape>`, :doc:`python.assert <python.assert>`, :doc:`python.data-structure <python.data-structure>`
 
     Support Level: SUPPORTED
 
@@ -202,7 +202,7 @@ list_unpack
 
 .. note::
 
-    Tags: :doc:`python.control-flow <python.control-flow>`, :doc:`python.data-structure <python.data-structure>`
+    Tags: :doc:`python.data-structure <python.data-structure>`, :doc:`python.control-flow <python.control-flow>`
 
     Support Level: SUPPORTED
 

main/_sources/generated/exportdb/python.control-flow.rst.txt

@@ -755,7 +755,7 @@ list_contains
 
 .. note::
 
-    Tags: :doc:`python.assert <python.assert>`, :doc:`torch.dynamic-shape <torch.dynamic-shape>`, :doc:`python.data-structure <python.data-structure>`
+    Tags: :doc:`torch.dynamic-shape <torch.dynamic-shape>`, :doc:`python.assert <python.assert>`, :doc:`python.data-structure <python.data-structure>`
 
     Support Level: SUPPORTED
 

main/_sources/generated/exportdb/python.data-structure.rst.txt

@@ -141,6 +141,18 @@ the autograd engine.
   to calling backward, and so your code will need to handle such objects as if they were
   tensors filled with zeros. The default value of this setting is True.
 
+In addition to ``ctx`` methods, the :class:`~Function` class supports the following
+class attribute:
+
+- :attr:`~Function.clear_saved_tensors_on_access`: When set to ``True`` on the
+  :class:`~Function` subclass, accessing ``ctx.saved_tensors`` in the backward pass
+  will clear the internal references to those tensors. This allows the tensors to be
+  freed as soon as the local variables returned by ``saved_tensors`` go out of scope,
+  rather than waiting for the buffers to be cleared at the end of the Node's execution.
+  This can reduce peak memory usage in backward passes where saved tensors are only
+  needed once. The default is ``False``. Note that ``saved_tensors`` can only be
+  accessed once when this is enabled; a second access will raise an error.
+
 **Step 3:** If your :class:`~Function` does not support double backward
 you should explicitly declare this by decorating backward with the
 :func:`~function.once_differentiable`. With this decorator, attempts to
@@ -257,6 +269,37 @@ And here, we optimize the above example by calling set_materialize_grads(False):
             # Gradients of non-Tensor arguments to forward must be None.
             return grad_output * ctx.constant, None
 
+Here is an example using ``clear_saved_tensors_on_access`` to reduce peak memory during
+the backward pass. This function computes two matrix multiplications, and in backward
+we free the intermediate tensor after computing its gradient but before computing
+the remaining gradients::
+
+    class TwoMatmuls(Function):
+        clear_saved_tensors_on_access = True
+
+        @staticmethod
+        def forward(ctx, x, weight1, weight2):
+            inter = x.mm(weight1)
+            ctx.save_for_backward(x, weight1, inter, weight2)
+            return inter.mm(weight2)
+
+        @staticmethod
+        def backward(ctx, grad_output):
+            x, weight1, inter, weight2 = ctx.saved_tensors
+
+            # Compute gradients for second matmul
+            grad_weight2 = inter.t().mm(grad_output)
+            grad_inter = grad_output.mm(weight2.t())
+
+            # Free inter and weight2, no longer needed
+            del inter, weight2
+
+            # Compute gradients for first matmul
+            grad_weight1 = x.t().mm(grad_inter)
+            grad_x = grad_inter.mm(weight1.t())
+
+            return grad_x, grad_weight1, grad_weight2
+
 If you need any "intermediate" Tensors computed in :meth:`~Function.forward` to be saved,
 either they must be returned as outputs, or combine ``forward`` and :meth:`~Function.setup_context`
 (see :ref:`combining-forward-context`).

main/_sources/generated/exportdb/torch.dynamic-shape.rst.txt

@@ -141,7 +141,7 @@
   fbq('track', 'PageView');
 </script>
 <script>
-  document.documentElement.setAttribute('data-version', 'main (2.11.0a0+gitbbc4e47 )');
+  document.documentElement.setAttribute('data-version', 'main (2.11.0a0+git46a5e12 )');
 </script>
 <noscript>
   <img height="1" width="1" src="https://www.facebook.com/tr?id=243028289693773&ev=PageView&noscript=1" />