norm

Documentation

treetensor.torch.norm(input, *args, **kwargs)

Returns the matrix norm or vector norm of a given tensor.

Examples:

>>> import torch
>>> import treetensor.torch as ttorch
>>> t1 = torch.randn(3, 4)
>>> t1
tensor([[ 0.0363, -1.7385,  1.0669,  2.6967],
        [ 0.0848,  0.2735,  0.3538,  0.2271],
        [-0.1014,  1.1351, -0.5761, -1.2671]])
>>> ttorch.norm(t1)
tensor(3.8638)

>>> tt1 = ttorch.randn({
...     'a': (2, 3),
...     'b': {'x': (3, 4)},
... })
>>> tt1
<Tensor 0x7f95f684f4a8>
├── a --> tensor([[-0.5012,  2.0900,  0.0151],
│                 [-0.5035,  0.2144,  0.8370]])
└── b --> <Tensor 0x7f95f684f400>
    └── x --> tensor([[ 0.3911,  0.3557, -2.2156,  0.3653],
                      [-0.3503,  1.2182, -0.2364, -0.2854],
                      [-1.5770, -0.7349,  0.8391, -0.2845]])
>>> ttorch.norm(tt1)
<Tensor 0x7f95f684fa20>
├── a --> tensor(2.3706)
└── b --> <Tensor 0x7f95f684f978>
    └── x --> tensor(3.2982)

Torch Version Related

This documentation is based on torch.norm in torch v2.0.1+cu117. Its arguments’ arrangements depend on the version of pytorch you installed.

If some arguments listed here are not working properly, please check your pytorch’s version with the following command and find its documentation.

python -c 'import torch;print(torch.__version__)'

The arguments and keyword arguments supported in torch v2.0.1+cu117 is listed below.

Description From Torch v2.0.1+cu117

torch.norm(input, p='fro', dim=None, keepdim=False, out=None, dtype=None)

Returns the matrix norm or vector norm of a given tensor.

Warning

torch.norm is deprecated and may be removed in a future PyTorch release.

Use torch.linalg.norm(), instead, or torch.linalg.vector_norm() when computing vector norms and torch.linalg.matrix_norm() when computing matrix norms. Note, however, the signature for these functions is slightly different than the signature for torch.norm.

Args:

input (Tensor): The input tensor. Its data type must be either a floating

point or complex type. For complex inputs, the norm is calculated using the absolute value of each element. If the input is complex and neither dtype nor out is specified, the result’s data type will be the corresponding floating point type (e.g. float if input is complexfloat).

p (int, float, inf, -inf, ‘fro’, ‘nuc’, optional): the order of norm. Default: 'fro'

The following norms can be calculated:

ord	matrix norm	vector norm
‘fro’	Frobenius norm	–
‘nuc’	nuclear norm	–
Number	–	sum(abs(x)**ord)**(1./ord)

The vector norm can be calculated across any number of dimensions. The corresponding dimensions of input are flattened into one dimension, and the norm is calculated on the flattened dimension.

Frobenius norm produces the same result as p=2 in all cases except when dim is a list of three or more dims, in which case Frobenius norm throws an error.

Nuclear norm can only be calculated across exactly two dimensions.

dim (int, tuple of ints, list of ints, optional):

Specifies which dimension or dimensions of input to calculate the norm across. If dim is None, the norm will be calculated across all dimensions of input. If the norm type indicated by p does not support the specified number of dimensions, an error will occur.

keepdim (bool, optional): whether the output tensors have dim

retained or not. Ignored if dim = None and out = None. Default: False

out (Tensor, optional): the output tensor. Ignored if

dim = None and out = None.

dtype (torch.dtype, optional): the desired data type of

returned tensor. If specified, the input tensor is casted to :attr:’dtype’ while performing the operation. Default: None.

Note

Even though p='fro' supports any number of dimensions, the true mathematical definition of Frobenius norm only applies to tensors with exactly two dimensions. torch.linalg.norm() with ord='fro' aligns with the mathematical definition, since it can only be applied across exactly two dimensions.

Example:

>>> import torch
>>> a = torch.arange(9, dtype= torch.float) - 4
>>> b = a.reshape((3, 3))
>>> torch.norm(a)
tensor(7.7460)
>>> torch.norm(b)
tensor(7.7460)
>>> torch.norm(a, float('inf'))
tensor(4.)
>>> torch.norm(b, float('inf'))
tensor(4.)
>>> c = torch.tensor([[ 1, 2, 3],[-1, 1, 4]] , dtype= torch.float)
>>> torch.norm(c, dim=0)
tensor([1.4142, 2.2361, 5.0000])
>>> torch.norm(c, dim=1)
tensor([3.7417, 4.2426])
>>> torch.norm(c, p=1, dim=1)
tensor([6., 6.])
>>> d = torch.arange(8, dtype= torch.float).reshape(2,2,2)
>>> torch.norm(d, dim=(1,2))
tensor([ 3.7417, 11.2250])
>>> torch.norm(d[0, :, :]), torch.norm(d[1, :, :])
(tensor(3.7417), tensor(11.2250))