目录
Spectral Ops
torch.fft(input, signal_ndim, normalized=False) → Tensor
torch.ifft(input, signal_ndim, normalized=False) → Tensor
torch.rfft(input, signal_ndim, normalized=False, onesided=True) → Tensor
torch.irfft(input, signal_ndim, normalized=False, onesided=True, signal_sizes=None) → Tensor
torch.stft(input, n_fft, hop_length=None, win_length=None, window=None, center=True, pad_mode='reflect', normalized=False, onesided=True)[source]
torch.bartlett_window(window_length, periodic=True, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor
torch.blackman_window(window_length, periodic=True, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor
torch.hamming_window(window_length, periodic=True, alpha=0.54, beta=0.46, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor
torch.hann_window(window_length, periodic=True, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor
torch.bincount(input, weights=None, minlength=0) → Tensor
torch.broadcast_tensors(*tensors) → List of Tensors[source]
torch.cartesian_prod(*tensors)[source]
torch.cdist(x1, x2, p=2) → Tensor
torch.combinations(input, r=2, with_replacement=False) → seq
torch.cross(input, other, dim=-1, out=None) → Tensor
torch.diag(input, diagonal=0, out=None) → Tensor
torch.diag_embed(input, offset=0, dim1=-2, dim2=-1) → Tensor
torch.diagflat(input, offset=0) → Tensor
torch.diagonal(input, offset=0, dim1=0, dim2=1) → Tensor
torch.einsum(equation, *operands) → Tensor[source]
torch.flatten(input, start_dim=0, end_dim=-1) → Tensor
torch.flip(input, dims) → Tensor
torch.rot90(input, k, dims) → Tensor
torch.histc(input, bins=100, min=0, max=0, out=None) → Tensor
torch.meshgrid(*tensors, **kwargs)[source]
torch.renorm(input, p, dim, maxnorm, out=None) → Tensor
torch.repeat_interleave()
torch.repeat_interleave(repeats) → Tensor
torch.tensordot(a, b, dims=2)[source]
torch.trace(input) → Tensor
torch.tril(input, diagonal=0, out=None) → Tensor
torch.tril_indices(row, col, offset=0, dtype=torch.long, device='cpu', layout=torch.strided) → Tensor
torch.triu(input, diagonal=0, out=None) → Tensor
torch.triu_indices(row, col, offset=0, dtype=torch.long, device='cpu', layout=torch.strided) → Tensor
BLAS and LAPACK Operations
torch.addbmm(beta=1, input, alpha=1, batch1, batch2, out=None) → Tensor
torch.addmm(beta=1, input, alpha=1, mat1, mat2, out=None) → Tensor
torch.addmv(beta=1, input, alpha=1, mat, vec, out=None) → Tensor
torch.addr(beta=1, input, alpha=1, vec1, vec2, out=None) → Tensor
torch.baddbmm(beta=1, input, alpha=1, batch1, batch2, out=None) → Tensor
torch.bmm(input, mat2, out=None) → Tensor
torch.chain_matmul(*matrices)[source]
torch.cholesky(input, upper=False, out=None) → Tensor
torch.cholesky_inverse(input, upper=False, out=None) → Tensor
torch.cholesky_solve(input, input2, upper=False, out=None) → Tensor
torch.dot(input, tensor) → Tensor
torch.eig(input, eigenvectors=False, out=None) -> (Tensor, Tensor)
torch.ger(input, vec2, out=None) → Tensor
torch.inverse(input, out=None) → Tensor
torch.det(input) → Tensor
torch.logdet(input) → Tensor
torch.slogdet(input) -> (Tensor, Tensor)
torch.lstsq(input, A, out=None) → Tensor
torch.lu(A, pivot=True, get_infos=False, out=None)[source]
torch.lu_solve(input, LU_data, LU_pivots, out=None) → Tensor
torch.matmul(input, other, out=None) → Tensor
torch.matrix_power(input, n) → Tensor
torch.matrix_rank(input, tol=None, bool symmetric=False) → Tensor
torch.mm(input, mat2, out=None) → Tensor
torch.mv(input, vec, out=None) → Tensor
torch.orgqr(input, input2) → Tensor
torch.ormqr(input, input2, input3, left=True, transpose=False) → Tensor
torch.pinverse(input, rcond=1e-15) → Tensor
torch.qr(input, some=True, out=None) -> (Tensor, Tensor)
torch.solve(input, A, out=None) -> (Tensor, Tensor)
torch.svd(input, some=True, compute_uv=True, out=None) -> (Tensor, Tensor, Tensor)
torch.symeig(input, eigenvectors=False, upper=True, out=None) -> (Tensor, Tensor)
torch.trapz()
torch.trapz(y, *, dx=1, dim=-1) → Tensor
torch.
fft
(input, signal_ndim, normalized=False) → TensorComplex-to-complex Discrete Fourier Transform
This method computes the complex-to-complex discrete Fourier transform. Ignoring the batch dimensions, it computes the following expression:
X[ω1,…,ωd]=∑n1=0N1−1⋯∑nd=0Nd−1x[n1,…,nd]e−j 2π∑i=0dωiniNi,X[\omega_1, \dots, \omega_d] = \sum_{n_1=0}^{N_1-1} \dots \sum_{n_d=0}^{N_d-1} x[n_1, \dots, n_d] e^{-j\ 2 \pi \sum_{i=0}^d \frac{\omega_i n_i}{N_i}}, X[ω1,…,ωd]=n1=0∑N1−1⋯nd=0∑Nd−1x[n1,…,nd]e−j 2π∑i=0dNiωini,
where ddd = signal_ndim
is number of dimensions for the signal, and NiN_iNi is the size of signal dimension iii .
This method supports 1D, 2D and 3D complex-to-complex transforms, indicated by signal_ndim
. input
must be a tensor with last dimension of size 2, representing the real and imaginary components of complex numbers, and should have at least signal_ndim + 1
dimensions with optionally arbitrary number of leading batch dimensions. If normalized
is set to True
, this normalizes the result by dividing it with ∏i=1KNi\sqrt{\prod_{i=1}^K N_i}∏i=1KNi
so that the operator is unitary.
Returns the real and the imaginary parts together as one tensor of the same shape of input
.
The inverse of this function is ifft()
.
Note
For CUDA tensors, an LRU cache is used for cuFFT plans to speed up repeatedly running FFT methods on tensors of same geometry with same configuration. See cuFFT plan cache for more details on how to monitor and control the cache.
Warning
For CPU tensors, this method is currently only available with MKL. Use torch.backends.mkl.is_available()
to check if MKL is installed.
Parameters
input (Tensor) – the input tensor of at least signal_ndim
+ 1
dimensions
signal_ndim (int) – the number of dimensions in each signal. signal_ndim
can only be 1, 2 or 3
normalized (bool, optional) – controls whether to return normalized results. Default: False
Returns
A tensor containing the complex-to-complex Fourier transform result
Return type
Tensor
Example:
>>> # unbatched 2D FFT
>>> x = torch.randn(4, 3, 2)
>>> torch.fft(x, 2)
tensor([[[-0.0876, 1.7835],
[-2.0399, -2.9754],
[ 4.4773, -5.0119]],
[[-1.5716, 2.7631],
[-3.8846, 5.2652],
[ 0.2046, -0.7088]],
[[ 1.9938, -0.5901],
[ 6.5637, 6.4556],
[ 2.9865, 4.9318]],
[[ 7.0193, 1.1742],
[-1.3717, -2.1084],
[ 2.0289, 2.9357]]])
>>> # batched 1D FFT
>>> torch.fft(x, 1)
tensor([[[ 1.8385, 1.2827],
[-0.1831, 1.6593],
[ 2.4243, 0.5367]],
[[-0.9176, -1.5543],
[-3.9943, -2.9860],
[ 1.2838, -2.9420]],
[[-0.8854, -0.6860],
[ 2.4450, 0.0808],
[ 1.3076, -0.5768]],
[[-0.1231, 2.7411],
[-0.3075, -1.7295],
[-0.5384, -2.0299]]])
>>> # arbitrary number of batch dimensions, 2D FFT
>>> x = torch.randn(3, 3, 5, 5, 2)
>>> y = torch.fft(x, 2)
>>> y.shape
torch.Size([3, 3, 5, 5, 2])
torch.
ifft
(input, signal_ndim, normalized=False) → TensorComplex-to-complex Inverse Discrete Fourier Transform
This method computes the complex-to-complex inverse discrete Fourier transform. Ignoring the batch dimensions, it computes the following expression:
X[ω1,…,ωd]=1∏i=1dNi∑n1=0N1−1⋯∑nd=0Nd−1x[n1,…,nd]e j 2π∑i=0dωiniNi,X[\omega_1, \dots, \omega_d] = \frac{1}{\prod_{i=1}^d N_i} \sum_{n_1=0}^{N_1-1} \dots \sum_{n_d=0}^{N_d-1} x[n_1, \dots, n_d] e^{\ j\ 2 \pi \sum_{i=0}^d \frac{\omega_i n_i}{N_i}}, X[ω1,…,ωd]=∏i=1dNi1n1=0∑N1−1⋯nd=0∑Nd−1x[n1,…,nd]e j 2π∑i=0dNiωini,
where ddd = signal_ndim
is number of dimensions for the signal, and NiN_iNi is the size of signal dimension iii .
The argument specifications are almost identical with fft()
. However, if normalized
is set to True
, this instead returns the results multiplied by ∏i=1dNi\sqrt{\prod_{i=1}^d N_i}∏i=1dNi
, to become a unitary operator. Therefore, to invert a fft()
, the normalized
argument should be set identically for fft()
.
Returns the real and the imaginary parts together as one tensor of the same shape of input
.
The inverse of this function is fft()
.
Note
For CUDA tensors, an LRU cache is used for cuFFT plans to speed up repeatedly running FFT methods on tensors of same geometry with same configuration. See cuFFT plan cache for more details on how to monitor and control the cache.
Warning
For CPU tensors, this method is currently only available with MKL. Use torch.backends.mkl.is_available()
to check if MKL is installed.
Parameters
input (Tensor) – the input tensor of at least signal_ndim
+ 1
dimensions
signal_ndim (int) – the number of dimensions in each signal. signal_ndim
can only be 1, 2 or 3
normalized (bool, optional) – controls whether to return normalized results. Default: False
Returns
A tensor containing the complex-to-complex inverse Fourier transform result
Return type
Tensor
Example:
>>> x = torch.randn(3, 3, 2)
>>> x
tensor([[[ 1.2766, 1.3680],
[-0.8337, 2.0251],
[ 0.9465, -1.4390]],
[[-0.1890, 1.6010],
[ 1.1034, -1.9230],
[-0.9482, 1.0775]],
[[-0.7708, -0.8176],
[-0.1843, -0.2287],
[-1.9034, -0.2196]]])
>>> y = torch.fft(x, 2)
>>> torch.ifft(y, 2) # recover x
tensor([[[ 1.2766, 1.3680],
[-0.8337, 2.0251],
[ 0.9465, -1.4390]],
[[-0.1890, 1.6010],
[ 1.1034, -1.9230],
[-0.9482, 1.0775]],
[[-0.7708, -0.8176],
[-0.1843, -0.2287],
[-1.9034, -0.2196]]])
torch.
rfft
(input, signal_ndim, normalized=False, onesided=True) → TensorReal-to-complex Discrete Fourier Transform
This method computes the real-to-complex discrete Fourier transform. It is mathematically equivalent with fft()
with differences only in formats of the input and output.
This method supports 1D, 2D and 3D real-to-complex transforms, indicated by signal_ndim
. input
must be a tensor with at least signal_ndim
dimensions with optionally arbitrary number of leading batch dimensions. If normalized
is set to True
, this normalizes the result by dividing it with ∏i=1KNi\sqrt{\prod_{i=1}^K N_i}∏i=1KNi
so that the operator is unitary, where NiN_iNi is the size of signal dimension iii .
The real-to-complex Fourier transform results follow conjugate symmetry:
X[ω1,…,ωd]=X∗[N1−ω1,…,Nd−ωd],X[\omega_1, \dots, \omega_d] = X^*[N_1 - \omega_1, \dots, N_d - \omega_d], X[ω1,…,ωd]=X∗[N1−ω1,…,Nd−ωd],
where the index arithmetic is computed modulus the size of the corresponding dimension, ∗\ ^* ∗ is the conjugate operator, and ddd = signal_ndim
. onesided
flag controls whether to avoid redundancy in the output results. If set to True
(default), the output will not be full complex result of shape (∗,2)(*, 2)(∗,2) , where ∗*∗ is the shape of input
, but instead the last dimension will be halfed as of size ⌊Nd2⌋+1\lfloor \frac{N_d}{2} \rfloor + 1⌊2Nd⌋+1 .
The inverse of this function is irfft()
.
Note
For CUDA tensors, an LRU cache is used for cuFFT plans to speed up repeatedly running FFT methods on tensors of same geometry with same configuration. See cuFFT plan cache for more details on how to monitor and control the cache.
Warning
For CPU tensors, this method is currently only available with MKL. Use torch.backends.mkl.is_available()
to check if MKL is installed.
Parameters
input (Tensor) – the input tensor of at least signal_ndim
dimensions
signal_ndim (int) – the number of dimensions in each signal. signal_ndim
can only be 1, 2 or 3
normalized (bool, optional) – controls whether to return normalized results. Default: False
onesided (bool, optional) – controls whether to return half of results to avoid redundancy. Default: True
Returns
A tensor containing the real-to-complex Fourier transform result
Return type
Tensor
Example:
>>> x = torch.randn(5, 5)
>>> torch.rfft(x, 2).shape
torch.Size([5, 3, 2])
>>> torch.rfft(x, 2, onesided=False).shape
torch.Size([5, 5, 2])
torch.
irfft
(input, signal_ndim, normalized=False, onesided=True, signal_sizes=None) → TensorComplex-to-real Inverse Discrete Fourier Transform
This method computes the complex-to-real inverse discrete Fourier transform. It is mathematically equivalent with ifft()
with differences only in formats of the input and output.
The argument specifications are almost identical with ifft()
. Similar to ifft()
, if normalized
is set to True
, this normalizes the result by multiplying it with ∏i=1KNi\sqrt{\prod_{i=1}^K N_i}∏i=1KNi
so that the operator is unitary, where NiN_iNi is the size of signal dimension iii .
Note
Due to the conjugate symmetry, input
do not need to contain the full complex frequency values. Roughly half of the values will be sufficient, as is the case when input
is given by rfft()
with rfft(signal, onesided=True)
. In such case, set the onesided
argument of this method to True
. Moreover, the original signal shape information can sometimes be lost, optionally set signal_sizes
to be the size of the original signal (without the batch dimensions if in batched mode) to recover it with correct shape.
Therefore, to invert an rfft()
, the normalized
and onesided
arguments should be set identically for irfft()
, and preferrably a signal_sizes
is given to avoid size mismatch. See the example below for a case of size mismatch.
See rfft()
for details on conjugate symmetry.
The inverse of this function is rfft()
.
Warning
Generally speaking, input to this function should contain values following conjugate symmetry. Note that even if onesided
is True
, often symmetry on some part is still needed. When this requirement is not satisfied, the behavior of irfft()
is undefined. Since torch.autograd.gradcheck()
estimates numerical Jacobian with point perturbations, irfft()
will almost certainly fail the check.
Note
For CUDA tensors, an LRU cache is used for cuFFT plans to speed up repeatedly running FFT methods on tensors of same geometry with same configuration. See cuFFT plan cache for more details on how to monitor and control the cache.
Warning
For CPU tensors, this method is currently only available with MKL. Use torch.backends.mkl.is_available()
to check if MKL is installed.
Parameters
input (Tensor) – the input tensor of at least signal_ndim
+ 1
dimensions
signal_ndim (int) – the number of dimensions in each signal. signal_ndim
can only be 1, 2 or 3
normalized (bool, optional) – controls whether to return normalized results. Default: False
onesided (bool, optional) – controls whether input
was halfed to avoid redundancy, e.g., by rfft()
. Default: True
signal_sizes (list or torch.Size
, optional) – the size of the original signal (without batch dimension). Default: None
Returns
A tensor containing the complex-to-real inverse Fourier transform result
Return type
Tensor
Example:
>>> x = torch.randn(4, 4)
>>> torch.rfft(x, 2, onesided=True).shape
torch.Size([4, 3, 2])
>>>
>>> # notice that with onesided=True, output size does not determine the original signal size
>>> x = torch.randn(4, 5)
>>> torch.rfft(x, 2, onesided=True).shape
torch.Size([4, 3, 2])
>>>
>>> # now we use the original shape to recover x
>>> x
tensor([[-0.8992, 0.6117, -1.6091, -0.4155, -0.8346],
[-2.1596, -0.0853, 0.7232, 0.1941, -0.0789],
[-2.0329, 1.1031, 0.6869, -0.5042, 0.9895],
[-0.1884, 0.2858, -1.5831, 0.9917, -0.8356]])
>>> y = torch.rfft(x, 2, onesided=True)
>>> torch.irfft(y, 2, onesided=True, signal_sizes=x.shape) # recover x
tensor([[-0.8992, 0.6117, -1.6091, -0.4155, -0.8346],
[-2.1596, -0.0853, 0.7232, 0.1941, -0.0789],
[-2.0329, 1.1031, 0.6869, -0.5042, 0.9895],
[-0.1884, 0.2858, -1.5831, 0.9917, -0.8356]])
torch.
stft
(input, n_fft, hop_length=None, win_length=None, window=None, center=True, pad_mode='reflect', normalized=False, onesided=True)[source]Short-time Fourier transform (STFT).
Ignoring the optional batch dimension, this method computes the following expression:
X[m,ω]=∑k=0win_length-1window[k] input[m×hop_length+k] exp(−j2π⋅ωkwin_length),X[m, \omega] = \sum_{k = 0}^{\text{win\_length-1}}% \text{window}[k]\ \text{input}[m \times \text{hop\_length} + k]\ % \exp\left(- j \frac{2 \pi \cdot \omega k}{\text{win\_length}}\right), X[m,ω]=k=0∑win_length-1window[k] input[m×hop_length+k] exp(−jwin_length2π⋅ωk),
where mmm is the index of the sliding window, and ω\omegaω is the frequency that 0≤ωTrue
,
input
must be either a 1-D time sequence or a 2-D batch of time sequences.
If hop_length
is None
(default), it is treated as equal to floor(n_fft / 4)
.
If win_length
is None
(default), it is treated as equal to n_fft
.
window
can be a 1-D tensor of size win_length
, e.g., from torch.hann_window()
. If window
is None
(default), it is treated as if having 111 everywhere in the window. If win_lengthn_fft
before being applied.
If center
is True
(default), input
will be padded on both sides so that the ttt -th frame is centered at time t×hop_lengtht \times \text{hop\_length}t×hop_length . Otherwise, the ttt -th frame begins at time t×hop_lengtht \times \text{hop\_length}t×hop_length .
pad_mode
determines the padding method used on input
when center
is True
. See torch.nn.functional.pad()
for all available options. Default is "reflect"
.
If onesided
is True
(default), only values for ω\omegaω in [0,1,2,…,⌊n_fft2⌋+1]\left[0, 1, 2, \dots, \left\lfloor \frac{\text{n\_fft}}{2} \right\rfloor + 1\right][0,1,2,…,⌊2n_fft⌋+1] are returned because the real-to-complex Fourier transform satisfies the conjugate symmetry, i.e., X[m,ω]=X[m,n_fft−ω]∗X[m, \omega] = X[m, \text{n\_fft} - \omega]^*X[m,ω]=X[m,n_fft−ω]∗ .
If normalized
is True
(default is False
), the function returns the normalized STFT results, i.e., multiplied by (frame_length)−0.5(\text{frame\_length})^{-0.5}(frame_length)−0.5 .
Returns the real and the imaginary parts together as one tensor of size (∗×N×T×2)(* \times N \times T \times 2)(∗×N×T×2) , where ∗*∗ is the optional batch size of input
, NNN is the number of frequencies where STFT is applied, TTT is the total number of frames used, and each pair in the last dimension represents a complex number as the real part and the imaginary part.
Warning
This function changed signature at version 0.4.1. Calling with the previous signature may cause error or return incorrect result.
Parameters
input (Tensor) – the input tensor
n_fft (int) – size of Fourier transform
hop_length (int, optional) – the distance between neighboring sliding window frames. Default: None
(treated as equal to floor(n_fft / 4)
)
win_length (int, optional) – the size of window frame and STFT filter. Default: None
(treated as equal to n_fft
)
window (Tensor, optional) – the optional window function. Default: None
(treated as window of all 111 s)
center (bool, optional) – whether to pad input
on both sides so that the ttt -th frame is centered at time t×hop_lengtht \times \text{hop\_length}t×hop_length . Default: True
pad_mode (string, optional) – controls the padding method used when center
is True
. Default: "reflect"
normalized (bool, optional) – controls whether to return the normalized STFT results Default: False
onesided (bool, optional) – controls whether to return half of results to avoid redundancy Default: True
Returns
A tensor containing the STFT result with shape described above
Return type
Tensor
torch.
bartlett_window
(window_length, periodic=True, dtype=None, layout=torch.strided, device=None, requires_grad=False) → TensorBartlett window function.
w[n]=1−∣2nN−1−1∣={2nN−1if 0≤n≤N−122−2nN−1if N−12 where NNN is the full window size. The input Note If Parameters window_length (int) – the size of returned window periodic (bool, optional) – If True, returns a window to be used as periodic function. If False, return a symmetric window. dtype ( layout ( device ( requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: Returns A 1-D tensor of size (window_length,)(\text{window\_length},)(window_length,) containing the window Return type Tensor Blackman window function. w[n]=0.42−0.5cos(2πnN−1)+0.08cos(4πnN−1)w[n] = 0.42 - 0.5 \cos \left( \frac{2 \pi n}{N - 1} \right) + 0.08 \cos \left( \frac{4 \pi n}{N - 1} \right) w[n]=0.42−0.5cos(N−12πn)+0.08cos(N−14πn) where NNN is the full window size. The input Note If Parameters window_length (int) – the size of returned window periodic (bool, optional) – If True, returns a window to be used as periodic function. If False, return a symmetric window. dtype ( layout ( device ( requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: Returns A 1-D tensor of size (window_length,)(\text{window\_length},)(window_length,) containing the window Return type Tensor Hamming window function. w[n]=α−β cos(2πnN−1),w[n] = \alpha - \beta\ \cos \left( \frac{2 \pi n}{N - 1} \right), w[n]=α−β cos(N−12πn), where NNN is the full window size. The input Note If Note This is a generalized version of Parameters window_length (int) – the size of returned window periodic (bool, optional) – If True, returns a window to be used as periodic function. If False, return a symmetric window. alpha (float, optional) – The coefficient α\alphaα in the equation above beta (float, optional) – The coefficient β\betaβ in the equation above dtype ( layout ( device ( requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: Returns A 1-D tensor of size (window_length,)(\text{window\_length},)(window_length,) containing the window Return type Tensor Hann window function. w[n]=12 [1−cos(2πnN−1)]=sin2(πnN−1),w[n] = \frac{1}{2}\ \left[1 - \cos \left( \frac{2 \pi n}{N - 1} \right)\right] = \sin^2 \left( \frac{\pi n}{N - 1} \right), w[n]=21 [1−cos(N−12πn)]=sin2(N−1πn), where NNN is the full window size. The input Note If Parameters window_length (int) – the size of returned window periodic (bool, optional) – If True, returns a window to be used as periodic function. If False, return a symmetric window. dtype ( layout ( device ( requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: Returns A 1-D tensor of size (window_length,)(\text{window\_length},)(window_length,) containing the window Return type Tensor Other Operations Count the frequency of each value in an array of non-negative ints. The number of bins (size 1) is one larger than the largest value in Note When using the CUDA backend, this operation may induce nondeterministic behaviour that is not easily switched off. Please see the notes on Reproducibility for background. Parameters input (Tensor) – 1-d int tensor weights (Tensor) – optional, weight for each value in the input tensor. Should be of same size as input tensor. minlength (int) – optional, minimum number of bins. Should be non-negative. Returns a tensor of shape Return type output (Tensor) Example: Broadcasts the given tensors according to Broadcasting semantics. Parameters *tensors – any number of tensors of the same type Warning More than one element of a broadcasted tensor may refer to a single memory location. As a result, in-place operations (especially ones that are vectorized) may result in incorrect behavior. If you need to write to the tensors, please clone them first. Example: Do cartesian product of the given sequence of tensors. The behavior is similar to python’s itertools.product. Parameters *tensors – any number of 1 dimensional tensors. Returns A tensor equivalent to converting all the input tensors into lists, do itertools.product on these lists, and finally convert the resulting list into tensor. Return type Tensor Example: Computes the p-norm distance between each pair of the two collections of row vectors. If x1 has shape P×MP \times MP×M and x2 has shape R×MR \times MR×M then the output will have shape P×RP \times RP×R . This function is equivalent to scipy.spatial.distance.cdist(input,’minkowski’, p=p) if p∈(0,∞)p \in (0, \infty)p∈(0,∞) . When p=0p = 0p=0 it is equivalent to scipy.spatial.distance.cdist(input, ‘hamming’) * M. When p=∞p = \inftyp=∞ , the closest scipy function is scipy.spatial.distance.cdist(xn, lambda x, y: np.abs(x - y).max()). Parameters x1 (Tensor) – input tensor of shape P×MP \times MP×M . x2 (Tensor) – input tensor of shape R×MR \times MR×M . p – p value for the p-norm distance to calculate between each vector pair ∈[0,∞]\in [0, \infty]∈[0,∞] . Example: Compute combinations of length rrr of the given tensor. The behavior is similar to python’s itertools.combinations when with_replacement is set to False, and itertools.combinations_with_replacement when with_replacement is set to True. Parameters input (Tensor) – 1D vector. r (int, optional) – number of elements to combine with_replacement (boolean, optional) – whether to allow duplication in combination Returns A tensor equivalent to converting all the input tensors into lists, do itertools.combinations or itertools.combinations_with_replacement on these lists, and finally convert the resulting list into tensor. Return type Tensor Example: Returns the cross product of vectors in dimension If Parameters input (Tensor) – the input tensor. other (Tensor) – the second input tensor dim (int, optional) – the dimension to take the cross-product in. out (Tensor, optional) – the output tensor. Example: If If The argument If If If Parameters input (Tensor) – the input tensor. diagonal (int, optional) – the diagonal to consider out (Tensor, optional) – the output tensor. See also Examples: Get the square matrix where the input vector is the diagonal: Get the k-th diagonal of a given matrix: Creates a tensor whose diagonals of certain 2D planes (specified by The argument If If If The size of the new matrix will be calculated to make the specified diagonal of the size of the last input dimension. Note that for Applying Parameters input (Tensor) – the input tensor. Must be at least 1-dimensional. offset (int, optional) – which diagonal to consider. Default: 0 (main diagonal). dim1 (int, optional) – first dimension with respect to which to take diagonal. Default: -2. dim2 (int, optional) – second dimension with respect to which to take diagonal. Default: -1. Example: If If The argument If If If Parameters input (Tensor) – the input tensor. offset (int, optional) – the diagonal to consider. Default: 0 (main diagonal). Examples: Returns a partial view of The argument If If If Applying Parameters input (Tensor) – the input tensor. Must be at least 2-dimensional. offset (int, optional) – which diagonal to consider. Default: 0 (main diagonal). dim1 (int, optional) – first dimension with respect to which to take diagonal. Default: 0. dim2 (int, optional) – second dimension with respect to which to take diagonal. Default: 1. Note To take a batch diagonal, pass in dim1=-2, dim2=-1. Examples: This function provides a way of computing multilinear expressions (i.e. sums of products) using the Einstein summation convention. Parameters equation (string) – The equation is given in terms of lower case letters (indices) to be associated with each dimension of the operands and result. The left hand side lists the operands dimensions, separated by commas. There should be one index letter per tensor dimension. The right hand side follows after -> and gives the indices for the output. If the -> and right hand side are omitted, it implicitly defined as the alphabetically sorted list of all indices appearing exactly once in the left hand side. The indices not apprearing in the output are summed over after multiplying the operands entries. If an index appears several times for the same operand, a diagonal is taken. Ellipses … represent a fixed number of dimensions. If the right hand side is inferred, the ellipsis dimensions are at the beginning of the output. operands (list of Tensors) – The operands to compute the Einstein sum of. Examples: Flattens a contiguous range of dims in a tensor. Parameters input (Tensor) – the input tensor. start_dim (int) – the first dim to flatten end_dim (int) – the last dim to flatten Example: Reverse the order of a n-D tensor along given axis in dims. Parameters input (Tensor) – the input tensor. dims (a list or tuple) – axis to flip on Example: Rotate a n-D tensor by 90 degrees in the plane specified by dims axis. Rotation direction is from the first towards the second axis if k > 0, and from the second towards the first for k < 0. Parameters input (Tensor) – the input tensor. k (int) – number of times to rotate dims (a list or tuple) – axis to rotate Example: Computes the histogram of a tensor. The elements are sorted into equal width bins between Parameters input (Tensor) – the input tensor. bins (int) – number of histogram bins min (int) – lower end of the range (inclusive) max (int) – upper end of the range (inclusive) out (Tensor, optional) – the output tensor. Returns Histogram represented as a tensor Return type Tensor Example: Take NNN tensors, each of which can be either scalar or 1-dimensional vector, and create NNN N-dimensional grids, where the iii th grid is defined by expanding the iii th input over dimensions defined by other inputs. Args: tensors (list of Tensor): list of scalars or 1 dimensional tensors. Scalars will be treated as tensors of size (1,)(1,)(1,) automatically Returns: seq (sequence of Tensors): If the input has kkk tensors of size (N1,),(N2,),…,(Nk,)(N_1,), (N_2,), \ldots , (N_k,)(N1,),(N2,),…,(Nk,) , then the output would also have kkk tensors, where all tensors are of size (N1,N2,…,Nk)(N_1, N_2, \ldots , N_k)(N1,N2,…,Nk) . Example: Returns a tensor where each sub-tensor of Note If the norm of a row is lower than maxnorm, the row is unchanged Parameters input (Tensor) – the input tensor. p (float) – the power for the norm computation dim (int) – the dimension to slice over to get the sub-tensors maxnorm (float) – the maximum norm to keep each sub-tensor under out (Tensor, optional) – the output tensor. Example: Repeat elements of a tensor. Warning This is different from Parameters input (Tensor) – the input tensor. repeats (Tensor or int) – The number of repetitions for each element. repeats is broadcasted to fit the shape of the given axis. dim (int, optional) – The dimension along which to repeat values. By default, use the flattened input array, and return a flat output array. Returns Repeated tensor which has the same shape as input, except along the given axis. Return type Tensor Example: If the repeats is tensor([n1, n2, n3, …]), then the output will be tensor([0, 0, …, 1, 1, …, 2, 2, …, …]) where 0 appears n1 times, 1 appears n2 times, 2 appears n3 times, etc. Roll the tensor along the given dimension(s). Elements that are shifted beyond the last position are re-introduced at the first position. If a dimension is not specified, the tensor will be flattened before rolling and then restored to the original shape. Parameters input (Tensor) – the input tensor. shifts (int or tuple of python:ints) – The number of places by which the elements of the tensor are shifted. If shifts is a tuple, dims must be a tuple of the same size, and each dimension will be rolled by the corresponding value dims (int or tuple of python:ints) – Axis along which to roll Example: Returns a contraction of a and b over multiple dimensions. Parameters a (Tensor) – Left tensor to contract b (Tensor) – Right tensor to contract dims (int or tuple of two lists of python:integers) – number of dimensions to contract or explicit lists of dimensions for When called with an integer argument ri0,...,im−d,id,...,in=∑k0,...,kd−1ai0,...,im−d,k0,...,kd−1×bk0,...,kd−1,id,...,in.r_{i_0,...,i_{m-d}, i_d,...,i_n} = \sum_{k_0,...,k_{d-1}} a_{i_0,...,i_{m-d},k_0,...,k_{d-1}} \times b_{k_0,...,k_{d-1}, i_d,...,i_n}. ri0,...,im−d,id,...,in=k0,...,kd−1∑ai0,...,im−d,k0,...,kd−1×bk0,...,kd−1,id,...,in. When called with Examples: Returns the sum of the elements of the diagonal of the input 2-D matrix. Example: Returns the lower triangular part of the matrix (2-D tensor) or batch of matrices The lower triangular part of the matrix is defined as the elements on and below the diagonal. The argument Parameters input (Tensor) – the input tensor. diagonal (int, optional) – the diagonal to consider out (Tensor, optional) – the output tensor. Example: Returns the indices of the lower triangular part of a The lower triangular part of the matrix is defined as the elements on and below the diagonal. The argument NOTE: when running on ‘cuda’, row * col must be less than 2592^{59}259 to prevent overflow during calculation. Parameters row ( col ( offset ( dtype ( device ( layout ( Example:: Returns the upper triangular part of a matrix (2-D tensor) or batch of matrices The upper triangular part of the matrix is defined as the elements on and above the diagonal. The argument Parameters input (Tensor) – the input tensor. diagonal (int, optional) – the diagonal to consider out (Tensor, optional) – the output tensor. Example: Returns the indices of the upper triangular part of a The upper triangular part of the matrix is defined as the elements on and above the diagonal. The argument NOTE: when running on ‘cuda’, row * col must be less than 2592^{59}259 to prevent overflow during calculation. Parameters row ( col ( offset ( dtype ( device ( layout ( Example:: Performs a batch matrix-matrix product of matrices stored in If out=β input+α (∑i=0b−1batch1i@batch2i)out = \beta\ \text{input} + \alpha\ (\sum_{i=0}^{b-1} \text{batch1}_i \mathbin{@} \text{batch2}_i) out=β input+α (i=0∑b−1batch1i@batch2i) For inputs of type FloatTensor or DoubleTensor, arguments Parameters beta (Number, optional) – multiplier for input (Tensor) – matrix to be added alpha (Number, optional) – multiplier for batch1 @ batch2 (α\alphaα ) batch1 (Tensor) – the first batch of matrices to be multiplied batch2 (Tensor) – the second batch of matrices to be multiplied out (Tensor, optional) – the output tensor. Example: Performs a matrix multiplication of the matrices If out=β input+α (mat1i@mat2i)\text{out} = \beta\ \text{input} + \alpha\ (\text{mat1}_i \mathbin{@} \text{mat2}_i) out=β input+α (mat1i@mat2i) For inputs of type FloatTensor or DoubleTensor, arguments Parameters beta (Number, optional) – multiplier for input (Tensor) – matrix to be added alpha (Number, optional) – multiplier for mat1@mat2mat1 @ mat2mat1@mat2 (α\alphaα ) mat1 (Tensor) – the first matrix to be multiplied mat2 (Tensor) – the second matrix to be multiplied out (Tensor, optional) – the output tensor. Example: Performs a matrix-vector product of the matrix If out=β input+α (mat@vec)\text{out} = \beta\ \text{input} + \alpha\ (\text{mat} \mathbin{@} \text{vec}) out=β input+α (mat@vec) For inputs of type FloatTensor or DoubleTensor, arguments Parameters beta (Number, optional) – multiplier for input (Tensor) – vector to be added alpha (Number, optional) – multiplier for mat@vecmat @ vecmat@vec (α\alphaα ) mat (Tensor) – matrix to be multiplied vec (Tensor) – vector to be multiplied out (Tensor, optional) – the output tensor. Example: Performs the outer-product of vectors Optional values out=β input+α (vec1⊗vec2)\text{out} = \beta\ \text{input} + \alpha\ (\text{vec1} \otimes \text{vec2}) out=β input+α (vec1⊗vec2) If For inputs of type FloatTensor or DoubleTensor, arguments Parameters beta (Number, optional) – multiplier for input (Tensor) – matrix to be added alpha (Number, optional) – multiplier for vec1⊗vec2\text{vec1} \otimes \text{vec2}vec1⊗vec2 (α\alphaα ) vec1 (Tensor) – the first vector of the outer product vec2 (Tensor) – the second vector of the outer product out (Tensor, optional) – the output tensor. Example: Performs a batch matrix-matrix product of matrices in If outi=β inputi+α (batch1i@batch2i)\text{out}_i = \beta\ \text{input}_i + \alpha\ (\text{batch1}_i \mathbin{@} \text{batch2}_i) outi=β inputi+α (batch1i@batch2i) For inputs of type FloatTensor or DoubleTensor, arguments Parameters beta (Number, optional) – multiplier for input (Tensor) – the tensor to be added alpha (Number, optional) – multiplier for batch1@batch2\text{batch1} \mathbin{@} \text{batch2}batch1@batch2 (α\alphaα ) batch1 (Tensor) – the first batch of matrices to be multiplied batch2 (Tensor) – the second batch of matrices to be multiplied out (Tensor, optional) – the output tensor. Example: Performs a batch matrix-matrix product of matrices stored in If outi=inputi@mat2i\text{out}_i = \text{input}_i \mathbin{@} \text{mat2}_i outi=inputi@mat2i Note This function does not broadcast. For broadcasting matrix products, see Parameters input (Tensor) – the first batch of matrices to be multiplied mat2 (Tensor) – the second batch of matrices to be multiplied out (Tensor, optional) – the output tensor. Example: Returns the matrix product of the NNN 2-D tensors. This product is efficiently computed using the matrix chain order algorithm which selects the order in which incurs the lowest cost in terms of arithmetic operations ([CLRS]). Note that since this is a function to compute the product, NNN needs to be greater than or equal to 2; if equal to 2 then a trivial matrix-matrix product is returned. If NNN is 1, then this is a no-op - the original matrix is returned as is. Parameters matrices (Tensors...) – a sequence of 2 or more 2-D tensors whose product is to be determined. Returns if the ithi^{th}ith tensor was of dimensions pi×pi+1p_{i} \times p_{i + 1}pi×pi+1 , then the product would be of dimensions p1×pN+1p_{1} \times p_{N + 1}p1×pN+1 . Return type Tensor Example: Computes the Cholesky decomposition of a symmetric positive-definite matrix AAA or for batches of symmetric positive-definite matrices. If A=UTUA = U^TUA=UTU If A=LLTA = LL^TA=LLT If Parameters input (Tensor) – the input tensor AAA of size (∗,n,n)(*, n, n)(∗,n,n) where * is zero or more batch dimensions consisting of symmetric positive-definite matrices. upper (bool, optional) – flag that indicates whether to return a upper or lower triangular matrix. Default: out (Tensor, optional) – the output matrix Example: Computes the inverse of a symmetric positive-definite matrix AAA using its Cholesky factor uuu : returns matrix If inv=(uuT)−1inv = (uu^{{T}})^{{-1}} inv=(uuT)−1 If inv=(uTu)−1inv = (u^T u)^{{-1}} inv=(uTu)−1 Parameters input (Tensor) – the input 2-D tensor uuu , a upper or lower triangular Cholesky factor upper (bool, optional) – whether to return a lower (default) or upper triangular matrix out (Tensor, optional) – the output tensor for inv Example: Solves a linear system of equations with a positive semidefinite matrix to be inverted given its Cholesky factor matrix uuu . If c=(uuT)−1bc = (u u^T)^{{-1}} b c=(uuT)−1b If c=(uTu)−1bc = (u^T u)^{{-1}} b c=(uTu)−1b torch.cholesky_solve(b, u) can take in 2D inputs b, u or inputs that are batches of 2D matrices. If the inputs are batches, then returns batched outputs c Note The Parameters input (Tensor) – input matrix bbb of size (∗,m,k)(*, m, k)(∗,m,k) , where ∗*∗ is zero or more batch dimensions input2 (Tensor) – input matrix uuu of size (∗,m,m)(*, m, m)(∗,m,m) , where ∗*∗ is zero of more batch dimensions composed of upper or lower triangular Cholesky factor upper (bool, optional) – whether to consider the Cholesky factor as a lower or upper triangular matrix. Default: out (Tensor, optional) – the output tensor for c Example: Computes the dot product (inner product) of two tensors. Note This function does not broadcast. Example: Computes the eigenvalues and eigenvectors of a real square matrix. Note Since eigenvalues and eigenvectors might be complex, backward pass is supported only for Parameters input (Tensor) – the square matrix of shape (n×n)(n \times n)(n×n) for which the eigenvalues and eigenvectors will be computed eigenvectors (bool) – out (tuple, optional) – the output tensors Returns A namedtuple (eigenvalues, eigenvectors) containing eigenvalues (Tensor): Shape (n×2)(n \times 2)(n×2) . Each row is an eigenvalue of eigenvectors (Tensor): If Return type (Tensor, Tensor) Computes the solution to the least squares and least norm problems for a full rank matrix AAA of size (m×n)(m \times n)(m×n) and a matrix BBB of size (m×k)(m \times k)(m×k) . For more information regarding Warning This is a low-level function for calling LAPACK directly. This function returns a namedtuple (a, tau) as defined in LAPACK documentation for geqrf . You’ll generally want to use Computes a QR decomposition of Rather, this directly calls the underlying LAPACK function ?geqrf which produces a sequence of ‘elementary reflectors’. See LAPACK documentation for geqrf for further details. Parameters input (Tensor) – the input matrix out (tuple, optional) – the output tuple of (Tensor, Tensor) Outer product of Note This function does not broadcast. Parameters input (Tensor) – 1-D input vector vec2 (Tensor) – 1-D input vector out (Tensor, optional) – optional output matrix Example: Takes the inverse of the square matrix Note Irrespective of the original strides, the returned tensors will be transposed, i.e. with strides like input.contiguous().transpose(-2, -1).stride() Parameters input (Tensor) – the input tensor of size (∗,n,n)(*, n, n)(∗,n,n) where * is zero or more batch dimensions out (Tensor, optional) – the output tensor. Example: Calculates determinant of a square matrix or batches of square matrices. Note Backward through Parameters input (Tensor) – the input tensor of size (*, n, n) where * is zero or more batch dimensions. Example: Calculates log determinant of a square matrix or batches of square matrices. Note Result is Note Backward through Parameters input (Tensor) – the input tensor of size (*, n, n) where * is zero or more batch dimensions. Example: Calculates the sign and log absolute value of the determinant(s) of a square matrix or batches of square matrices. Note If Note Backward through Parameters input (Tensor) – the input tensor of size (*, n, n) where * is zero or more batch dimensions. Returns A namedtuple (sign, logabsdet) containing the sign of the determinant, and the log value of the absolute determinant. Example: Computes the solution to the least squares and least norm problems for a full rank matrix AAA of size (m×n)(m \times n)(m×n) and a matrix BBB of size (m×k)(m \times k)(m×k) . If m≥nm \geq nm≥n , minX∥AX−B∥2.\begin{array}{ll} \min_X & \|AX-B\|_2. \end{array}minX∥AX−B∥2. If m minX∥X∥2subject toAX=B.\begin{array}{ll} \min_X & \|X\|_2 & \text{subject to} & AX = B. \end{array}minX∥X∥2subject toAX=B. Returned tensor XXX has shape (max(m,n)×k)(\max(m, n) \times k)(max(m,n)×k) . The first nnn rows of XXX contains the solution. If m≥nm \geq nm≥n , the residual sum of squares for the solution in each column is given by the sum of squares of elements in the remaining m−nm - nm−n rows of that column. Note The case when m Parameters input (Tensor) – the matrix BBB A (Tensor) – the mmm by nnn matrix AAA out (tuple, optional) – the optional destination tensor Returns A namedtuple (solution, QR) containing: solution (Tensor): the least squares solution QR (Tensor): the details of the QR factorization Return type (Tensor, Tensor) Note The returned matrices will always be transposed, irrespective of the strides of the input matrices. That is, they will have stride (1, m) instead of (m, 1). Example: Computes the LU factorization of a square matrix or batches of square matrices Note The pivots returned by the function are 1-indexed. If Note LU factorization with Note This function does not check if the factorization was successful or not if Parameters A (Tensor) – the tensor to factor of size (∗,m,m)(*, m, m)(∗,m,m) pivot (bool, optional) – controls whether pivoting is done. Default: get_infos (bool, optional) – if set to out (tuple, optional) – optional output tuple. If Returns A tuple of tensors containing factorization (Tensor): the factorization of size (∗,m,m)(*, m, m)(∗,m,m) pivots (IntTensor): the pivots of size (∗,m)(*, m)(∗,m) infos (IntTensor, optional): if Return type (Tensor, IntTensor, IntTensor (optional)) Example: Returns the LU solve of the linear system Ax=bAx = bAx=b using the partially pivoted LU factorization of A from Parameters b (Tensor) – the RHS tensor of size (∗,m,k)(*, m, k)(∗,m,k) , where ∗*∗ is zero or more batch dimensions. LU_data (Tensor) – the pivoted LU factorization of A from LU_pivots (IntTensor) – the pivots of the LU factorization from out (Tensor, optional) – the output tensor. Example: Unpacks the data and pivots from a LU factorization of a tensor. Returns a tuple of tensors as Parameters LU_data (Tensor) – the packed LU factorization data LU_pivots (Tensor) – the packed LU factorization pivots unpack_data (bool) – flag indicating if the data should be unpacked unpack_pivots (bool) – flag indicating if the pivots should be unpacked Example: Matrix product of two tensors. The behavior depends on the dimensionality of the tensors as follows: If both tensors are 1-dimensional, the dot product (scalar) is returned. If both arguments are 2-dimensional, the matrix-matrix product is returned. If the first argument is 1-dimensional and the second argument is 2-dimensional, a 1 is prepended to its dimension for the purpose of the matrix multiply. After the matrix multiply, the prepended dimension is removed. If the first argument is 2-dimensional and the second argument is 1-dimensional, the matrix-vector product is returned. If both arguments are at least 1-dimensional and at least one argument is N-dimensional (where N > 2), then a batched matrix multiply is returned. If the first argument is 1-dimensional, a 1 is prepended to its dimension for the purpose of the batched matrix multiply and removed after. If the second argument is 1-dimensional, a 1 is appended to its dimension for the purpose of the batched matrix multiple and removed after. The non-matrix (i.e. batch) dimensions are broadcasted (and thus must be broadcastable). For example, if Note The 1-dimensional dot product version of this function does not support an Parameters input (Tensor) – the first tensor to be multiplied other (Tensor) – the second tensor to be multiplied out (Tensor, optional) – the output tensor. Example: Returns the matrix raised to the power If Parameters input (Tensor) – the input tensor. n (int) – the power to raise the matrix to Example: Returns the numerical rank of a 2-D tensor. The method to compute the matrix rank is done using SVD by default. If Parameters input (Tensor) – the input 2-D tensor tol (float, optional) – the tolerance value. Default: symmetric (bool, optional) – indicates whether Example: Performs a matrix multiplication of the matrices If Note This function does not broadcast. For broadcasting matrix products, see Parameters input (Tensor) – the first matrix to be multiplied mat2 (Tensor) – the second matrix to be multiplied out (Tensor, optional) – the output tensor. Example: Performs a matrix-vector product of the matrix If Note This function does not broadcast. Parameters input (Tensor) – matrix to be multiplied vec (Tensor) – vector to be multiplied out (Tensor, optional) – the output tensor. Example: Computes the orthogonal matrix Q of a QR factorization, from the (input, input2) tuple returned by This directly calls the underlying LAPACK function ?orgqr. See LAPACK documentation for orgqr for further details. Parameters input (Tensor) – the a from input2 (Tensor) – the tau from Multiplies mat (given by This directly calls the underlying LAPACK function ?ormqr. See LAPACK documentation for ormqr for further details. Parameters input (Tensor) – the a from input2 (Tensor) – the tau from input3 (Tensor) – the matrix to be multiplied. Calculates the pseudo-inverse (also known as the Moore-Penrose inverse) of a 2D tensor. Please look at Moore-Penrose inverse for more details Note This method is implemented using the Singular Value Decomposition. Note The pseudo-inverse is not necessarily a continuous function in the elements of the matrix [1]. Therefore, derivatives are not always existent, and exist for a constant rank only [2]. However, this method is backprop-able due to the implementation by using SVD results, and could be unstable. Double-backward will also be unstable due to the usage of SVD internally. See Parameters input (Tensor) – The input 2D tensor of dimensions m×nm \times nm×n rcond (float) – A floating point value to determine the cutoff for small singular values. Default: 1e-15 Returns The pseudo-inverse of Example: Computes the QR decomposition of a matrix or a batch of matrices If Note precision may be lost if the magnitudes of the elements of Note While it should always give you a valid decomposition, it may not give you the same one across platforms - it will depend on your LAPACK implementation. Parameters input (Tensor) – the input tensor of size (∗,m,n)(*, m, n)(∗,m,n) where * is zero or more batch dimensions consisting of matrices of dimension m×nm \times nm×n . some (bool, optional) – Set to out (tuple, optional) – tuple of Q and R tensors satisfying Example: This function returns the solution to the system of linear equations represented by AX=BAX = BAX=B and the LU factorization of A, in order as a namedtuple solution, LU. LU contains L and U factors for LU factorization of A. torch.solve(B, A) can take in 2D inputs B, A or inputs that are batches of 2D matrices. If the inputs are batches, then returns batched outputs solution, LU. Note Irrespective of the original strides, the returned matrices solution and LU will be transposed, i.e. with strides like B.contiguous().transpose(-1, -2).stride() and A.contiguous().transpose(-1, -2).stride() respectively. Parameters input (Tensor) – input matrix BBB of size (∗,m,k)(*, m, k)(∗,m,k) , where ∗*∗ is zero or more batch dimensions. A (Tensor) – input square matrix of size (∗,m,m)(*, m, m)(∗,m,m) , where ∗*∗ is zero or more batch dimensions. out ((Tensor, Tensor), optional) – optional output tuple. Example: This function returns a namedtuple If If Note The implementation of SVD on CPU uses the LAPACK routine ?gesdd (a divide-and-conquer algorithm) instead of ?gesvd for speed. Analogously, the SVD on GPU uses the MAGMA routine gesdd as well. Note Irrespective of the original strides, the returned matrix U will be transposed, i.e. with strides Note Extra care needs to be taken when backward through U and V outputs. Such operation is really only stable when Note When Note When Parameters input (Tensor) – the input tensor of size (∗,m,n)(*, m, n)(∗,m,n) where * is zero or more batch dimensions consisting of m×nm \times nm×n matrices. some (bool, optional) – controls the shape of returned U and V compute_uv (bool, optional) – option whether to compute U and V or not out (tuple, optional) – the output tuple of tensors Example: This function returns eigenvalues and eigenvectors of a real symmetric matrix This function calculates all eigenvalues (and vectors) of The boolean argument If it is Since the input matrix If Note Irrespective of the original strides, the returned matrix V will be transposed, i.e. with strides V.contiguous().transpose(-1, -2).stride(). Note Extra care needs to be taken when backward through outputs. Such operation is really only stable when all eigenvalues are distinct. Otherwise, Parameters input (Tensor) – the input tensor of size (∗,n,n)(*, n, n)(∗,n,n) where * is zero or more batch dimensions consisting of symmetric matrices. eigenvectors (boolean, optional) – controls whether eigenvectors have to be computed upper (boolean, optional) – controls whether to consider upper-triangular or lower-triangular region out (tuple, optional) – the output tuple of (Tensor, Tensor) Returns A namedtuple (eigenvalues, eigenvectors) containing eigenvalues (Tensor): Shape (∗,m)(*, m)(∗,m) . The eigenvalues in ascending order. eigenvectors (Tensor): Shape (∗,m,m)(*, m, m)(∗,m,m) . If Return type (Tensor, Tensor) Examples: Estimate ∫y dx\int y\,dx∫ydx along dim, using the trapezoid rule. Parameters y (Tensor) – The values of the function to integrate x (Tensor) – The points at which the function y is sampled. If x is not in ascending order, intervals on which it is decreasing contribute negatively to the estimated integral (i.e., the convention ∫abf=−∫baf\int_a^b f = -\int_b^a f∫abf=−∫baf is followed). dim (int) – The dimension along which to integrate. By default, use the last dimension. Returns A Tensor with the same shape as the input, except with dim removed. Each element of the returned tensor represents the estimated integral ∫y dx\int y\,dx∫ydx along dim. Example: As above, but the sample points are spaced uniformly at a distance of dx. Parameters y (Tensor) – The values of the function to integrate dx (float) – The distance between points at which y is sampled. dim (int) – The dimension along which to integrate. By default, use the last dimension. Returns A Tensor with the same shape as the input, except with dim removed. Each element of the returned tensor represents the estimated integral ∫y dx\int y\,dx∫ydx along dim. Solves a system of equations with a triangular coefficient matrix AAA and multiple right-hand sides bbb . In particular, solves AX=bAX = bAX=b and assumes AAA is upper-triangular with the default keyword arguments. torch.triangular_solve(b, A) can take in 2D inputs b, A or inputs that are batches of 2D matrices. If the inputs are batches, then returns batched outputs X Note The Parameters input (Tensor) – multiple right-hand sides of size (∗,m,k)(*, m, k)(∗,m,k) where ∗*∗ is zero of more batch dimensions (bbb ) A (Tensor) – the input triangular coefficient matrix of size (∗,m,m)(*, m, m)(∗,m,m) where ∗*∗ is zero or more batch dimensions upper (bool, optional) – whether to solve the upper-triangular system of equations (default) or the lower-triangular system of equations. Default: transpose (bool, optional) – whether AAA should be transposed before being sent into the solver. Default: unitriangular (bool, optional) – whether AAA is unit triangular. If True, the diagonal elements of AAA are assumed to be 1 and not referenced from AAA . Default: Returns A namedtuple (solution, cloned_coefficient) where cloned_coefficient is a clone of AAA and solution is the solution XXX to AX=bAX = bAX=b (or whatever variant of the system of equations, depending on the keyword arguments.) Examples:window_length
is a positive integer controlling the returned window size. periodic
flag determines whether the returned window trims off the last duplicate value from the symmetric window and is ready to be used as a periodic window with functions like torch.stft()
. Therefore, if periodic
is true, the NNN in above formula is in fact window_length+1\text{window\_length} + 1window_length+1 . Also, we always have torch.bartlett_window(L, periodic=True)
equal to torch.bartlett_window(L + 1, periodic=False)[:-1])
.window_length
=1=1=1 , the returned window contains a single value 1.
torch.dtype
, optional) – the desired data type of returned tensor. Default: if None
, uses a global default (see torch.set_default_tensor_type()
). Only floating point types are supported.torch.layout
, optional) – the desired layout of returned window tensor. Only torch.strided
(dense layout) is supported.torch.device
, optional) – the desired device of returned tensor. Default: if None
, uses the current device for the default tensor type (see torch.set_default_tensor_type()
). device
will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.False
.torch.
blackman_window
(window_length, periodic=True, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensorwindow_length
is a positive integer controlling the returned window size. periodic
flag determines whether the returned window trims off the last duplicate value from the symmetric window and is ready to be used as a periodic window with functions like torch.stft()
. Therefore, if periodic
is true, the NNN in above formula is in fact window_length+1\text{window\_length} + 1window_length+1 . Also, we always have torch.blackman_window(L, periodic=True)
equal to torch.blackman_window(L + 1, periodic=False)[:-1])
.window_length
=1=1=1 , the returned window contains a single value 1.
torch.dtype
, optional) – the desired data type of returned tensor. Default: if None
, uses a global default (see torch.set_default_tensor_type()
). Only floating point types are supported.torch.layout
, optional) – the desired layout of returned window tensor. Only torch.strided
(dense layout) is supported.torch.device
, optional) – the desired device of returned tensor. Default: if None
, uses the current device for the default tensor type (see torch.set_default_tensor_type()
). device
will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.False
.torch.
hamming_window
(window_length, periodic=True, alpha=0.54, beta=0.46, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensorwindow_length
is a positive integer controlling the returned window size. periodic
flag determines whether the returned window trims off the last duplicate value from the symmetric window and is ready to be used as a periodic window with functions like torch.stft()
. Therefore, if periodic
is true, the NNN in above formula is in fact window_length+1\text{window\_length} + 1window_length+1 . Also, we always have torch.hamming_window(L, periodic=True)
equal to torch.hamming_window(L + 1, periodic=False)[:-1])
.window_length
=1=1=1 , the returned window contains a single value 1.torch.hann_window()
.
torch.dtype
, optional) – the desired data type of returned tensor. Default: if None
, uses a global default (see torch.set_default_tensor_type()
). Only floating point types are supported.torch.layout
, optional) – the desired layout of returned window tensor. Only torch.strided
(dense layout) is supported.torch.device
, optional) – the desired device of returned tensor. Default: if None
, uses the current device for the default tensor type (see torch.set_default_tensor_type()
). device
will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.False
.torch.
hann_window
(window_length, periodic=True, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensorwindow_length
is a positive integer controlling the returned window size. periodic
flag determines whether the returned window trims off the last duplicate value from the symmetric window and is ready to be used as a periodic window with functions like torch.stft()
. Therefore, if periodic
is true, the NNN in above formula is in fact window_length+1\text{window\_length} + 1window_length+1 . Also, we always have torch.hann_window(L, periodic=True)
equal to torch.hann_window(L + 1, periodic=False)[:-1])
.window_length
=1=1=1 , the returned window contains a single value 1.
torch.dtype
, optional) – the desired data type of returned tensor. Default: if None
, uses a global default (see torch.set_default_tensor_type()
). Only floating point types are supported.torch.layout
, optional) – the desired layout of returned window tensor. Only torch.strided
(dense layout) is supported.torch.device
, optional) – the desired device of returned tensor. Default: if None
, uses the current device for the default tensor type (see torch.set_default_tensor_type()
). device
will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.False
.torch.
bincount
(input, weights=None, minlength=0) → Tensorinput
unless input
is empty, in which case the result is a tensor of size 0. If minlength
is specified, the number of bins is at least minlength
and if input
is empty, then the result is tensor of size minlength
filled with zeros. If n
is the value at position i
, out[n] += weights[i]
if weights
is specified else out[n] += 1
.
Size([max(input) + 1])
if input
is non-empty, else Size(0)
>>> input = torch.randint(0, 8, (5,), dtype=torch.int64)
>>> weights = torch.linspace(0, 1, steps=5)
>>> input, weights
(tensor([4, 3, 6, 3, 4]),
tensor([ 0.0000, 0.2500, 0.5000, 0.7500, 1.0000])
>>> torch.bincount(input)
tensor([0, 0, 0, 2, 2, 0, 1])
>>> input.bincount(weights)
tensor([0.0000, 0.0000, 0.0000, 1.0000, 1.0000, 0.0000, 0.5000])
torch.
broadcast_tensors
(*tensors) → List of Tensors[source]>>> x = torch.arange(3).view(1, 3)
>>> y = torch.arange(2).view(2, 1)
>>> a, b = torch.broadcast_tensors(x, y)
>>> a.size()
torch.Size([2, 3])
>>> a
tensor([[0, 1, 2],
[0, 1, 2]])
torch.
cartesian_prod
(*tensors)[source]>>> a = [1, 2, 3]
>>> b = [4, 5]
>>> list(itertools.product(a, b))
[(1, 4), (1, 5), (2, 4), (2, 5), (3, 4), (3, 5)]
>>> tensor_a = torch.tensor(a)
>>> tensor_b = torch.tensor(b)
>>> torch.cartesian_prod(tensor_a, tensor_b)
tensor([[1, 4],
[1, 5],
[2, 4],
[2, 5],
[3, 4],
[3, 5]])
torch.
cdist
(x1, x2, p=2) → Tensor
>>> a = torch.tensor([[0.9041, 0.0196], [-0.3108, -2.4423], [-0.4821, 1.059]])
>>> a
tensor([[ 0.9041, 0.0196],
[-0.3108, -2.4423],
[-0.4821, 1.0590]])
>>> b = torch.tensor([[-2.1763, -0.4713], [-0.6986, 1.3702]])
>>> b
tensor([[-2.1763, -0.4713],
[-0.6986, 1.3702]])
>>> torch.cdist(a, b, p=2)
tensor([[3.1193, 2.0959],
[2.7138, 3.8322],
[2.2830, 0.3791]])
torch.
combinations
(input, r=2, with_replacement=False) → seq
>>> a = [1, 2, 3]
>>> list(itertools.combinations(a, r=2))
[(1, 2), (1, 3), (2, 3)]
>>> list(itertools.combinations(a, r=3))
[(1, 2, 3)]
>>> list(itertools.combinations_with_replacement(a, r=2))
[(1, 1), (1, 2), (1, 3), (2, 2), (2, 3), (3, 3)]
>>> tensor_a = torch.tensor(a)
>>> torch.combinations(tensor_a)
tensor([[1, 2],
[1, 3],
[2, 3]])
>>> torch.combinations(tensor_a, r=3)
tensor([[1, 2, 3]])
>>> torch.combinations(tensor_a, with_replacement=True)
tensor([[1, 1],
[1, 2],
[1, 3],
[2, 2],
[2, 3],
[3, 3]])
torch.
cross
(input, other, dim=-1, out=None) → Tensordim
of input
and other
.input
and other
must have the same size, and the size of their dim
dimension should be 3.dim
is not given, it defaults to the first dimension found with the size 3.
>>> a = torch.randn(4, 3)
>>> a
tensor([[-0.3956, 1.1455, 1.6895],
[-0.5849, 1.3672, 0.3599],
[-1.1626, 0.7180, -0.0521],
[-0.1339, 0.9902, -2.0225]])
>>> b = torch.randn(4, 3)
>>> b
tensor([[-0.0257, -1.4725, -1.2251],
[-1.1479, -0.7005, -1.9757],
[-1.3904, 0.3726, -1.1836],
[-0.9688, -0.7153, 0.2159]])
>>> torch.cross(a, b, dim=1)
tensor([[ 1.0844, -0.5281, 0.6120],
[-2.4490, -1.5687, 1.9792],
[-0.8304, -1.3037, 0.5650],
[-1.2329, 1.9883, 1.0551]])
>>> torch.cross(a, b)
tensor([[ 1.0844, -0.5281, 0.6120],
[-2.4490, -1.5687, 1.9792],
[-0.8304, -1.3037, 0.5650],
[-1.2329, 1.9883, 1.0551]])
torch.
diag
(input, diagonal=0, out=None) → Tensor
input
is a vector (1-D tensor), then returns a 2-D square tensor with the elements of input
as the diagonal.input
is a matrix (2-D tensor), then returns a 1-D tensor with the diagonal elements of input
.diagonal
controls which diagonal to consider:
diagonal
= 0, it is the main diagonal.diagonal
> 0, it is above the main diagonal.diagonal
< 0, it is below the main diagonal.
torch.diagonal()
always returns the diagonal of its input.torch.diagflat()
always constructs a tensor with diagonal elements specified by the input.>>> a = torch.randn(3)
>>> a
tensor([ 0.5950,-0.0872, 2.3298])
>>> torch.diag(a)
tensor([[ 0.5950, 0.0000, 0.0000],
[ 0.0000,-0.0872, 0.0000],
[ 0.0000, 0.0000, 2.3298]])
>>> torch.diag(a, 1)
tensor([[ 0.0000, 0.5950, 0.0000, 0.0000],
[ 0.0000, 0.0000,-0.0872, 0.0000],
[ 0.0000, 0.0000, 0.0000, 2.3298],
[ 0.0000, 0.0000, 0.0000, 0.0000]])
>>> a = torch.randn(3, 3)
>>> a
tensor([[-0.4264, 0.0255,-0.1064],
[ 0.8795,-0.2429, 0.1374],
[ 0.1029,-0.6482,-1.6300]])
>>> torch.diag(a, 0)
tensor([-0.4264,-0.2429,-1.6300])
>>> torch.diag(a, 1)
tensor([ 0.0255, 0.1374])
torch.
diag_embed
(input, offset=0, dim1=-2, dim2=-1) → Tensordim1
and dim2
) are filled by input
. To facilitate creating batched diagonal matrices, the 2D planes formed by the last two dimensions of the returned tensor are chosen by default.offset
controls which diagonal to consider:
offset
= 0, it is the main diagonal.offset
> 0, it is above the main diagonal.offset
< 0, it is below the main diagonal.offset
other than 000 , the order of dim1
and dim2
matters. Exchanging them is equivalent to changing the sign of offset
.torch.diagonal()
to the output of this function with the same arguments yields a matrix identical to input. However, torch.diagonal()
has different default dimensions, so those need to be explicitly specified.
>>> a = torch.randn(2, 3)
>>> torch.diag_embed(a)
tensor([[[ 1.5410, 0.0000, 0.0000],
[ 0.0000, -0.2934, 0.0000],
[ 0.0000, 0.0000, -2.1788]],
[[ 0.5684, 0.0000, 0.0000],
[ 0.0000, -1.0845, 0.0000],
[ 0.0000, 0.0000, -1.3986]]])
>>> torch.diag_embed(a, offset=1, dim1=0, dim2=2)
tensor([[[ 0.0000, 1.5410, 0.0000, 0.0000],
[ 0.0000, 0.5684, 0.0000, 0.0000]],
[[ 0.0000, 0.0000, -0.2934, 0.0000],
[ 0.0000, 0.0000, -1.0845, 0.0000]],
[[ 0.0000, 0.0000, 0.0000, -2.1788],
[ 0.0000, 0.0000, 0.0000, -1.3986]],
[[ 0.0000, 0.0000, 0.0000, 0.0000],
[ 0.0000, 0.0000, 0.0000, 0.0000]]])
torch.
diagflat
(input, offset=0) → Tensor
input
is a vector (1-D tensor), then returns a 2-D square tensor with the elements of input
as the diagonal.input
is a tensor with more than one dimension, then returns a 2-D tensor with diagonal elements equal to a flattened input
.offset
controls which diagonal to consider:
offset
= 0, it is the main diagonal.offset
> 0, it is above the main diagonal.offset
< 0, it is below the main diagonal.
>>> a = torch.randn(3)
>>> a
tensor([-0.2956, -0.9068, 0.1695])
>>> torch.diagflat(a)
tensor([[-0.2956, 0.0000, 0.0000],
[ 0.0000, -0.9068, 0.0000],
[ 0.0000, 0.0000, 0.1695]])
>>> torch.diagflat(a, 1)
tensor([[ 0.0000, -0.2956, 0.0000, 0.0000],
[ 0.0000, 0.0000, -0.9068, 0.0000],
[ 0.0000, 0.0000, 0.0000, 0.1695],
[ 0.0000, 0.0000, 0.0000, 0.0000]])
>>> a = torch.randn(2, 2)
>>> a
tensor([[ 0.2094, -0.3018],
[-0.1516, 1.9342]])
>>> torch.diagflat(a)
tensor([[ 0.2094, 0.0000, 0.0000, 0.0000],
[ 0.0000, -0.3018, 0.0000, 0.0000],
[ 0.0000, 0.0000, -0.1516, 0.0000],
[ 0.0000, 0.0000, 0.0000, 1.9342]])
torch.
diagonal
(input, offset=0, dim1=0, dim2=1) → Tensorinput
with the its diagonal elements with respect to dim1
and dim2
appended as a dimension at the end of the shape.offset
controls which diagonal to consider:
offset
= 0, it is the main diagonal.offset
> 0, it is above the main diagonal.offset
< 0, it is below the main diagonal.torch.diag_embed()
to the output of this function with the same arguments yields a diagonal matrix with the diagonal entries of the input. However, torch.diag_embed()
has different default dimensions, so those need to be explicitly specified.
>>> a = torch.randn(3, 3)
>>> a
tensor([[-1.0854, 1.1431, -0.1752],
[ 0.8536, -0.0905, 0.0360],
[ 0.6927, -0.3735, -0.4945]])
>>> torch.diagonal(a, 0)
tensor([-1.0854, -0.0905, -0.4945])
>>> torch.diagonal(a, 1)
tensor([ 1.1431, 0.0360])
>>> x = torch.randn(2, 5, 4, 2)
>>> torch.diagonal(x, offset=-1, dim1=1, dim2=2)
tensor([[[-1.2631, 0.3755, -1.5977, -1.8172],
[-1.1065, 1.0401, -0.2235, -0.7938]],
[[-1.7325, -0.3081, 0.6166, 0.2335],
[ 1.0500, 0.7336, -0.3836, -1.1015]]])
torch.
einsum
(equation, *operands) → Tensor[source]
>>> x = torch.randn(5)
>>> y = torch.randn(4)
>>> torch.einsum('i,j->ij', x, y) # outer product
tensor([[-0.0570, -0.0286, -0.0231, 0.0197],
[ 1.2616, 0.6335, 0.5113, -0.4351],
[ 1.4452, 0.7257, 0.5857, -0.4984],
[-0.4647, -0.2333, -0.1883, 0.1603],
[-1.1130, -0.5588, -0.4510, 0.3838]])
>>> A = torch.randn(3,5,4)
>>> l = torch.randn(2,5)
>>> r = torch.randn(2,4)
>>> torch.einsum('bn,anm,bm->ba', l, A, r) # compare torch.nn.functional.bilinear
tensor([[-0.3430, -5.2405, 0.4494],
[ 0.3311, 5.5201, -3.0356]])
>>> As = torch.randn(3,2,5)
>>> Bs = torch.randn(3,5,4)
>>> torch.einsum('bij,bjk->bik', As, Bs) # batch matrix multiplication
tensor([[[-1.0564, -1.5904, 3.2023, 3.1271],
[-1.6706, -0.8097, -0.8025, -2.1183]],
[[ 4.2239, 0.3107, -0.5756, -0.2354],
[-1.4558, -0.3460, 1.5087, -0.8530]],
[[ 2.8153, 1.8787, -4.3839, -1.2112],
[ 0.3728, -2.1131, 0.0921, 0.8305]]])
>>> A = torch.randn(3, 3)
>>> torch.einsum('ii->i', A) # diagonal
tensor([-0.7825, 0.8291, -0.1936])
>>> A = torch.randn(4, 3, 3)
>>> torch.einsum('...ii->...i', A) # batch diagonal
tensor([[-1.0864, 0.7292, 0.0569],
[-0.9725, -1.0270, 0.6493],
[ 0.5832, -1.1716, -1.5084],
[ 0.4041, -1.1690, 0.8570]])
>>> A = torch.randn(2, 3, 4, 5)
>>> torch.einsum('...ij->...ji', A).shape # batch permute
torch.Size([2, 3, 5, 4])
torch.
flatten
(input, start_dim=0, end_dim=-1) → Tensor
>>> t = torch.tensor([[[1, 2],
[3, 4]],
[[5, 6],
[7, 8]]])
>>> torch.flatten(t)
tensor([1, 2, 3, 4, 5, 6, 7, 8])
>>> torch.flatten(t, start_dim=1)
tensor([[1, 2, 3, 4],
[5, 6, 7, 8]])
torch.
flip
(input, dims) → Tensor
>>> x = torch.arange(8).view(2, 2, 2)
>>> x
tensor([[[ 0, 1],
[ 2, 3]],
[[ 4, 5],
[ 6, 7]]])
>>> torch.flip(x, [0, 1])
tensor([[[ 6, 7],
[ 4, 5]],
[[ 2, 3],
[ 0, 1]]])
torch.
rot90
(input, k, dims) → Tensor
>>> x = torch.arange(4).view(2, 2)
>>> x
tensor([[0, 1],
[2, 3]])
>>> torch.rot90(x, 1, [0, 1])
tensor([[1, 3],
[0, 2]])
>>> x = torch.arange(8).view(2, 2, 2)
>>> x
tensor([[[0, 1],
[2, 3]],
[[4, 5],
[6, 7]]])
>>> torch.rot90(x, 1, [1, 2])
tensor([[[1, 3],
[0, 2]],
[[5, 7],
[4, 6]]])
torch.
histc
(input, bins=100, min=0, max=0, out=None) → Tensormin
and max
. If min
and max
are both zero, the minimum and maximum values of the data are used.
>>> torch.histc(torch.tensor([1., 2, 1]), bins=4, min=0, max=3)
tensor([ 0., 2., 1., 0.])
torch.
meshgrid
(*tensors, **kwargs)[source]
>>> x = torch.tensor([1, 2, 3])
>>> y = torch.tensor([4, 5, 6])
>>> grid_x, grid_y = torch.meshgrid(x, y)
>>> grid_x
tensor([[1, 1, 1],
[2, 2, 2],
[3, 3, 3]])
>>> grid_y
tensor([[4, 5, 6],
[4, 5, 6],
[4, 5, 6]])
torch.
renorm
(input, p, dim, maxnorm, out=None) → Tensorinput
along dimension dim
is normalized such that the p-norm of the sub-tensor is lower than the value maxnorm
>>> x = torch.ones(3, 3)
>>> x[1].fill_(2)
tensor([ 2., 2., 2.])
>>> x[2].fill_(3)
tensor([ 3., 3., 3.])
>>> x
tensor([[ 1., 1., 1.],
[ 2., 2., 2.],
[ 3., 3., 3.]])
>>> torch.renorm(x, 1, 0, 5)
tensor([[ 1.0000, 1.0000, 1.0000],
[ 1.6667, 1.6667, 1.6667],
[ 1.6667, 1.6667, 1.6667]])
torch.
repeat_interleave
()torch.
repeat_interleave
(input, repeats, dim=None) → Tensortorch.repeat()
but similar to numpy.repeat.
>>> x = torch.tensor([1, 2, 3])
>>> x.repeat_interleave(2)
tensor([1, 1, 2, 2, 3, 3])
>>> y = torch.tensor([[1, 2], [3, 4]])
>>> torch.repeat_interleave(y, 2)
tensor([1, 1, 2, 2, 3, 3, 4, 4])
>>> torch.repeat_interleave(y, 3, dim=1)
tensor([[1, 1, 1, 2, 2, 2],
[3, 3, 3, 4, 4, 4]])
>>> torch.repeat_interleave(y, torch.tensor([1, 2]), dim=0)
tensor([[1, 2],
[3, 4],
[3, 4]])
torch.
repeat_interleave
(repeats) → Tensortorch.
roll
(input, shifts, dims=None) → Tensor
>>> x = torch.tensor([1, 2, 3, 4, 5, 6, 7, 8]).view(4, 2)
>>> x
tensor([[1, 2],
[3, 4],
[5, 6],
[7, 8]])
>>> torch.roll(x, 1, 0)
tensor([[7, 8],
[1, 2],
[3, 4],
[5, 6]])
>>> torch.roll(x, -1, 0)
tensor([[3, 4],
[5, 6],
[7, 8],
[1, 2]])
>>> torch.roll(x, shifts=(2, 1), dims=(0, 1))
tensor([[6, 5],
[8, 7],
[2, 1],
[4, 3]])
torch.
tensordot
(a, b, dims=2)[source]tensordot
implements a generalized matrix product.
a
and b
respectivelydims
= ddd , and the number of dimensions of a
and b
is mmm and nnn , respectively, it computesdims
of the list form, the given dimensions will be contracted in place of the last ddd of a
and the first ddd of bbb . The sizes in these dimensions must match, but tensordot
will deal with broadcasted dimensions.>>> a = torch.arange(60.).reshape(3, 4, 5)
>>> b = torch.arange(24.).reshape(4, 3, 2)
>>> torch.tensordot(a, b, dims=([1, 0], [0, 1]))
tensor([[4400., 4730.],
[4532., 4874.],
[4664., 5018.],
[4796., 5162.],
[4928., 5306.]])
>>> a = torch.randn(3, 4, 5, device='cuda')
>>> b = torch.randn(4, 5, 6, device='cuda')
>>> c = torch.tensordot(a, b, dims=2).cpu()
tensor([[ 8.3504, -2.5436, 6.2922, 2.7556, -1.0732, 3.2741],
[ 3.3161, 0.0704, 5.0187, -0.4079, -4.3126, 4.8744],
[ 0.8223, 3.9445, 3.2168, -0.2400, 3.4117, 1.7780]])
torch.
trace
(input) → Tensor>>> x = torch.arange(1., 10.).view(3, 3)
>>> x
tensor([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.]])
>>> torch.trace(x)
tensor(15.)
torch.
tril
(input, diagonal=0, out=None) → Tensorinput
, the other elements of the result tensor out
are set to 0.diagonal
controls which diagonal to consider. If diagonal
= 0, all elements on and below the main diagonal are retained. A positive value includes just as many diagonals above the main diagonal, and similarly a negative value excludes just as many diagonals below the main diagonal. The main diagonal are the set of indices {(i,i)}\lbrace (i, i) \rbrace{(i,i)} for i∈[0,min{d1,d2}−1]i \in [0, \min\{d_{1}, d_{2}\} - 1]i∈[0,min{d1,d2}−1] where d1,d2d_{1}, d_{2}d1,d2 are the dimensions of the matrix.
>>> a = torch.randn(3, 3)
>>> a
tensor([[-1.0813, -0.8619, 0.7105],
[ 0.0935, 0.1380, 2.2112],
[-0.3409, -0.9828, 0.0289]])
>>> torch.tril(a)
tensor([[-1.0813, 0.0000, 0.0000],
[ 0.0935, 0.1380, 0.0000],
[-0.3409, -0.9828, 0.0289]])
>>> b = torch.randn(4, 6)
>>> b
tensor([[ 1.2219, 0.5653, -0.2521, -0.2345, 1.2544, 0.3461],
[ 0.4785, -0.4477, 0.6049, 0.6368, 0.8775, 0.7145],
[ 1.1502, 3.2716, -1.1243, -0.5413, 0.3615, 0.6864],
[-0.0614, -0.7344, -1.3164, -0.7648, -1.4024, 0.0978]])
>>> torch.tril(b, diagonal=1)
tensor([[ 1.2219, 0.5653, 0.0000, 0.0000, 0.0000, 0.0000],
[ 0.4785, -0.4477, 0.6049, 0.0000, 0.0000, 0.0000],
[ 1.1502, 3.2716, -1.1243, -0.5413, 0.0000, 0.0000],
[-0.0614, -0.7344, -1.3164, -0.7648, -1.4024, 0.0000]])
>>> torch.tril(b, diagonal=-1)
tensor([[ 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[ 0.4785, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[ 1.1502, 3.2716, 0.0000, 0.0000, 0.0000, 0.0000],
[-0.0614, -0.7344, -1.3164, 0.0000, 0.0000, 0.0000]])
torch.
tril_indices
(row, col, offset=0, dtype=torch.long, device='cpu', layout=torch.strided) → Tensorrow
-by- col
matrix in a 2-by-N Tensor, where the first row contains row coordinates of all indices and the second row contains column coordinates. Indices are ordered based on rows and then columns.offset
controls which diagonal to consider. If offset
= 0, all elements on and below the main diagonal are retained. A positive value includes just as many diagonals above the main diagonal, and similarly a negative value excludes just as many diagonals below the main diagonal. The main diagonal are the set of indices {(i,i)}\lbrace (i, i) \rbrace{(i,i)} for i∈[0,min{d1,d2}−1]i \in [0, \min\{d_{1}, d_{2}\} - 1]i∈[0,min{d1,d2}−1] where d1,d2d_{1}, d_{2}d1,d2 are the dimensions of the matrix.
int
) – number of rows in the 2-D matrix.int
) – number of columns in the 2-D matrix.int
) – diagonal offset from the main diagonal. Default: if not provided, 0.torch.dtype
, optional) – the desired data type of returned tensor. Default: if None
, torch.long
.torch.device
, optional) – the desired device of returned tensor. Default: if None
, uses the current device for the default tensor type (see torch.set_default_tensor_type()
). device
will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.torch.layout
, optional) – currently only support torch.strided
.>>> a = torch.tril_indices(3, 3)
>>> a
tensor([[0, 1, 1, 2, 2, 2],
[0, 0, 1, 0, 1, 2]])
>>> a = torch.tril_indices(4, 3, -1)
>>> a
tensor([[1, 2, 2, 3, 3, 3],
[0, 0, 1, 0, 1, 2]])
>>> a = torch.tril_indices(4, 3, 1)
>>> a
tensor([[0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3],
[0, 1, 0, 1, 2, 0, 1, 2, 0, 1, 2]])
torch.
triu
(input, diagonal=0, out=None) → Tensorinput
, the other elements of the result tensor out
are set to 0.diagonal
controls which diagonal to consider. If diagonal
= 0, all elements on and below the main diagonal are retained. A positive value excludes just as many diagonals above the main diagonal, and similarly a negative value includes just as many diagonals below the main diagonal. The main diagonal are the set of indices {(i,i)}\lbrace (i, i) \rbrace{(i,i)} for i∈[0,min{d1,d2}−1]i \in [0, \min\{d_{1}, d_{2}\} - 1]i∈[0,min{d1,d2}−1] where d1,d2d_{1}, d_{2}d1,d2 are the dimensions of the matrix.
>>> a = torch.randn(3, 3)
>>> a
tensor([[ 0.2309, 0.5207, 2.0049],
[ 0.2072, -1.0680, 0.6602],
[ 0.3480, -0.5211, -0.4573]])
>>> torch.triu(a)
tensor([[ 0.2309, 0.5207, 2.0049],
[ 0.0000, -1.0680, 0.6602],
[ 0.0000, 0.0000, -0.4573]])
>>> torch.triu(a, diagonal=1)
tensor([[ 0.0000, 0.5207, 2.0049],
[ 0.0000, 0.0000, 0.6602],
[ 0.0000, 0.0000, 0.0000]])
>>> torch.triu(a, diagonal=-1)
tensor([[ 0.2309, 0.5207, 2.0049],
[ 0.2072, -1.0680, 0.6602],
[ 0.0000, -0.5211, -0.4573]])
>>> b = torch.randn(4, 6)
>>> b
tensor([[ 0.5876, -0.0794, -1.8373, 0.6654, 0.2604, 1.5235],
[-0.2447, 0.9556, -1.2919, 1.3378, -0.1768, -1.0857],
[ 0.4333, 0.3146, 0.6576, -1.0432, 0.9348, -0.4410],
[-0.9888, 1.0679, -1.3337, -1.6556, 0.4798, 0.2830]])
>>> torch.triu(b, diagonal=1)
tensor([[ 0.0000, -0.0794, -1.8373, 0.6654, 0.2604, 1.5235],
[ 0.0000, 0.0000, -1.2919, 1.3378, -0.1768, -1.0857],
[ 0.0000, 0.0000, 0.0000, -1.0432, 0.9348, -0.4410],
[ 0.0000, 0.0000, 0.0000, 0.0000, 0.4798, 0.2830]])
>>> torch.triu(b, diagonal=-1)
tensor([[ 0.5876, -0.0794, -1.8373, 0.6654, 0.2604, 1.5235],
[-0.2447, 0.9556, -1.2919, 1.3378, -0.1768, -1.0857],
[ 0.0000, 0.3146, 0.6576, -1.0432, 0.9348, -0.4410],
[ 0.0000, 0.0000, -1.3337, -1.6556, 0.4798, 0.2830]])
torch.
triu_indices
(row, col, offset=0, dtype=torch.long, device='cpu', layout=torch.strided) → Tensorrow
by col
matrix in a 2-by-N Tensor, where the first row contains row coordinates of all indices and the second row contains column coordinates. Indices are ordered based on rows and then columns.offset
controls which diagonal to consider. If offset
= 0, all elements on and above the main diagonal are retained. A positive value excludes just as many diagonals above the main diagonal, and similarly a negative value includes just as many diagonals below the main diagonal. The main diagonal are the set of indices {(i,i)}\lbrace (i, i) \rbrace{(i,i)} for i∈[0,min{d1,d2}−1]i \in [0, \min\{d_{1}, d_{2}\} - 1]i∈[0,min{d1,d2}−1] where d1,d2d_{1}, d_{2}d1,d2 are the dimensions of the matrix.
int
) – number of rows in the 2-D matrix.int
) – number of columns in the 2-D matrix.int
) – diagonal offset from the main diagonal. Default: if not provided, 0.torch.dtype
, optional) – the desired data type of returned tensor. Default: if None
, torch.long
.torch.device
, optional) – the desired device of returned tensor. Default: if None
, uses the current device for the default tensor type (see torch.set_default_tensor_type()
). device
will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.torch.layout
, optional) – currently only support torch.strided
.>>> a = torch.triu_indices(3, 3)
>>> a
tensor([[0, 0, 0, 1, 1, 2],
[0, 1, 2, 1, 2, 2]])
>>> a = torch.triu_indices(4, 3, -1)
>>> a
tensor([[0, 0, 0, 1, 1, 1, 2, 2, 3],
[0, 1, 2, 0, 1, 2, 1, 2, 2]])
>>> a = torch.triu_indices(4, 3, 1)
>>> a
tensor([[0, 0, 1],
[1, 2, 2]])
BLAS and LAPACK Operations
torch.
addbmm
(beta=1, input, alpha=1, batch1, batch2, out=None) → Tensorbatch1
and batch2
, with a reduced add step (all matrix multiplications get accumulated along the first dimension). input
is added to the final result.batch1
and batch2
must be 3-D tensors each containing the same number of matrices.batch1
is a (b×n×m)(b \times n \times m)(b×n×m) tensor, batch2
is a (b×m×p)(b \times m \times p)(b×m×p) tensor, input
must be broadcastable with a (n×p)(n \times p)(n×p) tensor and out
will be a (n×p)(n \times p)(n×p) tensor.beta
and alpha
must be real numbers, otherwise they should be integers.
input
(β\betaβ )>>> M = torch.randn(3, 5)
>>> batch1 = torch.randn(10, 3, 4)
>>> batch2 = torch.randn(10, 4, 5)
>>> torch.addbmm(M, batch1, batch2)
tensor([[ 6.6311, 0.0503, 6.9768, -12.0362, -2.1653],
[ -4.8185, -1.4255, -6.6760, 8.9453, 2.5743],
[ -3.8202, 4.3691, 1.0943, -1.1109, 5.4730]])
torch.
addmm
(beta=1, input, alpha=1, mat1, mat2, out=None) → Tensormat1
and mat2
. The matrix input
is added to the final result.mat1
is a (n×m)(n \times m)(n×m) tensor, mat2
is a (m×p)(m \times p)(m×p) tensor, then input
must be broadcastable with a (n×p)(n \times p)(n×p) tensor and out
will be a (n×p)(n \times p)(n×p) tensor.alpha
and beta
are scaling factors on matrix-vector product between mat1
and mat2
and the added matrix input
respectively.beta
and alpha
must be real numbers, otherwise they should be integers.
input
(β\betaβ )>>> M = torch.randn(2, 3)
>>> mat1 = torch.randn(2, 3)
>>> mat2 = torch.randn(3, 3)
>>> torch.addmm(M, mat1, mat2)
tensor([[-4.8716, 1.4671, -1.3746],
[ 0.7573, -3.9555, -2.8681]])
torch.
addmv
(beta=1, input, alpha=1, mat, vec, out=None) → Tensormat
and the vector vec
. The vector input
is added to the final result.mat
is a (n×m)(n \times m)(n×m) tensor, vec
is a 1-D tensor of size m, then input
must be broadcastable with a 1-D tensor of size n and out
will be 1-D tensor of size n.alpha
and beta
are scaling factors on matrix-vector product between mat
and vec
and the added tensor input
respectively.beta
and alpha
must be real numbers, otherwise they should be integers
input
(β\betaβ )>>> M = torch.randn(2)
>>> mat = torch.randn(2, 3)
>>> vec = torch.randn(3)
>>> torch.addmv(M, mat, vec)
tensor([-0.3768, -5.5565])
torch.
addr
(beta=1, input, alpha=1, vec1, vec2, out=None) → Tensorvec1
and vec2
and adds it to the matrix input
.beta
and alpha
are scaling factors on the outer product between vec1
and vec2
and the added matrix input
respectively.vec1
is a vector of size n and vec2
is a vector of size m, then input
must be broadcastable with a matrix of size (n×m)(n \times m)(n×m) and out
will be a matrix of size (n×m)(n \times m)(n×m) .beta
and alpha
must be real numbers, otherwise they should be integers
input
(β\betaβ )>>> vec1 = torch.arange(1., 4.)
>>> vec2 = torch.arange(1., 3.)
>>> M = torch.zeros(3, 2)
>>> torch.addr(M, vec1, vec2)
tensor([[ 1., 2.],
[ 2., 4.],
[ 3., 6.]])
torch.
baddbmm
(beta=1, input, alpha=1, batch1, batch2, out=None) → Tensorbatch1
and batch2
. input
is added to the final result.batch1
and batch2
must be 3-D tensors each containing the same number of matrices.batch1
is a (b×n×m)(b \times n \times m)(b×n×m) tensor, batch2
is a (b×m×p)(b \times m \times p)(b×m×p) tensor, then input
must be broadcastable with a (b×n×p)(b \times n \times p)(b×n×p) tensor and out
will be a (b×n×p)(b \times n \times p)(b×n×p) tensor. Both alpha
and beta
mean the same as the scaling factors used in torch.addbmm()
.beta
and alpha
must be real numbers, otherwise they should be integers.
input
(β\betaβ )>>> M = torch.randn(10, 3, 5)
>>> batch1 = torch.randn(10, 3, 4)
>>> batch2 = torch.randn(10, 4, 5)
>>> torch.baddbmm(M, batch1, batch2).size()
torch.Size([10, 3, 5])
torch.
bmm
(input, mat2, out=None) → Tensorinput
and mat2
.input
and mat2
must be 3-D tensors each containing the same number of matrices.input
is a (b×n×m)(b \times n \times m)(b×n×m) tensor, mat2
is a (b×m×p)(b \times m \times p)(b×m×p) tensor, out
will be a (b×n×p)(b \times n \times p)(b×n×p) tensor.torch.matmul()
.
>>> input = torch.randn(10, 3, 4)
>>> mat2 = torch.randn(10, 4, 5)
>>> res = torch.bmm(input, mat2)
>>> res.size()
torch.Size([10, 3, 5])
torch.
chain_matmul
(*matrices)[source]>>> a = torch.randn(3, 4)
>>> b = torch.randn(4, 5)
>>> c = torch.randn(5, 6)
>>> d = torch.randn(6, 7)
>>> torch.chain_matmul(a, b, c, d)
tensor([[ -2.3375, -3.9790, -4.1119, -6.6577, 9.5609, -11.5095, -3.2614],
[ 21.4038, 3.3378, -8.4982, -5.2457, -10.2561, -2.4684, 2.7163],
[ -0.9647, -5.8917, -2.3213, -5.2284, 12.8615, -12.2816, -2.5095]])
torch.
cholesky
(input, upper=False, out=None) → Tensorupper
is True
, the returned matrix U
is upper-triangular, and the decomposition has the form:upper
is False
, the returned matrix L
is lower-triangular, and the decomposition has the form:upper
is True
, and AAA is a batch of symmetric positive-definite matrices, then the returned tensor will be composed of upper-triangular Cholesky factors of each of the individual matrices. Similarly, when upper
is False
, the returned tensor will be composed of lower-triangular Cholesky factors of each of the individual matrices.
False
>>> a = torch.randn(3, 3)
>>> a = torch.mm(a, a.t()) # make symmetric positive-definite
>>> l = torch.cholesky(a)
>>> a
tensor([[ 2.4112, -0.7486, 1.4551],
[-0.7486, 1.3544, 0.1294],
[ 1.4551, 0.1294, 1.6724]])
>>> l
tensor([[ 1.5528, 0.0000, 0.0000],
[-0.4821, 1.0592, 0.0000],
[ 0.9371, 0.5487, 0.7023]])
>>> torch.mm(l, l.t())
tensor([[ 2.4112, -0.7486, 1.4551],
[-0.7486, 1.3544, 0.1294],
[ 1.4551, 0.1294, 1.6724]])
>>> a = torch.randn(3, 2, 2)
>>> a = torch.matmul(a, a.transpose(-1, -2)) + 1e-03 # make symmetric positive-definite
>>> l = torch.cholesky(a)
>>> z = torch.matmul(l, l.transpose(-1, -2))
>>> torch.max(torch.abs(z - a)) # Max non-zero
tensor(2.3842e-07)
torch.
cholesky_inverse
(input, upper=False, out=None) → Tensorinv
. The inverse is computed using LAPACK routines dpotri
and spotri
(and the corresponding MAGMA routines).upper
is False
, uuu is lower triangular such that the returned tensor isupper
is True
or not provided, uuu is upper triangular such that the returned tensor is
>>> a = torch.randn(3, 3)
>>> a = torch.mm(a, a.t()) + 1e-05 * torch.eye(3) # make symmetric positive definite
>>> u = torch.cholesky(a)
>>> a
tensor([[ 0.9935, -0.6353, 1.5806],
[ -0.6353, 0.8769, -1.7183],
[ 1.5806, -1.7183, 10.6618]])
>>> torch.cholesky_inverse(u)
tensor([[ 1.9314, 1.2251, -0.0889],
[ 1.2251, 2.4439, 0.2122],
[-0.0889, 0.2122, 0.1412]])
>>> a.inverse()
tensor([[ 1.9314, 1.2251, -0.0889],
[ 1.2251, 2.4439, 0.2122],
[-0.0889, 0.2122, 0.1412]])
torch.
cholesky_solve
(input, input2, upper=False, out=None) → Tensorupper
is False
, uuu is and lower triangular and c is returned such that:upper
is True
or not provided, uuu is upper triangular and c is returned such that:out
keyword only supports 2D matrix inputs, that is, b, u must be 2D matrices.
False
.>>> a = torch.randn(3, 3)
>>> a = torch.mm(a, a.t()) # make symmetric positive definite
>>> u = torch.cholesky(a)
>>> a
tensor([[ 0.7747, -1.9549, 1.3086],
[-1.9549, 6.7546, -5.4114],
[ 1.3086, -5.4114, 4.8733]])
>>> b = torch.randn(3, 2)
>>> b
tensor([[-0.6355, 0.9891],
[ 0.1974, 1.4706],
[-0.4115, -0.6225]])
>>> torch.cholesky_solve(b, u)
tensor([[ -8.1625, 19.6097],
[ -5.8398, 14.2387],
[ -4.3771, 10.4173]])
>>> torch.mm(a.inverse(), b)
tensor([[ -8.1626, 19.6097],
[ -5.8398, 14.2387],
[ -4.3771, 10.4173]])
torch.
dot
(input, tensor) → Tensor>>> torch.dot(torch.tensor([2, 3]), torch.tensor([2, 1]))
tensor(7)
torch.
eig
(input, eigenvectors=False, out=None) -> (Tensor, Tensor)torch.symeig()
True
to compute both eigenvalues and eigenvectors; otherwise, only eigenvalues will be computed
input
, where the first element is the real part and the second element is the imaginary part. The eigenvalues are not necessarily ordered.eigenvectors=False
, it’s an empty tensor. Otherwise, this tensor of shape (n×n)(n \times n)(n×n) can be used to compute normalized (unit length) eigenvectors of corresponding eigenvalues as follows. If the corresponding eigenvalues[j] is a real number, column eigenvectors[:, j] is the eigenvector corresponding to eigenvalues[j]. If the corresponding eigenvalues[j] and eigenvalues[j + 1] form a complex conjugate pair, then the true eigenvectors can be computed as true eigenvector[j]=eigenvectors[:,j]+i×eigenvectors[:,j+1]\text{true eigenvector}[j] = eigenvectors[:, j] + i \times eigenvectors[:, j + 1]true eigenvector[j]=eigenvectors[:,j]+i×eigenvectors[:,j+1] , true eigenvector[j+1]=eigenvectors[:,j]−i×eigenvectors[:,j+1]\text{true eigenvector}[j + 1] = eigenvectors[:, j] - i \times eigenvectors[:, j + 1]true eigenvector[j+1]=eigenvectors[:,j]−i×eigenvectors[:,j+1] .torch.
gels
(input, A, out=None)[source]torch.gels()
, please check torch.lstsq()
.torch.gels()
is deprecated in favour of torch.lstsq()
and will be removed in the next release. Please use torch.lstsq()
instead.torch.
geqrf
(input, out=None) -> (Tensor, Tensor)torch.qr()
instead.input
, but without constructing QQQ and RRR as explicit separate matrices.
torch.
ger
(input, vec2, out=None) → Tensorinput
and vec2
. If input
is a vector of size nnn and vec2
is a vector of size mmm , then out
must be a matrix of size (n×m)(n \times m)(n×m) .
>>> v1 = torch.arange(1., 5.)
>>> v2 = torch.arange(1., 4.)
>>> torch.ger(v1, v2)
tensor([[ 1., 2., 3.],
[ 2., 4., 6.],
[ 3., 6., 9.],
[ 4., 8., 12.]])
torch.
inverse
(input, out=None) → Tensorinput
. input
can be batches of 2D square tensors, in which case this function would return a tensor composed of individual inverses.
>>> x = torch.rand(4, 4)
>>> y = torch.inverse(x)
>>> z = torch.mm(x, y)
>>> z
tensor([[ 1.0000, -0.0000, -0.0000, 0.0000],
[ 0.0000, 1.0000, 0.0000, 0.0000],
[ 0.0000, 0.0000, 1.0000, 0.0000],
[ 0.0000, -0.0000, -0.0000, 1.0000]])
>>> torch.max(torch.abs(z - torch.eye(4))) # Max non-zero
tensor(1.1921e-07)
>>> # Batched inverse example
>>> x = torch.randn(2, 3, 4, 4)
>>> y = torch.inverse(x)
>>> z = torch.matmul(x, y)
>>> torch.max(torch.abs(z - torch.eye(4).expand_as(x))) # Max non-zero
tensor(1.9073e-06)
torch.
det
(input) → Tensordet()
internally uses SVD results when input
is not invertible. In this case, double backward through det()
will be unstable in when input
doesn’t have distinct singular values. See svd()
for details.>>> A = torch.randn(3, 3)
>>> torch.det(A)
tensor(3.7641)
>>> A = torch.randn(3, 2, 2)
>>> A
tensor([[[ 0.9254, -0.6213],
[-0.5787, 1.6843]],
[[ 0.3242, -0.9665],
[ 0.4539, -0.0887]],
[[ 1.1336, -0.4025],
[-0.7089, 0.9032]]])
>>> A.det()
tensor([1.1990, 0.4099, 0.7386])
torch.
logdet
(input) → Tensor-inf
if input
has zero log determinant, and is nan
if input
has negative determinant.logdet()
internally uses SVD results when input
is not invertible. In this case, double backward through logdet()
will be unstable in when input
doesn’t have distinct singular values. See svd()
for details.>>> A = torch.randn(3, 3)
>>> torch.det(A)
tensor(0.2611)
>>> torch.logdet(A)
tensor(-1.3430)
>>> A
tensor([[[ 0.9254, -0.6213],
[-0.5787, 1.6843]],
[[ 0.3242, -0.9665],
[ 0.4539, -0.0887]],
[[ 1.1336, -0.4025],
[-0.7089, 0.9032]]])
>>> A.det()
tensor([1.1990, 0.4099, 0.7386])
>>> A.det().log()
tensor([ 0.1815, -0.8917, -0.3031])
torch.
slogdet
(input) -> (Tensor, Tensor)input
has zero determinant, this returns (0, -inf)
.slogdet()
internally uses SVD results when input
is not invertible. In this case, double backward through slogdet()
will be unstable in when input
doesn’t have distinct singular values. See svd()
for details.>>> A = torch.randn(3, 3)
>>> A
tensor([[ 0.0032, -0.2239, -1.1219],
[-0.6690, 0.1161, 0.4053],
[-1.6218, -0.9273, -0.0082]])
>>> torch.det(A)
tensor(-0.7576)
>>> torch.logdet(A)
tensor(nan)
>>> torch.slogdet(A)
torch.return_types.slogdet(sign=tensor(-1.), logabsdet=tensor(-0.2776))
torch.
lstsq
(input, A, out=None) → Tensorlstsq()
solves the least-squares problem:
>>> A = torch.tensor([[1., 1, 1],
[2, 3, 4],
[3, 5, 2],
[4, 2, 5],
[5, 4, 3]])
>>> B = torch.tensor([[-10., -3],
[ 12, 14],
[ 14, 12],
[ 16, 16],
[ 18, 16]])
>>> X, _ = torch.lstsq(B, A)
>>> X
tensor([[ 2.0000, 1.0000],
[ 1.0000, 1.0000],
[ 1.0000, 2.0000],
[ 10.9635, 4.8501],
[ 8.9332, 5.2418]])
torch.
lu
(A, pivot=True, get_infos=False, out=None)[source]A
. Returns a tuple containing the LU factorization and pivots of A
. Pivoting is done if pivot
is set to True
.pivot
is False
, then the returned pivots is a tensor filled with zeros of the appropriate size.pivot
= False
is not available for CPU, and attempting to do so will throw an error. However, LU factorization with pivot
= False
is available for CUDA.get_infos
is True
since the status of the factorization is present in the third element of the return tuple.
True
True
, returns an info IntTensor. Default: False
get_infos
is True
, then the elements in the tuple are Tensor, IntTensor, and IntTensor. If get_infos
is False
, then the elements in the tuple are Tensor, IntTensor. Default: None
get_infos
is True
, this is a tensor of size (∗)(*)(∗) where non-zero values indicate whether factorization for the matrix or each minibatch has succeeded or failed>>> A = torch.randn(2, 3, 3)
>>> A_LU, pivots = torch.lu(A)
>>> A_LU
tensor([[[ 1.3506, 2.5558, -0.0816],
[ 0.1684, 1.1551, 0.1940],
[ 0.1193, 0.6189, -0.5497]],
[[ 0.4526, 1.2526, -0.3285],
[-0.7988, 0.7175, -0.9701],
[ 0.2634, -0.9255, -0.3459]]])
>>> pivots
tensor([[ 3, 3, 3],
[ 3, 3, 3]], dtype=torch.int32)
>>> A_LU, pivots, info = torch.lu(A, get_infos=True)
>>> if info.nonzero().size(0) == 0:
... print('LU factorization succeeded for all samples!')
LU factorization succeeded for all samples!
torch.
lu_solve
(input, LU_data, LU_pivots, out=None) → Tensortorch.lu()
.
torch.lu()
of size (∗,m,m)(*, m, m)(∗,m,m) , where ∗*∗ is zero or more batch dimensions.torch.lu()
of size (∗,m)(*, m)(∗,m) , where ∗*∗ is zero or more batch dimensions. The batch dimensions of LU_pivots
must be equal to the batch dimensions of LU_data
.>>> A = torch.randn(2, 3, 3)
>>> b = torch.randn(2, 3, 1)
>>> A_LU = torch.lu(A)
>>> x = torch.lu_solve(b, *A_LU)
>>> torch.norm(torch.bmm(A, x) - b)
tensor(1.00000e-07 *
2.8312)
torch.
lu_unpack
(LU_data, LU_pivots, unpack_data=True, unpack_pivots=True)[source](the pivots, the L tensor, the U tensor)
.
>>> A = torch.randn(2, 3, 3)
>>> A_LU, pivots = A.lu()
>>> P, A_L, A_U = torch.lu_unpack(A_LU, pivots)
>>>
>>> # can recover A from factorization
>>> A_ = torch.bmm(P, torch.bmm(A_L, A_U))
torch.
matmul
(input, other, out=None) → Tensor
input
is a (j×1×n×m)(j \times 1 \times n \times m)(j×1×n×m) tensor and other
is a (k×m×p)(k \times m \times p)(k×m×p) tensor, out
will be an (j×k×n×p)(j \times k \times n \times p)(j×k×n×p) tensor.out
parameter.
>>> # vector x vector
>>> tensor1 = torch.randn(3)
>>> tensor2 = torch.randn(3)
>>> torch.matmul(tensor1, tensor2).size()
torch.Size([])
>>> # matrix x vector
>>> tensor1 = torch.randn(3, 4)
>>> tensor2 = torch.randn(4)
>>> torch.matmul(tensor1, tensor2).size()
torch.Size([3])
>>> # batched matrix x broadcasted vector
>>> tensor1 = torch.randn(10, 3, 4)
>>> tensor2 = torch.randn(4)
>>> torch.matmul(tensor1, tensor2).size()
torch.Size([10, 3])
>>> # batched matrix x batched matrix
>>> tensor1 = torch.randn(10, 3, 4)
>>> tensor2 = torch.randn(10, 4, 5)
>>> torch.matmul(tensor1, tensor2).size()
torch.Size([10, 3, 5])
>>> # batched matrix x broadcasted matrix
>>> tensor1 = torch.randn(10, 3, 4)
>>> tensor2 = torch.randn(4, 5)
>>> torch.matmul(tensor1, tensor2).size()
torch.Size([10, 3, 5])
torch.
matrix_power
(input, n) → Tensorn
for square matrices. For batch of matrices, each individual matrix is raised to the power n
.n
is negative, then the inverse of the matrix (if invertible) is raised to the power n
. For a batch of matrices, the batched inverse (if invertible) is raised to the power n
. If n
is 0, then an identity matrix is returned.
>>> a = torch.randn(2, 2, 2)
>>> a
tensor([[[-1.9975, -1.9610],
[ 0.9592, -2.3364]],
[[-1.2534, -1.3429],
[ 0.4153, -1.4664]]])
>>> torch.matrix_power(a, 3)
tensor([[[ 3.9392, -23.9916],
[ 11.7357, -0.2070]],
[[ 0.2468, -6.7168],
[ 2.0774, -0.8187]]])
torch.
matrix_rank
(input, tol=None, bool symmetric=False) → Tensorsymmetric
is True
, then input
is assumed to be symmetric, and the computation of the rank is done by obtaining the eigenvalues.tol
is the threshold below which the singular values (or the eigenvalues when symmetric
is True
) are considered to be 0. If tol
is not specified, tol
is set to S.max() * max(S.size()) * eps
where S is the singular values (or the eigenvalues when symmetric
is True
), and eps
is the epsilon value for the datatype of input
.
None
input
is symmetric. Default: False
>>> a = torch.eye(10)
>>> torch.matrix_rank(a)
tensor(10)
>>> b = torch.eye(10)
>>> b[0, 0] = 0
>>> torch.matrix_rank(b)
tensor(9)
torch.
mm
(input, mat2, out=None) → Tensorinput
and mat2
.input
is a (n×m)(n \times m)(n×m) tensor, mat2
is a (m×p)(m \times p)(m×p) tensor, out
will be a (n×p)(n \times p)(n×p) tensor.torch.matmul()
.
>>> mat1 = torch.randn(2, 3)
>>> mat2 = torch.randn(3, 3)
>>> torch.mm(mat1, mat2)
tensor([[ 0.4851, 0.5037, -0.3633],
[-0.0760, -3.6705, 2.4784]])
torch.
mv
(input, vec, out=None) → Tensorinput
and the vector vec
.input
is a (n×m)(n \times m)(n×m) tensor, vec
is a 1-D tensor of size mmm , out
will be 1-D of size nnn .
>>> mat = torch.randn(2, 3)
>>> vec = torch.randn(3)
>>> torch.mv(mat, vec)
tensor([ 1.0404, -0.6361])
torch.
orgqr
(input, input2) → Tensortorch.geqrf()
.
torch.geqrf()
.torch.geqrf()
.torch.
ormqr
(input, input2, input3, left=True, transpose=False) → Tensorinput3
) by the orthogonal Q matrix of the QR factorization formed by torch.geqrf()
that is represented by (a, tau) (given by (input
, input2
)).
torch.geqrf()
.torch.geqrf()
.torch.
pinverse
(input, rcond=1e-15) → Tensorsvd()
for more details.
input
of dimensions n×mn \times mn×m>>> input = torch.randn(3, 5)
>>> input
tensor([[ 0.5495, 0.0979, -1.4092, -0.1128, 0.4132],
[-1.1143, -0.3662, 0.3042, 1.6374, -0.9294],
[-0.3269, -0.5745, -0.0382, -0.5922, -0.6759]])
>>> torch.pinverse(input)
tensor([[ 0.0600, -0.1933, -0.2090],
[-0.0903, -0.0817, -0.4752],
[-0.7124, -0.1631, -0.2272],
[ 0.1356, 0.3933, -0.5023],
[-0.0308, -0.1725, -0.5216]])
torch.
qr
(input, some=True, out=None) -> (Tensor, Tensor)input
, and returns a namedtuple (Q, R) of tensors such that input=QR\text{input} = Q Rinput=QR with QQQ being an orthogonal matrix or batch of orthogonal matrices and RRR being an upper triangular matrix or batch of upper triangular matrices.some
is True
, then this function returns the thin (reduced) QR factorization. Otherwise, if some
is False
, this function returns the complete QR factorization.input
are large
True
for reduced QR decomposition and False
for complete QR decomposition.input = torch.matmul(Q, R)
. The dimensions of Q and R are (∗,m,k)(*, m, k)(∗,m,k) and (∗,k,n)(*, k, n)(∗,k,n) respectively, where k=min(m,n)k = \min(m, n)k=min(m,n) if some:
is True
and k=mk = mk=m otherwise.>>> a = torch.tensor([[12., -51, 4], [6, 167, -68], [-4, 24, -41]])
>>> q, r = torch.qr(a)
>>> q
tensor([[-0.8571, 0.3943, 0.3314],
[-0.4286, -0.9029, -0.0343],
[ 0.2857, -0.1714, 0.9429]])
>>> r
tensor([[ -14.0000, -21.0000, 14.0000],
[ 0.0000, -175.0000, 70.0000],
[ 0.0000, 0.0000, -35.0000]])
>>> torch.mm(q, r).round()
tensor([[ 12., -51., 4.],
[ 6., 167., -68.],
[ -4., 24., -41.]])
>>> torch.mm(q.t(), q).round()
tensor([[ 1., 0., 0.],
[ 0., 1., -0.],
[ 0., -0., 1.]])
>>> a = torch.randn(3, 4, 5)
>>> q, r = torch.qr(a, some=False)
>>> torch.allclose(torch.matmul(q, r), a)
True
>>> torch.allclose(torch.matmul(q.transpose(-2, -1), q), torch.eye(5))
True
torch.
solve
(input, A, out=None) -> (Tensor, Tensor)
>>> A = torch.tensor([[6.80, -2.11, 5.66, 5.97, 8.23],
[-6.05, -3.30, 5.36, -4.44, 1.08],
[-0.45, 2.58, -2.70, 0.27, 9.04],
[8.32, 2.71, 4.35, -7.17, 2.14],
[-9.67, -5.14, -7.26, 6.08, -6.87]]).t()
>>> B = torch.tensor([[4.02, 6.19, -8.22, -7.57, -3.03],
[-1.56, 4.00, -8.67, 1.75, 2.86],
[9.81, -4.09, -4.57, -8.61, 8.99]]).t()
>>> X, LU = torch.solve(B, A)
>>> torch.dist(B, torch.mm(A, X))
tensor(1.00000e-06 *
7.0977)
>>> # Batched solver example
>>> A = torch.randn(2, 3, 1, 4, 4)
>>> B = torch.randn(2, 3, 1, 4, 6)
>>> X, LU = torch.solve(B, A)
>>> torch.dist(B, A.matmul(X))
tensor(1.00000e-06 *
3.6386)
torch.
svd
(input, some=True, compute_uv=True, out=None) -> (Tensor, Tensor, Tensor)(U, S, V)
which is the singular value decomposition of a input real matrix or batches of real matrices input
such that input=U×diag(S)×VTinput = U \times diag(S) \times V^Tinput=U×diag(S)×VT .some
is True
(default), the method returns the reduced singular value decomposition i.e., if the last two dimensions of input
are m
and n
, then the returned U and V matrices will contain only min(n,m)min(n, m)min(n,m) orthonormal columns.compute_uv
is False
, the returned U and V matrices will be zero matrices of shape (m×m)(m \times m)(m×m) and (n×n)(n \times n)(n×n) respectively. some
will be ignored here.U.contiguous().transpose(-2, -1).stride()
input
is full rank with all distinct singular values. Otherwise, NaN
can appear as the gradients are not properly defined. Also, notice that double backward will usually do an additional backward through U and V even if the original backward is only on S.some
= False
, the gradients on U[..., :, min(m, n):]
and V[..., :, min(m, n):]
will be ignored in backward as those vectors can be arbitrary bases of the subspaces.compute_uv
= False
, backward cannot be performed since U and V from the forward pass is required for the backward operation.
>>> a = torch.randn(5, 3)
>>> a
tensor([[ 0.2364, -0.7752, 0.6372],
[ 1.7201, 0.7394, -0.0504],
[-0.3371, -1.0584, 0.5296],
[ 0.3550, -0.4022, 1.5569],
[ 0.2445, -0.0158, 1.1414]])
>>> u, s, v = torch.svd(a)
>>> u
tensor([[ 0.4027, 0.0287, 0.5434],
[-0.1946, 0.8833, 0.3679],
[ 0.4296, -0.2890, 0.5261],
[ 0.6604, 0.2717, -0.2618],
[ 0.4234, 0.2481, -0.4733]])
>>> s
tensor([2.3289, 2.0315, 0.7806])
>>> v
tensor([[-0.0199, 0.8766, 0.4809],
[-0.5080, 0.4054, -0.7600],
[ 0.8611, 0.2594, -0.4373]])
>>> torch.dist(a, torch.mm(torch.mm(u, torch.diag(s)), v.t()))
tensor(8.6531e-07)
>>> a_big = torch.randn(7, 5, 3)
>>> u, s, v = torch.svd(a_big)
>>> torch.dist(a_big, torch.matmul(torch.matmul(u, torch.diag_embed(s)), v.transpose(-2, -1)))
tensor(2.6503e-06)
torch.
symeig
(input, eigenvectors=False, upper=True, out=None) -> (Tensor, Tensor)input
or a batch of real symmetric matrices, represented by a namedtuple (eigenvalues, eigenvectors).input
such that input=Vdiag(e)VT\text{input} = V \text{diag}(e) V^Tinput=Vdiag(e)VT .eigenvectors
defines computation of both eigenvectors and eigenvalues or eigenvalues only.False
, only eigenvalues are computed. If it is True
, both eigenvalues and eigenvectors are computed.input
is supposed to be symmetric, only the upper triangular portion is used by default.upper
is False
, then lower triangular portion is used.NaN
can appear as the gradients are not properly defined.
eigenvectors=False
, it’s a tensor filled with zeros. Otherwise, this tensor contains the orthonormal eigenvectors of the input
.>>> a = torch.randn(5, 5)
>>> a = a + a.t() # To make a symmetric
>>> a
tensor([[-5.7827, 4.4559, -0.2344, -1.7123, -1.8330],
[ 4.4559, 1.4250, -2.8636, -3.2100, -0.1798],
[-0.2344, -2.8636, 1.7112, -5.5785, 7.1988],
[-1.7123, -3.2100, -5.5785, -2.6227, 3.1036],
[-1.8330, -0.1798, 7.1988, 3.1036, -5.1453]])
>>> e, v = torch.symeig(a, eigenvectors=True)
>>> e
tensor([-13.7012, -7.7497, -2.3163, 5.2477, 8.1050])
>>> v
tensor([[ 0.1643, 0.9034, -0.0291, 0.3508, 0.1817],
[-0.2417, -0.3071, -0.5081, 0.6534, 0.4026],
[-0.5176, 0.1223, -0.0220, 0.3295, -0.7798],
[-0.4850, 0.2695, -0.5773, -0.5840, 0.1337],
[ 0.6415, -0.0447, -0.6381, -0.0193, -0.4230]])
>>> a_big = torch.randn(5, 2, 2)
>>> a_big = a_big + a_big.transpose(-2, -1) # To make a_big symmetric
>>> e, v = a_big.symeig(eigenvectors=True)
>>> torch.allclose(torch.matmul(v, torch.matmul(e.diag_embed(), v.transpose(-2, -1))), a_big)
True
torch.
trapz
()torch.
trapz
(y, x, *, dim=-1) → Tensor
>>> y = torch.randn((2, 3))
>>> y
tensor([[-2.1156, 0.6857, -0.2700],
[-1.2145, 0.5540, 2.0431]])
>>> x = torch.tensor([[1, 3, 4], [1, 2, 3]])
>>> torch.trapz(y, x)
tensor([-1.2220, 0.9683])
torch.
trapz
(y, *, dx=1, dim=-1) → Tensor
torch.
triangular_solve
(input, A, upper=True, transpose=False, unitriangular=False) -> (Tensor, Tensor)out
keyword only supports 2D matrix inputs, that is, b, A must be 2D matrices.
True
.False
.False
.>>> A = torch.randn(2, 2).triu()
>>> A
tensor([[ 1.1527, -1.0753],
[ 0.0000, 0.7986]])
>>> b = torch.randn(2, 3)
>>> b
tensor([[-0.0210, 2.3513, -1.5492],
[ 1.5429, 0.7403, -1.0243]])
>>> torch.triangular_solve(b, A)
torch.return_types.triangular_solve(
solution=tensor([[ 1.7841, 2.9046, -2.5405],
[ 1.9320, 0.9270, -1.2826]]),
cloned_coefficient=tensor([[ 1.1527, -1.0753],
[ 0.0000, 0.7986]]))