norm1 - 1-norm of a vector or element-wise 1-norm of a matrix
norm2 - Euclidean norm of a vector or Frobenius norm of a matrix
tr, det, logdet, inv
There are different layout conventions (numerator layout, denominator layout, mixed layout). Numerator layout is just the transpose of the denominator layout and mixed layout is a mixture of both.
We use a mixed layout convention here. The resulting derivative is such that it can be used in a linear approximation of the function by forming a contraction along the corresponding last axes of the gradient. It is best illustrated by a few examples.
Only scalars, vectors, and matrices are displayed as output. If the derivative is a higher order tensor it will be computed but it cannot be displayed in matrix notation. Sometimes higher order tensors are represented using Kronecker products. However, this can be ambiguous in some cases. Here, only in unambiguous cases the result is displayed using Kronecker products. The python code still works on the true higher order tensors.