About broadening

[Confusio]

PEPSKit.jl/src/utility/svd.jl

Lines 442 to 448 in 7cfae1a

	function _safe_inv(x, tol, ε=0)
	if abs(x) < tol
	return zero(x)
	else
	return iszero(ε) ? inv(x) : _lorentz_broaden(x, ε)
	end
	end

In the above lines, when the singular values are degenerate, the inv is set to zero. When they are not degenerate, the broadening is applied. Is this the purpose for broadening? I think it is in the opposite.

[pbrehmer]

Thanks for bringing that up! I think you're right that when the singular values are degenerate, such that x is approximately zero, we want to apply broadening instead of setting it to zero, which we don't do here - that should definitely be fixed.

On the other hand, we also want to catch the case where singular values might be really small such that taking the inverse directly would be a bad idea. So perhaps we would need a separate function for taking inverses with broadening that only applies to singular value differences.

I'm wondering, if it would make a difference to only apply broadening to the singular value differences that fall below a certain tolerance. I thought that applying the broadening to singular values that are not degenerate will only have a small or negligible effect. @lkdvos What do you think?

[Yue-Zhengyuan]

In the above lines, when the singular values are degenerate, the inv is set to zero

This is only when all singular values are very small, as Paul said. I think this is effectively doing truncation of the SVD when changing the shape/size of the SVD result tensors is not convenient.

Maybe there's an improvement: there can be cases where only part of elements of x are very small. In this case we only set small elements in x to zero, calculate inv (and possibly broadening) for the not-small part, and finally combine these two parts.

[lkdvos]

There might also be some implementation detail here: we are abusing this function to simply take _safe_inv.(S' .- S), counting on the fact that the diagonal entries are zero and should remain zero, and only the off-diagonal entries are relevant. I would have to check in more detail how this function is used throughout the rest of the code though

[pbrehmer]

I updated the broadening in #194 although I found that, at least for random matrices, using a cutoff to set the divergent inverse singular value differences to zero kind of does the same thing as broadening. Anyways, I think the broadening should be improved through that PR.

[Confusio]

This makes sense as I think if the matrix is not so singular, the broadening has no effect. If it is singular, we need a more accurate svd.

[Confusio]

As a follow up suggestion, during the ctmrg, can we first use some fast svd to get a env, then switch to sdd for a better 'env' and finally 'svd'? The reason of 'svd' is for the improvement of the gradient.

[pbrehmer]

That is an interesting suggestion. Although I do think that for many purposes the sdd algorithm should be sufficient since we often don't need CTMRG gradients up to such a high precision that the svd would be a significant improvement. Of course there are pathological cases where sdd might fail but I don't think these are super common? I'll keep the suggestion in mind but for now I'll have to work on something else :-)

In any case, #194 is now merged so that should hopefully take care of the Lorentzian broadening issue.