Improve normalizations during simple update

[Yue-Zhengyuan]

This is a minor improvement to properly normalize the tensors during simple update.

Previously, after absorbing the weights into the vertex tensors, the norm of the vertex tensors may become too small, causing issues for the following SVD truncation (the singular values can be too small as well).

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines	Coverage Δ
src/algorithms/time_evolution/simpleupdate.jl	100.00% <100.00%> (ø)
src/algorithms/time_evolution/simpleupdate3site.jl	100.00% <100.00%> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Yue-Zhengyuan]

The gradient tests have been running out of time quite frequently in recent GitHub actions. We may need to further investigate it?

[lkdvos]

While I can see that this is useful, I'm a bit hesitant to enforce this, mostly because it would prevent us from using this for finite temperature results since the norm is important there. Do you think we can just provide an easy way to manually ensure the norm doesn't run away between simple update steps?

[Yue-Zhengyuan]

Do you mind informing me how finite-T calculations use the norm (and what kind of norm it is using)? Here I'm setting the maximal abs value in each vertex tensor and each bond weight to 1, which shouldn't affect any physical observables of the state.

[lkdvos]

The problem is mostly that for computing the free energy, you need to know the norm, or at least the change in norm between two steps. For actual time evolution it is fair to discard this information because you want a normalized state at the end, but for finite temperature it has some meaning (it is the partition function value, connected to the free energy)

[Yue-Zhengyuan]

How about I also return the number used to normalize the PEPS in each 2/3-site SU iteration? But to test it's done correctly, can you provide an example where free energy needs to be extracted?

[Yue-Zhengyuan]

@sanderdemeyer Do you have suggestions on handling normalization for finite-T simple update?

[lkdvos]

Sorry for being slightly slow to get back to you on this. I think the main issue is that the choice of Inf norm is somewhat arbitrary, and this might not be the one you want.
Given this, what do you think about adding a keyword argument p::Union{Real,Nothing}=1 that selects between truncating and not truncating, and selects which norm? I am not sure if it is best in the algorithm struct or as separate keyword argument.

[Yue-Zhengyuan]

adding a keyword argument p::Union{Real,Nothing}=1

Good idea. I'll flesh it out when I get some time. For imaginary time evolution, normalization is inevitable since the norm of $e^{-\beta H}$ can be large as $\beta$ increases.

[lkdvos]

Absolutely, you always have to renormalize but it is important to keep track of the norm in that case. I think typically, one computes log(N) along the way and simply add them together, but there are definitely different valid strategies. I would say that it is just important to:
A. have an easy way to normalize, with whatever norm we choose
B. have this information not lost in some inner part of the code.

Anything that achieves that is good for me :)

[Yue-Zhengyuan]

I increasingly feel the difficulty in writing a simpleupdate that suits all different purposes:

Purpose	Imag/Real time	Convergence test	Intermediate state	Normalization matters
Ground state search	Imag	Yes	No	No
Real time dynamics	Real	No	Yes	No
Finite-T density operator	Imag	No	No	No
Free energy (at many T)	Imag	No	Yes	Yes

In the last two cases, in the future I think we'll use iPEPS with 2 physical legs (physical + ancilla; though I'm not sure how much improvement this can achieve), further differentiating them from the first two cases. In addition, if sometimes I return the norm, and sometimes I don't, I guess this cause "type instability"? So I want to:

• Making the current simpleupdate an internal function for test purposes, and not export it.
• Ask end users to build their own SU pipeline from the most basic su_iter; I can make it always return the normalization used in each step.
• If we don't provide simpleupdate, then in the algorithm struct SimpleUpdate, the convergence criterion tol can be removed.

Though there are some things that only need to be done once at the beginning, and are cumbersome to be moved into su_iter:

• create the evolution gate $exp(-\epsilon H)$;
• auto dispatch to 2-site or 3-site version of SU.

@lkdvos @sanderdemeyer Do you have any design suggestions? (In full update #222 we have a similar situation.)

[sanderdemeyer]

We could define a struct that takes into account the differences. Then the user can set dt to real or imaginary depending on whether we are performing real-time or imaginary-time evolution, set tol = 0.0 if we don't want a convergence criterion, an extra field denoting the type of normalization, and a boolean denoting whether we want to remember these norms.

Another option would be to have subtypes of the struct SimpleUpdate, e.g. SimpleUpdateGroundState etc... (name to be decided). We could define this as subtypes or just constructors of the more general SimpleUpdate struct that I mentioned above that take into account whether we want to normalize, whether it is real or imaginary, etc... In the former case, we could dispatch functions like _simpleupdate2site on this subtype. There will probably some code duplication, but I think with some smart choices we can keep this to a minimum. I think this is a bit cleaner than having to define you're own pipeline and also avoids the type instability. That way, you also don't have to move the dispatch to two-site or three-site SU and the construction of the gate to su_iter.

[lkdvos]

Let me elaborate a bit on what we are doing in MPSKit, this might help us find a better interface (and also, it might be nice to simply use the same functions and interface?)

MPSKit.timestep!(state, H, t, dt, alg, envs; kwargs...) -> state is the function that takes a state at time t and evolves it to t + dt.
alg can be used to dispatch and control the parameters, and envs contain caches of precomputed data.

Importantly, this function does only that: it evolves the state according to the arguments it gets. This can then be used to build more evolved pipelines, either for imaginary or real time evolution purposes.

Then, we have MPSKit.time_evolve(state, H, t_span, alg, envs; kwargs...) -> state that now can be used to take a state at t_span[1] and evolve it in steps from t_span[i-1] to t_span[i]. This calls timestep for each step.
From what I know, this function is not used that much, mostly because as you mentioned, the requirements change quite a bit depending on what you are trying to do. For real time evolution, typically you might actually be interested in the intermediate states, while for imaginary time evolution this is less the case. For finite-T results you also want access to the intermediate steps.

Note also that the algorithm struct TDVP does not contain things like tol and maxiter for the total time evolution, and is built with timestep! in mind.

---

Given this, I would argue to replace su_iter with timestep(state, H, t, dt, ::SimpleUpdate, ...) = timestep(state, gate, ::SimpleUpdate) and indeed remove most of the current properties from that struct. In particular, it should probably just contain the SVD algorithm used for the truncation along with the truncation scheme.
This function should do precisely what it says: a single timestep without anything else. (and no normalization!). It should be able to take in either the gate or the hamiltonian and dt for convenience.

We may still build some default time_evolve pipeline on top of this, which could indeed normalize the state after every timestep. In principle I'd also be okay with having a p::Union{Nothing,Real} in the SimpleUpdate struct to enable/disable normalization within that algorithm, but I think I would prefer that this is not the case, and have a more verbose: state = normalize(timestep(state, gate, alg)) kind of approach, which would also mean that the user has to decide whether or not they care about the information of the norm.

---

Something I have been playing around with as well is having a imaginary_evolution flag in the keyword arguments to indicate whether or not we are solving exp(iHdt) or exp(Hdtau). The advantage is that this tends to more easily allow the use of real numbers for imaginary time evolution, which otherwise becomes a bit more work on our end because we'd have to check if t and dt are purely real, purely imaginary, or complex.
I know that t will not really be used in our current system, but I would still keep it in there as a dummy argument, just to not have to change the interface in the future.

---

In summary, I think it is probably best to separate out the doing one step and full pipeline, and both should be exposed because people do really need both. Making SimpleUpdate be the struct that controls how to do a single step, and then using the kwargs in time_evolve to control other properties seems like a good solution.

[Yue-Zhengyuan]

That's very illuminating indeed! It's also kind of similar to what Olivier once suggested. But that may require a big overhaul of what we input to SU.

One thing that's been in my mind for a while is to get rid of InfiniteWeightPEPS (since it's rotation is quite annoying already), and replace it with InfinitePEPS + SUWeight (serving as the env for SU). Each tensor in this InfinitePEPS is then always with sqrt(bond weight) absorbed into it.

If we want to be more ambitious, SUWeight should be replaced by BPEnv as well, after we figure out how to efficiently transform the message matrices to diagonal singular value spectrums for each bond (#223; or, is this translation necessary if we want to extend SU with loop expansions later?). This is the approach of YASTN, which is added after I wrote SU for PEPSKit.

After these changes to SU, the interface can be more easily ported to full update by just changing env to the CTMRGEnv,.

[lkdvos]

I think I agree with this indeed. I'd propose to do this in different steps though, and not all at the same time to make it a bit more comprehensive

[Yue-Zhengyuan]

Sure. The BP things can be done later, and we can use SUWeight as env for now. But currently I'm busy learning about Gaussian fPEPS, and cannot work on SU for a while.

[Yue-Zhengyuan]

To be continued in another PR.