Interface discussion

[Jutho]

@lkdvos , maybe we can discuss the interface here.

For the family of methods like qr, eigen and svd (possibly trunced) there needs to be support for:

• backends (LAPACK, CUSOLVER, KrylovKit, ...)

• algorithms supported by these backends (QRIteration, DivideAndConquer, RobustRepresentations, Jacobi,...)

• possible options/keywords to configure those algorithms

• possible keywords to control truncation? specific to svd?

Then there is something like polar? Do we want a specific method for polar decomposition. That could be computed via SVD (and thus need the previous levels of control), but there are also other packages (=backends?) that can do this without SVD.

If we also define methods like leftorth and friends here (this could also be restricted to TensorKit), they can be computed using e.g. either svd or qr. I don't know if there are other relevant strategies? So the question is how to convey that information? A strategy or kind arg/kwarg?

Something similar exists for matrix functions, like exponential, which can use things like Pade approximations, Schur-Parlett, diagonalisation in the case of hermitian matrices, ….

[Jutho]

I also forgot GenericLinearAlgebra.jl as backend. Am I correct that what you were proposing, @lkdvos , is to combine algorithms and backends in one name, something like LAPACK_QRIteration, … . That reminds me of what is done in LinearSolve.jl: https://docs.sciml.ai/LinearSolve/stable/solvers/solvers/

[lkdvos]

I think that indeed is very similar to what I had in mind. To me, the difference between backend and algorithm seems rather minimal, so I would just merge the two concepts. It almost sounds like what you describe as backend is simply just a family of algorithms that are found in the same package, but then it might as well be a type union or abstract type, if this is even needed.
I am very intrigued by the LinearSolve link, in the sense that it almost looks like what we need, except they don't actually expose the algorithms at the level of the factorizations. Scrolling through the available algorithms, it seems like there as well, there was demand for FastLAPACK - in-place results, which is interesting.
I quite like that the full is just a field in their SVD algorithm as well, that seems quite reasonable.

Just as an aside, this is what I had laying around locally, which was from before I looked at your most recent changes: https://github.com/QuantumKitHub/MatrixAlgebraKit.jl/tree/ld-interfaceproposal
It's definitely not finished, nor am I perfectly happy with the design, but it does have some of the aspects that I was thinking about:

• Having a struct to bundle the output, whose constructor simultaneously serves as "pre-allocating" the results
• Having structs to define the algorithms
• Having an interface consisting of svd(A; kwargs...), svd(A, alg), svd!(A, [F]; kwargs...) and svd!(A, F, alg), somewhat reminiscent of KrylovKit, with a default algorithm selector (which is more or less equivalent to the default_backend, which I had somehow missed)
• After talking to you: it is unclear how factorization structs would interact with ChainRules, as this sounds like a mess, so before we can decide on that, this needs to be checked.

Something that I had been thinking about, specifically for svd, is whether or not it would make sense to have something like:

struct TruncatedSVD{A,T}
    svd_alg::A
    trunc_alg::T
end

and just treat truncated svd on the exact same footing as svd. Of course, depending on the algorithm being Krylov-based or LAPACK, the implementation might vary quite a bit, but at least the interface would be the same.