Element-wise square root of a MPS

Hi Miles,

I wonder whether there is a way to take an element-wise square root of a MPS? Namely, given a non-negative tensor \psi_{i1,\dots,iN}, I want to obtain a new tensor with the element \psi'_{i1,\dots,iN}= \sqrt{\psi_{i1,\dots,iN}}.

This is actually a follow-up question to my previous question: Compute the sum of norm of an MPO

Currently, I have some thoughts, but I am not sure whether this is the optimal/right way. Therefore, your comments would be greatly appreciated.

My ultimate goal is to compute the sum of the norm of a (pure state) density matrix \rho, i.e., \sum_{ij}|{\rho_{ij}}|.

The summation can actually be done by contracting with an “all-one tensor.” So the problem boils down to how to find the tensor |{\rho_{ij}}|.

Since in my problem, \rho comes from a pure state, i.e., \rho =|\psi\rangle\langle\psi|.
Assume |\psi\rangle = [a_1,a_2,\dots, a_i, \dots]^\intercal , clearly, |\rho_{ij}| = |{a_i}| |{a_j}|.
This is to say, if I can find a way to convert the MPS of |{\psi}\rangle to a |{\psi}'\rangle, where the “phase of each component is dropped”, then I can just construct an outer product of |{\psi'}\rangle, and take the sum of all the elements.
Then, this problem boils down to how to “apply a norm to an MPS.”
(assume the new MPS can be represented by MPS, even if not, I still want to find an algorithm that will be exact if allowing exponential growth of bond dim, such that I can later do some truncation to approach the exact results)

In order to do so, the norm of a is actually just \sqrt{a_i a_i^*}, which can be further broken into 2 steps:

  1. Element-wise product of psi and conj(psi), which we call psi_abs2
  2. Element-wise square root of psi_abs2.

The first step can be obtained by the element-wise product as follows

function elementwise_product(mps1::MPS, mps2::MPS;cutoff::Float64=1e-10,maxdim::Int=25,method::String="densitymatrix")
    site_idx=siteinds(mps1)
    mpo1=MPO(site_idx)
    for i in 1:length(mps1)
        mpo1[i]=mps1[i]* delta(site_idx[i],prime(site_idx[i],1),prime(site_idx[i],2))
    end
    mps_prod=apply(mpo1,prime(mps2;tags="Site"),method=method,cutoff=cutoff,maxdim=maxdim)
    return noprime!(mps_prod)
end

following Sec. IIIB in https://journals.aps.org/prx/abstract/10.1103/PhysRevX.13.021015

My attempt for the second step is as follows:

a_0=S
c_0=S-1
a_{n+1}=a_n-a_n c_n/2
c_{n+1}=c_n^2(c_n-3)/4

Here, we take S to be the MPS we want to take the square of. Namely, S=psi_abs2, and then iterate a_i, until it reaches convergence (e.g., the update is smaller than a threshold)
Then the final tensor a_n is the square root of the tensor S.

The iterative method is from Methods of computing square roots - Wikipedia
Mathematically, this method converges given 0<S<3, which is naturally satisfied here as the amplitude of each component cannot be larger than 1.

Because each iteration rule here only involves “element-wise product” and “add/minus a constant MPS,” it, in principle, can be efficiently performed.

My minimal implementation is

function sqrt_mps(mps::MPS;eps::Float64=1e-5,maxiter::Int=100,cutoff::Float64=1e-10,maxdim::Int=20)
    L=siteinds(mps)
    allone=CT.all_one_mps(siteinds(mps))
    a=mps
    c=mps-allone
    diff_a = 100
    while diff_a > eps && maxiter > 0
        a_new = -(a,elementwise_product(a,c,cutoff=cutoff,maxdim=maxdim)/2,cutoff=cutoff,maxdim=maxdim)
        truncate!(a_new)
        c = elementwise_product(elementwise_product(c,c, cutoff=cutoff,maxdim=maxdim),-(c,3*allone,cutoff=cutoff,maxdim=maxdim), cutoff=cutoff,maxdim=maxdim)/4
        truncate!(c)
        diff_a_mps=a-a_new
        diff_a = abs(inner(conj(diff_a_mps),diff_a_mps))
        a=a_new
        maxiter -= 1
    end
    return a,c
end

function all_one_mps(sites::Vector{Index{Int64}})
    mps=MPS(sites)
    for i in 1:length(mps)
        mps[i]=ITensor([1,1],sites[i])
    end
    return mps
end

I experimented on several examples and found that for random MPS with bond dim ~6 and system size ~ 12. this can give you an approximation of the “element-wise norm of an MPS.”
However, I do also find numerical stability to be an issue— sometimes, when the amplitude of the wavefunction is too small, c_i could numerically diverge, and therefore ruin the next a_{i+1}.
Another thing is that I found the memory allocation is very large in my code, which I guess is due to the “element-wise product”, where it returns a new MPS each time. So, it also takes quite a long time to perform the iteration.

Do you think my method is feasible, or is there another better way?

In general, is it an open question to ask how to apply an arbitrary function f(x) to an MPS or QTT, and what is the function space such that the new MPS aftering the element-wise mapping is also representable by MPS?

In general, the best method in terms of flexibility and power that I know of right now for computing arbitrary functions of MPS and loading them into other MPS is tensor cross interpolation (TCI). I had meant to link you to a code you could use for this in another question you asked on here – sorry that I never got back to you about that.

Here is a publicly available code for TCI:
https://github.com/tensor4all/TensorCrossInterpolation.jl

It also includes some functions for converting the results into ITensors so that you can load them into an ITensor MPS object (or I think it might already convert to an ITensor MPS object).

In general, TCI can learn an MPS from any function, and you get to code what the function is yourself. You can therefore code the function to take values from a prior MPS, then take the square root of these values before returning them out of the function. Let me know if that could work for you!

Hi Miles,

Thanks for your reply! If I understand correctly, do you mean I can first obtain each “tensor element” of the original tensor, then manually take a square root, and finally use TCI to construct a new MPS whose “tensor element” is the sqrt of the original ones?

If so, wouldn’t the step to code the function to take values from a prior MPS take exponentially large steps? Do I misunderstand something here?

I’m not sure what you mean by take exponentially large steps. Do you mean have an exponential cost?

The way the code would use the prior MPS would be that the TCI code would ask for a value, say the (1,2,1,1,2) value of the square root of the prior MPS. Then in the code you would implement, you would first compute the (1,2,1,1,2) value of the prior MPS then take its square root.

Computing a specific value of the MPS, meaning for a specific setting of its indices, has a cost of n\chi^2 for an MPS of n sites and bond dimension \chi. Here is a code example showing how to do it:
https://itensor.github.io/ITensors.jl/dev/examples/MPSandMPO.html#Obtaining-Elements-of-a-Tensor-Represented-by-an-MPS

1 Like

Hi Miles,

Thanks for your reply! I intended to say that, obtaining “all” the elements requires exponentially large steps. That is, in order to get the mapping f: x_{original} \rightarrow x_{new} for all x_{original}, wouldn’t I need to query all the combinations of (i1,i2,i3,i4,i5), which is exponentially large?

Tensor cross interpolation performs a kind of ‘machine learning’ algorithm, so it is able to learn a function taking n variables by making only about n\chi^2 calls to the function, where \chi is the final bond dimension needed to represent that function accurately in MPS form. So it will not call your function exponentially many times, unless the function cannot be compressed (i.e. if the square root state happens to have an exponentially large bond dimension).

Unfortunately for your case, since your function will take \chi^2 work to call each time, and will be called about \chi^2 times, that’s an overall \chi^4 scaling which is not ideal (\chi^3 or better is usually ideal). There might be ways to fix this using ‘caching’ inside your function.

For more information about TCI you can see this recent paper:
https://arxiv.org/abs/2407.02454

Thanks Miles! I have tried a couple of examples and it works (with a given maximal bond dimension).

Glad to hear it Haining! I would be interested to hear more details in person.

I have been making some small but nice improvements to the (currently private) ITensor-based TCI codes which could help. But I forgot which one are you using? The publicly available one by Marc Ritter?

Hi Miles,

Sure! I’m happy to meet you in person. I’ll be at CCQ this entire week (08/05 - 08/09) except for Tuesday.

For my code, I used the TensorCrossInterpolation.
The previous TCI package that you shared with me is called QTTOperations.jl

Best,
Haining

1 Like

Good choice – the QTTOperations code is coming along but I would consider it more “experimental” right now, and TensorCrossInterpolation is more mature. We will actually be releasing a more general “ITensorTCI” algorithm later on which can perform TCI on arbitrary tree tensor networks (including MPS as a special case).

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.