I’m not quite following you’re response, since the output you printed doesn’t show an error. I’m also not following what you mean by:
Could you try to rephrase that so I can understand what you ran that caused an error? It would also help to know which model you are running, what symmetries you tried to conserve, etc.
My best interpretation is that you saw VUMPS run out of memory when you were running it with a bond dimension fixed at 928. One of my colleagues successfully ran VUMPS for the extended Hubbard model conserving particle number and spin projection symmetry for bond dimensions above 2000, so it’s definitely possible to reach higher bond dimensions than the one you seem to be using, but of course the bond dimensions you can reach before running out of memory can depend on the symmetries you conserve, the memory limit of the node you are using, if there are other calculations being run on the same node that are taking up memory, etc.
Thanks for your reply and sorry for not clarifying my question.
There is no problem in the output when running a vumps on a node and it gives me the expected energy.
But it uses all the RAM a node has, which is much bigger than it actually needs. And I find this problem from the fact that it takes 80G to run dimensions at 928 and ~2500 and ~5000. This takes almost all the RAM my node has.
That may not be so friendly for running code on shared computation resources.
Anyway it is acceptable and I can get the new information
Hi @L_Zhang,
Regarding the RAM usage, it’s something for us to keep in mind, to know if perhaps we can alter the code in the future so that RAM does not get consumed as highly.
Thanks for the clarification. We are working on rewriting the package to use a more compact representation of the infinite MPO, if you are using InfiniteSum{MPO} it is probably using more memory than necessary. I think there are also a few places where we are caching more tensors than is necessary, and we could use fancier features that we have in the DMRG code in ITensors.jl like writing tensors to disk to decrease peak RAM usage. Unfortunately I just haven’t had a lot of time to develop that package recently, if you want to investigate memory issues please feel free and I will consider merging high quality PRs.
Just want to let Miles @miles and Matthew @mtfishman know that I find ITensors.jl dmrg also uses a lot of more RAM than expected in Julia 1.9.1, if without using --heap-size-hint command!
Check the following example, %MEM usage comparison for two identical jobs:
Thanks for sharing this finding. We will continue to advertise the --heap-size-hint feature since it seems to be very helpful. It may be that ITensors.jl dmrg is using a lot of ram for some reason, but it could also just be Julia itself especially when doing multithreading. (I.e. the computations happen very quickly before the garbage collector decides to act.)
We do have one major improvement planned for our tensor contraction layer of ITensors.jl (actually NDTensors.jl underneath) that ought to help to reduce ram consumption (technically reduce the number of allocations that happen).
Yes, I indeed find that the memory consumption increases with the number of multithreading. And this memory usage is still unacceptable even using --heap-size-hint.
Currently, I have to fix to a small number of multithreading to prevent Out-Of-Memory error. Hope your major improvement on NDTensors.jl will solve this problem
