I am using DMRG to calculate a 6x40 lattice with maxdim=2000, but I found that the maximum memory consumption during the calculation has reached 700 GB, far exceeding the theoretically expected memory. MPS is approximately 13GB. Meanwhile I have writen it into disk. Where in the calculation process does it require a large amount of memory?
Sorry to hear this. Have you tried the --heap-size-hint flag? For that and other tips about RAM usage with ITensor please see this FAQ page in the documentation:
https://itensor.github.io/ITensors.jl/dev/faq/HPC.html
Yes I have used this to limit Julia’s memory usage, but it did not work very effectively.
Sorry to hear that. Was the high memory usage causing your runs to crash or stop early?
Thanks for your attention. Yes, the high memory usage was causing my runs to stop prematurely. I noticed that in the initial sweeps, his memory usage was not high, but as the number of sweeps increased, the memory usage gradually went up. I did not alter my bond truncation dimension during this process. And each mps has reached its maxbond. Does this indicate that the program has accumulated a lot of memory during runtime without releasing it? Should I forcibly release some garbage memory at a suitable location?
Yes we are finding this issue too - Julia accumulates memory during runtime. You can see some benchmarking of this issue on in this post Memory usage in DMRG with Julia 1.x - #7 by mtfishman. We are working on creating a memory buffer system to minimize this problem and improve performance. I think the recommended solution is still the --heap-size-hint flag but you can follow Matt’s thread for updates.
Thanks for your reply. I will try it.