Description of issue : Repeated use of HDF5.write(parent::Union{HDF5.File, HDF5.Group}, name::AbstractString, M::MPS) or HDF5.read(parent::Union{HDF5.File, HDF5.Group}, name::AbstractString, ::Type{MPS}) on the same file object becomes increasingly slower with iterations, even when performing the exact same action. Issue does not depend trivially on file size. I stored MPOs of several GB without issues in the same file but encountered a crippling slowdown on a file of 40 MB storing MPS.
Workarounds : Closing and reopening the file in between each read makes access time consistent at the cost of file opening overhead. Manually calling GC.gc() after each read action also makes reading time consistent.
Suspected cause : I suspect this may be related to HDF5 group handles not being explicitly closed in the read/write implementations. Since they become out of scope, they will be closed when garbage collection occurs, but it is inconsistent. In most cases this does not cause issues, but when repeatedly accessing the same file in a loop, garbage collection can be held until the end of the loop, leading to an accumulation of handles which significantly slowdown further access to the file.
The fact that manually calling GC.gc()makes the behaviour consistent again is consistent with this hypothesis. Also, as explained above, I did not encounter the issue on files storing multiple GB of MPO, but these MPOs were large because of bond dimension (\chi \sim 1000), not because of the number of sites (around 30), while the file with very slow access stores MPS with very small bond dimension (\chi \sim 4) but more sites (258). This may indicate that the problem scales more strongly with the number of objects/groups opened during an MPS read than with the amount of data read.
I have not independently verified that the number of open HDF5 handles is increasing.
function HDF5.read(parent::Union{HDF5.File, HDF5.Group}, name::AbstractString, ::Type{MPS})
g = open_group(parent, name)
# Work ...
# No closing of the group before this point
return MPS(v, llim, rlim)
end
function HDF5.read(
parent::Union{HDF5.File, HDF5.Group}, name::AbstractString, ::Type{ITensor}; kwargs...
)
g = open_group(parent, name)
# Work ...
for key in ["storage", "store"]
if haskey(g, key)
# Work ...
# No closing of the group before this point
return itensor(storage, inds)
end
end
# Error case return here ...
end
Julia version info
Julia Version 1.12.6
Commit 15346901f0 (2026-04-09 19:20 UTC)
Build Info:
Official https://julialang.org release
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: 8 × Intel(R) Core(TM) i5-10210U CPU @ 1.60GHz
WORD_SIZE: 64
LLVM: libLLVM-18.1.7 (ORCJIT, skylake)
GC: Built with stock GC
Threads: 8 default, 1 interactive, 8 GC (on 8 virtual cores)
Package information
HDF5 v0.17.3
ITensorMPS v0.4.1
ITensors v0.9.30
Minimal reproducible example : I assume that a problematic file mps.h5 exists here as the issue appears consistently with the same data on the same machine, however I got inconsistent results from my limited testing when switching machines. I believe this is because automatic garbage collection is dependent on system state and not just Julia state. Consequently, I cannot be certain of how to generate problematic data on someone else’s machine. However, if my interpretation is correct, to maximize the probability of encountering the issue, the stored MPS should be defined over many sites while bond dimension should be irrelevant.
using ITensorMPS
using HDF5
# In this minimal example I am just loading the same data 4 times in a loop, but in actual code this will typically be an iterator over the different MPS stored inside the file
iterator = 1:4
h5open("mps.h5","r") do f
for i in iterator
@time dummy = read(f, "MPS", MPS)
# Calling GC.gc() here will make behaviour consistent again
end
end
Expected behaviour : read time should be identical for every iteration of the loop
Actual behaviour : in some cases, read time increases sharply with iterations.
Example output without GC.gc()
1.320790 seconds (127.44 k allocations: 4.843 MiB)
4.024463 seconds (127.44 k allocations: 4.880 MiB)
8.084418 seconds (127.44 k allocations: 4.890 MiB)
14.724404 seconds (127.44 k allocations: 4.891 MiB)
Example output with GC.gc()
1.194527 seconds (127.44 k allocations: 4.843 MiB)
1.130845 seconds (127.44 k allocations: 4.880 MiB)
1.172700 seconds (127.44 k allocations: 4.890 MiB)
1.212647 seconds (127.44 k allocations: 4.891 MiB)