I am trying to set up a new Julia environment to do some GPU testing on my university’s HPC and I am encountering an issue I’ve never seen before. For reference, I am using Julia 1.10.
I create a new environment by doing ] activate --temp and try to do add ITensors. Pkg finds version 0.3.66 and installs it, but then it fails with the following error on precompilation.
julia> err
PkgPrecompileError: The following 1 direct dependency failed to precompile:
ITensors [9136182c-28ba-11e9-034c-db9fb085ebd5]
Failed to precompile ITensors [9136182c-28ba-11e9-034c-db9fb085ebd5] to "/PATH/TO/JULIA/.julia/compiled/v1.10/ITensors/jl_93BImO".
ERROR: LoadError: ArgumentError: Package KrylovKit [0b1a1467-8014-51b9-945f-bf0ae24f4b77] is required but does not seem to be installed:
- Run `Pkg.instantiate()` to install all recorded dependencies.
I am confused because I would have thought installing ITensor would have brought in all dependencies.
I also tried explicitly adding KrylovKiv by doing ] add KrylovKit@0.7 but then I get the following strange error. I have never seen this before either.
julia> err
PkgPrecompileError: The following 1 direct dependency failed to precompile:
KrylovKit [0b1a1467-8014-51b9-945f-bf0ae24f4b77]
Error: Missing source file for KrylovKit [0b1a1467-8014-51b9-945f-bf0ae24f4b77
Does anyone have any ideas what might be going on here or how to fix it? I am confused because I assumed a fresh environment would usually fix package issues. Is there a problem somehow in .julia/compiled/?
I am looking into the issue now, I will let you know. Can you send your Julia versioninfo() please and can you please send the exact script you are using? Thanks!
I have just ran the following command using Pkg; Pkg.activate(temp=true); Pkg.add("ITensors") and I found no issues on my mac M1
julia> versioninfo()
Julia Version 1.10.2
Commit bd47eca2c8a (2024-03-01 10:14 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: macOS (arm64-apple-darwin22.4.0)
CPU: 10 × Apple M1 Max
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-15.0.7 (ORCJIT, apple-m1)
Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores)
I also tried on our linux machines and see no Pkg issue. I have found that creating a fresh environment doesn’t always fix the problem. Can you try removing your registries via this post
Hey @kmp5, thank you for the quick response and taking a look at it. Here is what I get when I do versioninfo() on our login node.
Julia Version 1.10.0
Commit 3120989f39b (2023-12-25 18:01 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 12 × Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-15.0.7 (ORCJIT, haswell)
Threads: 1 on 12 virtual cores
If it matters, one thing I tried yesterday was to install packages while in an interactive session on a GPU node. I got some weird errors about curl not working or something? I assumed it was because the GPU node could not talk to the internet or something. Either way I ctrl+C that and exited the node, then added the packages on the login node and I didn’t get the error anymore. But then I get the separate errors I posted about yesterday. I have no idea if the two are related.
As far as exact script - I am doing everything interactively in the REPL. Here is the equivalent in a script form I think.
# mwe.jl
using Pkg
Pkg.add("ITensors")
I create a temporary environment and do include("mwe.jl"). ITensor and all its dependencies are added and then precompilation starts but fails with the messages I gave in yesterday’s post. See below.
I tried deleting and re-adding the registry per the link above but I still get the same error.
(jl_HqCJpQ) pkg> instantiate
0 dependencies successfully precompiled in 0 seconds. 109 already precompiled. 2 skipped during auto due to previous errors.
You could try deleting your ~/.julia directory, though make sure to make backups of folders like ~/.julia/dev and ~/.julia/config if you’ve been developing packages or have custom startup configurations. (This is considered a “nuclear option”: Danger of deleting .julia? - General Usage - Julia Programming Language but it seems like you’ve tried everything else, I’ve never seen an error message like the one you are seeing.)
Bob,
It looks like from your last post that the problem is no longer related to the Pkg manager and seems to be some precompiling issue. Can you please send the precompiling error trace?
On the cluster I went to ~/.julia/packages and did rm -rf KrylovKit.
Then I created a temporary environment and did add KrylovKit and it appeared to work but I got the following warning. I’m not sure what it means or why it occurs.
Precompiling project...
1 dependency successfully precompiled in 3 seconds. 8 already precompiled.
1 dependency had output during precompilation:
┌ KrylovKit
│ WARNING: using ChainRulesCore.add!! in module KrylovKit conflicts with an existing identifier.
└
Now I can compile itensors though, so that appears to be fixed.
Somehow something in the KrylovKit package directory got messed up? Any theories? So that I can avoid this in the future?