# Run on multiple GPUs

**URL:** <https://itensor.discourse.group/t/run-on-multiple-gpus/2626>\
**Category:** ITensor Julia Questions\
**Created:** [June 15, 2026, 6:04am UTC](https://itensor.discourse.group/t/run-on-multiple-gpus/2626 "2026-06-15T06:04:38Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![gc6020](https://avatars.discourse-cdn.com/v4/letter/g/ec9cab/32.png) [@gc6020](https://itensor.discourse.group/u/gc6020)\
**Post date:** [June 15, 2026, 6:04am UTC](https://itensor.discourse.group/t/run-on-multiple-gpus/2626/1 "2026-06-15T06:04:38Z")

</div>

Hello everybody,

I am running a tensor-network code using ITensors (Julia version) with GPUs (Nvidia A100) on a HPC cluster. I tried to parallelize it to use both GPUs available on a node, but I realized the code is actually only using one.

Is there a way to use both? I couldn’t find anything in the forum.

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![Joacop16](https://yyz2.discourse-cdn.com/free1/user_avatar/itensor.discourse.group/joacop16/32/560_2.png) [@Joacop16](https://itensor.discourse.group/u/Joacop16)\
**Post date:** [June 17, 2026, 6:53pm UTC](https://itensor.discourse.group/t/run-on-multiple-gpus/2626/2 "2026-06-17T18:53:24Z")

</div>

Hi!

I tried this two years ago but never found a way to make it work efficiently. I am not sure if maybe now, as the gpu implementation works better, there may be a way to do it (please let me now if you find one). Anyway, maybe some of what I learned during the process can be useful for you. **In principle, you can use the function `CUDA.device!()` to switch which GPU you are using.**

For example. if you have two devices and run `CUDA.devices()` your output will be something like:

```auto
CUDA.DeviceIterator() for 2 devices:
0. Tesla V100-SXM2-32GB
1. Tesla V100-SXM2-32GB

```

Then `CUDA.device!(0)` will choose the first one for usage and `CUDA.device!(1)` the second one. I never found a way to use both at the same time, but in principle you could use the first one until the memory is full and then switch to the second one (for example checking the memory during the sweeps of a DMRG code and use a custom observer for switching the GPU), but I could never make that work efficiently.

I have this GitHub repository with some tests I did in the past about it: [ITensor\_wGPU/DMRG - Multiple GPUs.ipynb at main · Tensor-Networks-PITT-MEMS/ITensor\_wGPU · GitHub](https://github.com/Tensor-Networks-PITT-MEMS/ITensor_wGPU/blob/main/DMRG%20-%20Multiple%20GPUs.ipynb) no sure if that can help you.

---

<div class="post-metadata">

**Author:** ![gc6020](https://avatars.discourse-cdn.com/v4/letter/g/ec9cab/32.png) [@gc6020](https://itensor.discourse.group/u/gc6020)\
**Post date:** [June 18, 2026, 6:32am UTC](https://itensor.discourse.group/t/run-on-multiple-gpus/2626/3 "2026-06-18T06:32:09Z")

</div>

Hi,

Thanks a lot for the information! I will give it a try.