Leave a common index as free in contraction

I have a tensor that can be written in the following form:

T_{i\, i'}^{j\,j'}= A_{i\,j} \delta(i,\,i') \delta(j,\,j')

Where i, j etc. Are site indices.
From this writing it is clear that all the indices are free and do not have to be contracted, but how can I prevent that? If I double prime the index to make it different, then how I’m going to undo it the moment that I have to actually apply the Tensor? (Consider that this tensor will be in a MPO so it will use the default implementation of MPS*MPO).
In my case the sites are Qudits of dim d=4, so T would have a size of 256 while A of 16, so I think it would be both space and time saving.

EDIT:
I’ve found something that works:

T = zeros(3,2,2)
T[1,1,1] = 1
T[1,2,2] = 2
T[2,1,1] = 3
T[2,2,2] = 4
T[3,1,1] = 5
T[3,2,2] = 6

i,j = Index.((3,2))

T = itensor(T,(i, j', j))

A = zeros(3,2)
A[1,1] = 1
A[1,2] = 2
A[2,1] = 3
A[2,2] = 4
A[3,1] = 5
A[3,2] = 6

A=ITensor(A,(i,j))

function foo(A::ITensor, j::Index)
  k=Index(1)
  A=ITensors.setinds(A,(inds(A)...,k)) #add trivial index
  pairs = [A*onehot(j=>i)*onehot(j'=>i)=>k for i in 1:dim(j)]
  S,s = directsum(pairs...)
  return S*delta(s,j)
end

T == foo(A,j)

The only problem is that on exit I still have something dense, while I’d like to obtain something that employs the sparsity of the matrix

Hi @VinceNeede,
The question you’re asking is a good one, about how to obtain a sparse result. Currently we dont support general sparse tensors (only rather special-case types of sparsity) but with a current redesign that is happening of the underlying layers of ITensor we will be supporting general sparsity in the near future. So for now please just use dense tensors unless it is a major bottleneck in your calculation and we will be making news posts when the more general sparse features become available.