linux – Mounting virtual GPUs on different machines


I have multiple computers connected over a VPN, each computer having one or more Nvidia GPUs. All computers run Linux. I’m wondering if it’s possible to mount them over network such that they appear to be attached on a single machine.

For example, if I have a machine (say MA) with one GPU and another machine (say MB) with two GPUs. Running nvidia-smi on MA would list the one GPU and running it on MB would list two GPUs.

I want to mount MB’s GPUs on MA as virtual GPUs such that running nvidia-smi on MA will now list three GPUs. Is this possible?

I realize this will have high latency. But in my case, that is not a concern.