Design Benefits of Using a Message Broker for Scaling Web Socket Client to Client Communication

I'm designing a system where exactly two clients need to exchange messages sent by a backend service. My original plan was to use Websockets and connect the clients to a Websocket server. However, this can not be scaled to more than one server without modification. A popular design is the use of multiple websocket servers and broker messages in the backend with a kind of pub / sub-message broker (Redis or similar).

Since my request is for client pair communication only (no chat-like behavior) and I only need one-time delivery, is there any benefit to using a message broker just because the hosts communicate directly with each other? For example, client A sends a message to host A, which sends directly to host B, which sends to client B. I expect several thousand concurrent sessions, each consisting of a couple of clients.