linux – QLogic/Broadcom 57800 SR-IOV guest driver fails to load

I have a setup with a Dell R730 with a 57800 network card, with SR-IOV enabled on a virtual switch. The issue I am having is that even though I can SR-IOV and it works with Windows guests, I am unable to get it to work with my Debian Linux Buster guests. I am trying to load the kernel module BNX2X, and have loaded the firmware from the debian non-free repositories. I have also tried Debian server 20.04, with and without the HWE kernel, and get the exact same error. The errors in the system log are as follows:

Sep 07 09:19:58 xxxx kernel: bnx2x: QLogic 5771x/578xx 10/20-Gigabit Ethernet Driver bnx2x 1.712.30-0 (2014/02/10)
Sep 07 09:19:58 xxxx kernel: bnx2x b79a:00:02.0: msix capability found
Sep 07 09:20:08 xxxx kernel: (bnx2x_send_msg2pf:189(eth%d))PF response has timed out
Sep 07 09:20:08 xxxx kernel: bnx2x: probe of b79a:00:02.0 failed with error -11

lshw -c network also lists the adapter as unclaimed.

lspci shows the adapter as b79a:00:02.0 Ethernet controller: Broadcom Limited NetXtreme II BCM57800 1/10 Gigabit Ethernet Virtual Function

Any ideas? I have tried everything I can think of here