Binding to `0.0.0.0` in Windows leads to slow connection establishment

I’m having a problem when trying to setup a TCP server listening on all addresses in Windows (Win10 build 19042.1110).

If I specify my server to listen on 0.0.0.0 it works, but connections take about 2 seconds to take place from the client. Any server bound to 0.0.0.0 shows this behavior (e.g. with python -m http.server --bind 0.0.0.0).

If I explicitly bind to 127.0.0.1 connections are instant, but obviously my program is not accessible from LAN.

Surprisingly binding to (::) works fine. It’s just when trying to explicitly bind to IPv4.

How can I even begin to debug this?