keyboard – Switching layouts with Caps Lock doesn’t work if Caps Lock was pressed right after another key

I’m on macOS Big Sur 11.2.1 and I’m experiencing a weird behavior of Caps Lock as the layout switcher.

Caps Lock can be used for switching layouts by turning the Use Caps Lock key to switch layout setting in Settings > Keyboard.

It’s working fine, except when Caps Lock is pressed right after another key, e.g. Backspace.

Is the issue reproducible for someone else? Is it the macOS issue or the issue of my setup? If it’s the macOS issue, could it be fixed somehow?