firewall – systemd-timesyncd requires incoming ephemeral ports opened

I’m trying to get some sort of time synchronization configured for a Ubuntu server. The server is behind a cloud-provider-stateless firewall. Through some trial and error I found out that in order for ntp to work, I have to open the incoming UDP port 123.

Then I read that using systemd-timesyncd is preferred nowadays, so I tried switching over to that. But that did not work. The service log was full of

systemd-timesyncd(2656121): Timed out waiting for reply from 91.189.89.199:123 (ntp.ubuntu.com).
systemd-timesyncd(2656121): Timed out waiting for reply from 91.189.94.4:123 (ntp.ubuntu.com).

Only after I also whitelisted the ephemeral UDP ports 32768–65535 in the firewall did this start to work:

systemd-timesyncd(2656121): Initial synchronization to time server 91.189.91.157:123 (ntp.ubuntu.com).

Is opening that range of ports really necessary to operate systemd-timesync?