PostgreSQL upper() and lower() functions don’t understand UNICODE?

I just discovered something very strange! My PostgreSQL database is set up to use UTF-8 internally and locale C and all very straight forward. Yet look at this:

select upper('Lösungsmittel') = upper('LÖSUNGSMITTEL');

 ?column?
----------
 f
(1 row)

select upper('Lösungsmittel') union all
select upper('LÖSUNGSMITTEL');

     upper
---------------
 LöSUNGSMITTEL
 LÖSUNGSMITTEL
(2 rows)

isn’t that strange?! The upper() function was unable to realize upper(‘ö’) = ‘Ö’!

The Java Character class can do that without a problem. I am surprised to see that, and we are on PostgreSQL server 13.1, not something super old.

Is there some configuration parameter to configure this character set stuff beyond selection of UTF-8?