oauth2 – Scope Narrowing Access Token with Refresh token

I have recently picked up the oauth2 spec and decided to use keycloak as an auth server. I am facing difficulties in a certain aspect. I want to be able to request multiple access tokens with narrow scope when a user is first authenticated or whenever one of those narrowed tokens expires. This is due to the fact that i want a level of isolation between the different resource servers which are going to receive the tokens. I.e avoid one resource server receiving token with too much privileges (full scope).
So I read in the oauth2 spec that the refresh token endpoint can be used to retrieve a token with a narrower scope. At the moment when ever a user authenticates a refresh and access token with full scope are generated (but they are not used for communication). I imagined that i can use the issued refresh token to generate a couple of new narrower access tokens which are going to be used against the resource servers. But whenever i call the /token endpoint with grant type refresh_token, and even include the scope with the request I still receive an access token with the same full scope. I do know that the refresh token also contains a list of the initial requested scopes (in this case the full scopes) And i was wondering if the scopes in the refresh token somehow override auth server’s (keycloak in my case) descision when I request a narrower scope in the /token request POST payload. I have configured keycloak’s optional scopes and default scopes are left empty. I am using spring security’s oauth2 implementation.

Any help would be greatly appreciated.