Spells only do what they say they do.
As written, there is nothing in the spell description that says the arrows are on fire, much less that they emit light.
And we should hope the arrows aren’t on fire – they are magical inside our quiver:
You touch a quiver containing arrows or bolts.
A spell says when it makes something emit light, for example, holy weapon:
You imbue a weapon you touch with holy power. Until the spell ends, the weapon emits bright light in a 30-foot radius and dim light for an additional 30 feet.
This answer makes the case that fire emits light to answer a similar question about the spell create bonfire. The key difference between create bonfire and flame arrows is that create bonfire actually creates fire:
You create a bonfire.
Flame arrows imbues our arrows with a magical fire damage effect, but never says it creates any fire.
A DM could rule otherwise.
Sure, a DM could rule otherwise, but in the partiuclar use-case given, I would definitely recommend talking it through with the player playing the gloomstalker. As a player, I would find it quite disagreeable for one of my abilities to be nerfed by a ruling that diverges from the rules as written.