The vortex can be between 5 and 30 feet wide
The largest area of water the nereid can control is 30 feet wide, so this is the maximum size of vortex they can create. Note that the description of the whirlpool effect specifies that the top of the vortex can be “up to 50 feet wide at the top”, which means the caster may choose to make the top of the vortex smaller than 50 feet wide if they like. In the nereid’s case, they must make the vortex smaller in order to fit it into their spell’s limited area of effect.
The whirlpool still requires body of water 50 feet wide
Regardless of the size of the vortex, it can only be created in a body of water that is at least 50 feet wide. This is true even for a narrow vortex that is only 5 feet wide at the top, and even if it is created by a “regular” Control Water spell with the full size area of effect. The spell’s area does not need to cover the entire body of water; it only needs to cover the area occupied by the vortex. Note that in all cases, creatures in the water within 25 feet of the vortex will be pulled toward it regardless of whether they are inside the spell’s area of effect.
Aside: the vortex can affect things outside the spell’s area
The size of the area affected by a whirlpool is a bit nebulous, because there is an extra 25-foot region around the vortex itself in which creatures and objects can be pulled toward the vortex. However, this extended region is not part of the spell’s area of effect; it is simply the region within 25 feet of the vortex. Spells that have effects outside their formally defined area are actually quite common. For example, a Flaming Sphere is a single 5-foot-diameter sphere, but it deals fire damage to creatures within 5 feet outside this area. Thunderwave can be heard up to 300 feet away. And of course there are many spells (including Flaming Sphere) that produce light in a certain radius around their actual area of effect.
Hence, if a nereid casts Control Water and makes a maximum size vortex 30 feet in diameter, the full area that will be affected in some way is a circular region 80 feet across (assuming the body of water is wide enough).