The maximum damage on a missed attack is zero.
The effect still applies, but in the case of a missed attack roll, the maximum damage you can deal is zero damage.1
The effect applies on the
next damaging spell you cast.
Notably, it is not on the next damage spell that does damage, only on the next damaging spell you cast. So if you cast a damaging spell that calls for an attack roll and you miss, the maximum damage for that spell is zero.
I read this as a description of spells in general, not a description of the results. I believe the intent is that I should be able to determine if a spell is a “damaging spell” by reading its description, before the spell is even cast. But, because I had to say “I believe” in that last sentence, we have the next section.
There is room for a different ruling.
The first section is how I would rule it and why, but I can see a case to be made for ruling differently. The issue is that you could also argue that a spell that can do damage but fails to is not a damaging spell. The reason I am inclined to disagree with this ruling is that the wording could very easily be rearranged to quite clearly affirm this ruling, if the wording was instead something like,
Maximize the damage of the next spell you cast that deals damage within the next minute.
Notably, this exact wording is used in the Evocation Wizard’s ability Overchannel:
Starting at 14th level, you can increase the power of your simpler spells. When you cast a wizard spell of 1st through 5th-level that deals damage, you can deal maximum damage with that spell.
I know this isn’t a strong argument against, but I find it compelling enough, especially when compared to the language of Overchannel. Like I said, there is room to rule this either way.
1 Without loss of generality, this applies similarly to saving throw spells that deal no damage on a success.