Multivariable calc problem implies that you can’t draw a least squares line when variance is zero?

There is a discussion here about least squares fit and multivariable calculus:

Given a set of points $left(x_i,y_iright)$, and a vertical distance $d_i=y_i-left(ax_i+bright)$ from each point to some line $y=ax+b$, we want to find the $a, b$ so as to minimise:

$$D=sum_{i}{d_i^2}=sum_{i}left(y_i-left(ax_i+bright)right)^2=sum_{i}left(y_i^2-2ax_iy_i-2by_i+a^2x_i^2+2abx_i+b^2right)$$

To minimise $a$ and $b$, we want

$$frac{partial D}{partial a}=sum_{i}left(-2x_iy_i+2ax_i^2+2bx_iright)=0$$

and

$$frac{partial D}{partial b}=sum_{i}left(-2y_i+2ax_i+2bright)=0$$

Cancelling the $2$s and bringing the negative terms to the other side, you get:

$$ begin{pmatrix} sum{x_i^2} & sum{x_i} \ sum{x_i} & n \ end{pmatrix} cdot begin{pmatrix} a \ b \ end{pmatrix}=begin{pmatrix} sum{x_iy_i} \ sum{y_i} \ end{pmatrix}$$

My question is about the matrix on the left. From what I understand, the matrix is not invertible (and hence the equation has no solution) just when $detleft(Mright)=0$, right? But that happens when $ad-bc=0$, i.e. when

$$nsum{x^2}=left(sum{x}right)^2implies nsum{x^2}=n^2mu^2 implies sum{x^2}=nmu^2 implies frac{sum{x^2}}{n}-mu^2=0$$

But the LHS of the last equation is just $sigma^2$, right? So is the conclusion that you can’t draw a least squares line when Variance = 0??

Have I interpreted this correctly? I would have thought that, with zero variance, that would be somehow the easiest case to model (maybe infinite variance might have caused a problem, I might naively have thought)? Have I misunderstood something here?

Thanks