soft question – Quantum mechanics outside $L^{2}$ spaces

To this day, it is known that a satisfying mathematical formulation of quantum field theory is far from sight, even though some noninteracting theories can be described in rigorous mathematical language.

On the other hand, quantum mechanics is (at least to my knowledge) highly considered as a subject in which a rigorous mathematical treatment for a more mathematical audience is avaiable. There are plenty of books dedicated to introduce quantum mechanics from a mathematical perspective, and many others use quantum mechanics as an application to functional analysis and related concepts.

Ever since I got in touch with this subject from the first time, I noticed that most books (if not all books I know) provide discussions which are centered in $L^{2}$ spaces, which is an appropriate space to describe wave functions. Von Neumann’s work on spectral theory provides the central tool to understand the principles of quantum mechanics from a mathematical solid point of view. However, I had the feeling that Dirac’s approach to quantum mechanics was not usually being taken into account in those books; some books mention Dirac’s notation and some of them mention the fact that some objects in his approach do not live in a Hilbert space, like position or momentum eigenvectors, but these discussions are usually brief and nothing else is said about it.

I think this feeling was reinforced after my this previous post of mine, where I ask why are Rigged Hilbert spaces are still some sort of research material rather than book material.

On the other hand, the more I get in touch with physics books about quantum mechanics, the more I see how the use of Dirac’s approach is important to the theory. Depending on what you are studying, Dirac’s approach becomes the standard way to do it. For instance, spin systems, path integrals and perturbation theory are usually described under Dirac’s point of view in physics textbooks. In addition, I had to face expressions like:
$$int dxpsi(x)|xrangle $$
countless times in physics books in order to understand some concept or theory, and the latter can only be fully understood using Rigged Hilbert spaces.

In summary, it seems to me that $L^{2}$ spaces provide a nice tool to study the basics of quantum mechanics from a mathematical point of view, but by any means it seems enough. Physicists don’t always talk about wave functions. So, my question is: to what extent are $L^{2}$ spaces enough and to what extent can we say that quantum mechanics has a solid mathematical treatment? I believe the answer to this question might pass through one of the following points (but not necessarily): (1) quantum mechanics is not fully understood from a mathematical point of view and my premises are false and (2) there exists a way of fully translating every concept or system studied by physicists to some $L^{2}$ (maybe some isomorphism theorem?) in which case $L^{2}$ spaces are enough.