memory hardware – Computing with Extra States

So, computing largely operates through states of 0 or 1, this I am aware of. But I do have a question, is there a meaningful difference in computing using more states. For example, 0, 1, and 2. Logically, looking at the surface, this would allow encoding more information in a smaller number of bits. But is this actually how it would work in theory?

New question looking at answers given, how exactly would a Base-e system work? or does this only work in Pure Mathematics and not the physical world?