In order for changes to the Bitcoin protocol to be made, 95% consensus must be achieved, otherwise a hard fork would need to occur? Is this the primary reason why changes to the core protocol are so difficult to make?
Not really – most recent changes have been made by using miner activated forks, which requires miners to vote on proposed changes. Only once a certain threshold is hit (which is very much configurable, and not fixed to something like 95%), the changes are activated.
That said, it is entirely possible to create upgrades without such voting – many other blockchains do it all the time. It’s simply a matter of the community agreeing that those changes are valid and not an attempt at an attack.
Even with 95% of miner support, if the broader community feels that recent changes are not in their best interests (such as the somewhat contentious segwit changes), they may choose to fork the chain at a point before the changes (as done by Bitcoin Cash).
You may find the answers to How is Bitcoin governed by mathematics? to be of interest here. Consensus is very different from governance, and one does not imply the other.
With regards to consensus mechanism, moving aside whether PoW/PoS/Other is the proper solution, since the vast majority of voting belongs to large mining pools why would they ever vote for PoS or another non-mining solution? It seems even if a superior consensus mechanism exists it is highly doubtful BTC would ever deviate without a hard fork?
Personally, I’d agree with that – any attempt to reduce the power of miners may result in them attempting to protect it.
However, at the end of the day, if the broader community moves to a different fork, the value will follow it. Any miners left on the other chain will be subject to vastly reduced economic incentives (BCH vs BTC price and adoption being a good example).
However, is it possible that a 51% attack could already be occurring and we have no idea?
One could potentially be happening right now, and our ability to detect it would be limited.
How do we know the mining operations aren’t in collusion already and have not already been double spending, etc? If fraudulent activity occurs would we actually ever know?
There are many research groups, exchanges, coin tracking firms, and other entities actively tracking such things – it is fairly easy to detect a double spend after the fact, especially if you were on the receiving side.
Additionally, a double spend attack on BTC would be very detrimental to the BTC price, which is not something most miners would be able to absorb. It is in their best interests to secure the chain, not attack it.