Hi ! The recent major security issues about Meltdown and Spectre are really important, when userland program can access data that are supposed to be "memory protected" by the MMU of the CPU.
Most important, those issues were here since years, probably used by some people, actually we do not know but if 2 distinct teams were able to discover it, there is no reason that anyone was not able to do so.
Since several years, secured cryptographics co-processors are used for very secured environment. A Ledger or Trezor is built on this principle were "basically" your wallet asks it to sign a specific transaction using one of your private keys that are "stored" in another physical device: there is a clean interface and protocol to ask your device to sign a given transaction.
The problem is on the server side, you need to sign/crypt/decrypt several thousands of transactions by second.
I hope we will start to see servers implementing natively such kind of protocol were there is a small co-processor/memory with its secure firmware where you can "store" a few private keys with its own physical network and its fast PCIExpress connection or DMA.
Probably vendors will start to say, we have a good secured solution for this or that at the MMU level. I just think that the physical solution is simpler and better. First such secure co-processors are now very cheap, it makes sense to have private keys inside another physical memory and to have a clean and secure protocol to access it. Such secured co-processor can be put on the same die of the regular 4,6,8 cores with its own memory and flash, the area used on the die of the chip is nearly nothing, it will be very useful for many applications and the cost will be zero at scale.
There are a few standard protocols and APIs that exist and can be used for this purpose: U2F for instance. A bit more work is needed at the standard and integration at the OS and application level, but it is not a big deal.