This month, Nvidia released the Titan V. A 3000$ video card equipped with the biggest GPU to ever be available for the general public. 5120 CUDA cores, half of those being built for double precision operations, and with dedicated tensor cores for deep learning. So, even though it is a Titan, it is more in line to what the Titan initially was, a great compute card for the fraction of the price of a Tesla card. It also happens to crush every game possible under its mighty boot, but don't expect it to go on sale for less than 3000$ anytime soon, or for video cards with a GPU of this size to cost less any time soon.
Why? Because making a GPU with 5120 individual cores, with over 21 billion transistors, in a 815 mm² die is difficult. It's a matter of reaching the limits of what the semiconductor fabrication can achieve with current technology. And even if we do get to 7 nanometer soon, we're still hitting a brick wall in terms of actual usable dies resulted from printing them on a wafer of silicon. If you're not familiar with the process, take a look at the picture below. The individual circuits are printed using light passing through a mask upon the layers of the wafer. It can go through many passes of this process, called photolithography, and also through several chemical processes that serve better define the features of the circuits or to add certain materials to the circuit.
The basic idea is that this process is not really easy, and the bigger the die, the more circuits, the more transistors you need to have in a single GPU, the higher the chances are that you will get a dud. Generally, dies made near the center of the wafer benefit from the highest resolution possible and have the highest chance of actually being usable. They have the highest yield. The closer you get to the edges, the higher the chances of errors occurring. So making a single chip the size of the one in the Titan V is very expensive, since there will be a lot of loses along the way. That's why you won't see a GPU of this size in a mainstream card, with an affordable price.
There is, however, an alternative. The Multi Chip Module GPU.
Nvidia is currently experimenting with this idea, and there is also the assumption that AMD's Navi is also going in the same direction. So, what is it? As the picture above shows, instead of on single gigantic GPU, there are several smaller GPUs contained within the same package. This isn't to be confused with SLI or Crossfire, where multiple video cards are connected together. Here, the computing cores themselves are linked together with a high speed connection that even goes beyond what we saw in previous years with the dual-GPU video cards, such as the Titan Z or the Radeon 295X2. That high-speed link may be done through AMD's Infinity Fabric, while Nvidia has its own method that it has already tested in limited capacity.
The advantages of this method of creating a single GPU out of smaller GPUs are very important for the future development of video cards. As in the example above, making those four GPUs is a lot cheaper that making the larger one. Each of them contain less circuits, there are fewer chances for errors, yields are higher and therefore they are cheaper to produce. Yes, you do need 4 of them for one MCM, but that's still cheaper and easier than making a giant one. And then there's the scaling of the performance. It the testing done so far, which did involve a lot of simulations and not a lot of actual actual finished hardware, they got to about 30% more performance that you would see from a traditional multi-GPU configuration, through SLI. That may not seem like a lot, but there's a bigger upside here. Since the MCM-GPU is seen as a single GPU, there's not much need to implement special cases to actually use them as they're supposed to be used. Meaning, that unlike SLI or Crossfire, it should actually work by default.
And then there's the other upside. A 5120 CUDA core GPU is currently just about the best that Nvidia can make. But if you bolt together four GPUs with, let's say, 2560 CUDA cores, you've got a final GPU that is larger and has higher performance than anything that can be built today. Sure, I am exaggerating a bit here, since a 2560 CUDA core GPU will still be expensive. But let's say a 1280 core one. Four of those in an MCM would be the size of a Tesla V, would be much easier to build than the 5120 core single GPU and would be considerably cheaper. I mean, you can get 4 GeForce 1060 for 1000$, and that's with the VRAM and all the packaging. The GPU itself is a lot cheaper. So you could probably have something with the core count of a Titan V but at a third of the price, and with similar performance. How similar? Preliminary testing indicated that they can get within 90% of the actual power of what a single large GPU would have.
This seems like it's going to be the future of the video card. Multi Chip Modules made out of cheaper, smaller GPUs. Sort of like we've now got CPUs with multiple smaller cores, rather than one physical core and a virtual one running at 10GHz , mostly because the technology failed to live up to the expectations with things like the Pentium 4. Though, this isn't really a correct analogy, since they work on different ideas, but in terms of the general idea that's kinda where we're going. And when it does happen, we should see some really interesting chance ups in the GPU market. Especially if Intel goes in again.