The phrase “Owe the bank 500 dollars, that is your problem. Owe the bank 500 million – that is the bank’s problem.” is something that comes to mind for some of the tech monopolies right now. There is a shifting relationship between the largest software companies in the world and their suppliers, and as the leading software companies have become ever-larger portions of the compute pie, it’s kind of become the problem of the tech companies, and not the semiconductor companies that service them to push forward the natural limits of hardware. Software ate the world so completely that now the large tech companies have to deal with the actual hardware that underlies their stack. Especially as some companies like Intel have fallen behind. #intrest
‘This is a critical issue of our time ‘ From Googles talk about this issue:
However, if you look at Google’s products, were our demand for compute power continues to grow ??substantially, frequently at exponential type numbers, and this used to be a free ride with Moore’s law. Giving us increasing compute power to keep up with this increasing demand for computation. But that’s kind of come to an end now which isn’t great, and so we have a lot of projects at Google that are trying to solve this and I’m working on one of them. I don’t work on ?>the most successful project we’ve had here, which is the TPU. This is a ML accelerator that has drastically increased our ability to do ML compute, and this is kind of an interesting thing in its showing that hardware, that is the main specific, can potentially keep up with this growing demand for compute. The problem is it’s taking a lot of effort to create these hardware accelerators and while some groups at Google are big enough, like the ML people to have dedicated teams working on dedicated hardware like the TPU um, we are looking at the problem that every team at Google is eventually probably going to have to look at hardware accelerating their workloads, especially if their demand continues to rise.
Intresting.