There are at least two ways to believe in the idea of quality. You can believe there's something ineffable going on within the human mind, or you can believe we just don't understand what quality in a mind is yet, even though we might someday. Either of those opinions allows one to distinguish quantity and quality. In order to confuse quantity and quality, you have to reject both possibilities. The mere possibility of there being something ineffable about personhood is what drives many technologists to reject the notion of quality. They want to live in an airtight reality that resembles an idealized computer program, in which everything is understood and there are no fundamental mysteries. They recoil from even the hint of a potential zone of mystery or an unresolved seam in one's worldview. This desire for absolute order usually leads to tears in human affairs, so there is a historical reason to distrust it. Materialist extremists have long seemed determined to win a race with religious fanatics: Who can do the most damage to the most people?
Author: Jaron Lanier