Do you think IT and some lower-level programming jobs are going to go the way of the dodo? Seems a bit like a massive job bubble that’s gonna burst. It’s my opinion that one of the only things keeping tech and lower-level computer science-related jobs “prestigious” and well-paid is ridiculous industry jargon and public ignorance about computers, which are both going to go away in the next 10 years. […]”
This question is simultaneously on point about the future of technology jobs and exemplary of some pervasive misunderstandings regarding the field of software engineering. While it’s true that there is a great deal of “ridiculous industry jargon” there are equally many genuinely difficult problems waiting to be solved by those with the right skill-set. Some software jobs are definitely going away but programmers with the right experience and knowledge will continue to be prestigious and well remunerated for many years to come; as an example look at the recent explosion of AI researcher salaries and the corresponding dearth of available talent.
Staying relevant in the ever changing technology landscape can be a challenge. By looking at the technologies that are replacing programmers in the status quo we should be able to predict what jobs might disappear from the market. Additionally, to predict how salaries and demand for specific skills might change we should consider the growing body of people learning to program. As Hannah pointed out “public ignorance” about computers is keeping wages high for those who can program and the public is becoming more computer savvy each year.
The Continuing Drive Towards Commodification
The fear of automation replacing jobs is neither new nor unfounded. In any field, and especially in technology, market forces drive corporations toward automation and commodification. Gartner’s Hype Cycles are one way of contextualizing this phenomenon.
Gartner’s 2017 Hype Cycle
As time goes on, specific ideas and technologies push towards the “plateau of productivity” where they are eventually automated. Looking at history one must conclude that automation has the power to destroy specific job markets. In diverse industries ranging from crop harvesting to automobile assembly technology advances have consistently replaced and augmented human labor to reduce costs. A professor once put it this way in his compilers course, “take historical note of textile and steel industries: do you want to build machines and tools, or do you want to operate those machines?”
In this metaphor the “machine” is a computer programming language. This professor was really asking: Do you want to build websites using JavaScript, or do you want to build the V8 engine that powers JavaScript?
The creation of websites is being automated by WordPress (and others) today. V8 on the other hand has a growing body of competitors some of whom are solving open research questions. Languages will come and go (how many Fortran job openings are there?) but there will always be someone building the next language. Lucky for us, programming language implementations are written with programming languages themselves. Being a “machine operator” in software puts you on the path to being a “machine creator” in a way which was not true of the steel mill workers of the past.
The growing number of languages, interpreters, and compilers shows us that every job-destroying machine also brings with it new opportunities to improve those machines, maintain those machines, and so forth. Despite the growing body of jobs which no longer exist, there has yet to be a moment in history where humanity has collectively said, “I guess there isn’t any work left for us to do.”
Pinsetters
Commodification is coming for us all, not just software engineers. Throughout history, human labor has consistently been replaced with non-humans or augmented to require fewer and less skilled humans. Self-driving cars and trucks are the flavor of the week in this grand human tradition. If the cycle of creation and automation are a fact of life, the natural question to answer next is: which jobs and industries are at risk, and which are not?
Who’s Automating Who?
AWS, Heroku, and other similar hosting platforms have forever changed the role of the System Administrator/DevOps engineer. Internet businesses used to absolutely need their own server master. Someone who was well versed in Linux; someone who could configure a server with Apache or NGINX; someone who could not only physically wire up the server, the routers, and all the other physical components, but who could also configure the routing tables and all the software required to make that server accessible on the public web. While there are definitely still people applying this skill-set professionally, AWS is making some of those skills obsolete — especially at the lower experience levels and on the physical side of things. There are very lucrative roles within Amazon (and Netflix, and Google…) for people with deep expertise in networking infrastructure, but there is much less demand at the small-to-medium business scale.
“Business Intelligence” tools such as SalesForce, Tableau and SpotFire are also beginning to occupy spaces historically held by software engineers. These systems have reduced the demand for in-house Database Administrators, but they have also increased the demand for SQL as a general-purpose skill. They have decreased demand for in-house reporting technology, but increased demand for “integration engineers” who automate the flow of data from the business to the third-party software platform(s). A field that was previously dominated by Excel and Spreadsheets is increasingly being pushed towards scripting languages like Python or R, and towards SQL for data management. Some jobs have disappeared, but demand for people who can write software has seen an increase overall.
Data Science is a fascinating example of commodification at a level closer to software. Scikit.learn, Tensorflow, and PyTorch are all software libraries that make it easier for people to build machine learning applications without building the algorithms from scratch. In fact, it’s possible to run a dataset through many different machine learning algorithms, with many different parameter sets for those algorithms, with little to no understanding of how those algorithms are actually implemented (it’s not necessarily wise to do this, just possible). You can bet that business intelligence companies will be trying to integrate these kinds of algorithms into their own tools over the next few years as well.
In many ways data science looks like web development did 5–8 years ago — a booming field where a little bit of knowledge can get you in the door due to a “skills gap”. As web development bootcamps are closing and consolidating, data science bootcamps are popping up in their place. Kaplan, who bought the original web development bootcamp (Dev Bootcamp) and started a data science bootcamp (Metis) has decided to close DevBootcamp and keep Metis running.
Content management systems are among the most visible of the tools automating away the need for a software engineer. SquareSpace and WordPress are among the most popular CMS systems today. These platforms are significantly reducing the value of people with a just a little bit of front end web development skill. In fact the barriers for making a website and getting it online have come down so dramatically that people with zero programming experience are successfully launching websites every day. Those same people aren’t making deeply interactive websites that serve billions of people, but they absolutely do make websites for their own businesses that give customers the information they need. A lovely landing page with information such as how to find the establishment and how to contact them is more than enough for a local restaurant, bar, or retail store.
If your business is not primarily an “internet business” it has never been easier to get a working site on the public web. As a result, the once thriving industry of web contractors who can quickly set up a simple website and get it online is becoming less lucrative.
Finally, it would border on hubris to ignore the physical aspect of computers in this context. In the words of Mike Acton: “software is not the platform, hardware is the platform”. Software people would be wise to study at least a little computer architecture and electrical engineering. A big shake up in hardware, such as the arrival of consumer grade quantum computers would (will) change everything about professional software engineering.
Quantum computers are still a ways off, but the growing interest in GPUs and the drive toward parallelization is an imminent shift. CPU speeds have been stagnant for several years now and in that time a seemingly unquenchable thirst for machine learning and “big data” has emerged. With more desire than ever to process large data-sets OpenMP, OpenCL, Go, CUDA, and other parallel processing languages and frameworks will continue to become mainstream. To be competitively fast in the near-term future, significant parallelization will be a requirement across the board, not just in high-performance niches like operating systems, infrastructure and video games.
Everybody Is Learning To Code
Websites are ubiquitous. The 2017 Stack Overflow Survey reports that about 15% of professional software engineers are working in an “Internet/Web Services” company. The Bureau of Labor Statistics expects growth in Web Development to continue much faster than average (24% between 2014 and 2024). Due to its visibility, there has been a massive focus on “solving the skills gap” in this industry. Coding bootcamps teach Web Development almost exclusively and Web Development online courses have flooded Udemy, Udacity, Coursera and similar marketplaces.
The combination of increasing automation throughout the Web Development technology stack and the influx of new entry level programmers with an explicit focus on Web Development has led some to predict a slide towards a “blue collar” market for software developers. Some have gone further, suggesting that the push towards a blue collar market is a strategy architected by big tech firms. Others, of course, say we’re headed for another bursting bubble.
Change in demand for specific technologies is not news. Languages and frameworks are always rising and falling in technology. Web Development in its current incarnation (“JS Is King”) will eventually go the way of Web Development of the early 2000’s (remember Flash?). What is new, is that a lot of people are receiving an education explicitly (and solely) in the current trendy web development frameworks. Before you decide to label yourself a “React developer” remember there were people who once identified themselves as “Flash developers”. Banking your career on a specific language, framework, or technology is a game of roulette. Of course it’s quite difficult to predict what technologies will remain relevant, but if you’re going to go all in on something, I suggest relying on The Lindy Effect and picking something like C that has already withstood the test of time.
The next generation will have a level of de facto tech literacy that Generation X and even Millennials do not have. One outcome of this will be that using the next generation of CMS tools will be a given. These tools will get better and young workers will be better at using them. This combination will definitely will bring down the value of low-level IT and web development skills as eager and skilled youngsters enter the job market. High schools are catching on as well, offering computer science and programming classes — some well educated high school students will likely be entering the workforce as programming interns immediately upon graduation.
Another big group of newcomers to programming are MBAs and data analysts. Job listings which were once dominated by Excel are starting to list SQL as a “nice to have” and even “requirement”. Tools such as Tableau, SpotFire, SalesForce, and other web-based metrics systems continue to replace the spreadsheet as the primary tool for report generation. If this continues more data analysts will learn to use SQL directly simply because it is easier than exporting the data into a spreadsheet.
People looking to climb the ranks and out-perform their peers in these roles are taking online courses to learn about databases and statistical programming languages. With these new skills they can begin to position themselves as data scientists by learning a combination of machine learning and statistical libraries. Look at Metis’ curriculum as a prime example of this path.
Finally, the number of people earning Computer Science and Software Engineering degrees continues to climb. Purdue, for example, reports that applications to their CS program have doubled over five years. Cornell reports a similar explosion of CS graduates. This trend isn’t surprising given the growth and ubiquity of software. It’s hard for young people to imagine that computers will play a smaller role in our futures, so why not study something that’s going to give you job security.
Rarity and Expectation
A common argument in the industry nowadays is around the idea that the education you receive in a four-year Computer Science program is mostly unnecessary cruft. I have heard this argument repeatedly in the halls of bootcamps, web development shops, and online from big names in the field such as this piece by Eric Elliott. The opposition view is popular as well, with some going so far as saying “all programmers should earn a master’s degree”.
Like Eric Elliott, I think it’s good that there are more options than ever to break into programming, and a 4 year degree might not be the best option for many. Simultaneously, I agree with William Bain that the foundational skills which apply across programming disciplines are crucial for career longevity, and that it is still hard to find that information outside of university courses. I’ve written previously about what skills I think aspiring engineers should learn as a foundation of a long career, and joined Bradfield in order to help share this knowledge.
Coding schools of many shapes and sizes are becoming ubiquitous, and for good reasons. There is quite a lot you can learn about programming without getting into the minutia of Big O notation, obscure data structures, and algorithmic trivia. However, while it’s true that fresh graduates from Stanford are competing for some jobs with fresh graduates from Hack Reactor, it’s only true in one or two sub-industries. Code school and bootcamp graduates are not yet applying to work on embedded systems, cryptography/security, robotics, network infrastructure, or AI research and development. Yet these fields, like web development, are growing quickly.
Some programming-related skills have already started their transition from “rare skill” to “baseline expectation”. Conversely, the engineering that goes into creating beastly engines like AWS is anything but common. The big companies driving technology forward — Amazon, Google, Facebook, Nvidia, Space-X, and so on — are typically not looking for people with a ‘basic understanding of JavaScript’. AWS serves billions of users per day. To support that kind of load an AWS infrastructure engineer needs a deep knowledge of network protocols, computer architecture, and several years of relevant experience. As with any discipline there are amateurs and artisans.
These prestigious firms are solving research problems and building systems that are truly pushing against the boundaries of what is possible. Yet they still struggle to fill open roles even while basic programming skills are increasingly common. People who can write algorithms to predict changes in genetic sequences that will yield a desired result are going to be highly valuable in the future. People who can program satellites, spacecraft, and automate machinery will continue to be highly valued. These are not fields that lend themselves as readily to a “3 month intensive program” as front end web development, at least not without significant prior experience.
Because computer science starts with the word “computer” it is assumed that young people will all have an innate understanding of it by 2025. Unfortunately, the ubiquity of computers has not created a new generation of people who de facto understand mathematics, computer science, network infrastructure, electrical engineering and so on. Computer literacy is not the same as the study of computation. Despite mathematics having existed since the dawn of time there is still a relatively small portion of the population with strong statistical literacy, and computer science is similarly old. Euclid invented several algorithms, one of which is used every time you make an HTTPS request; the fact that we use HTTPS every time we login to a website does not automatically imbue anyone with a knowledge of how those protocols work.
Bimodal Wage Distributions
More established professional fields often have a bimodal wage distribution: a relatively small number of practitioners make quite a lot of money, and the majority of them earn a good wage but do not find themselves in the top 1% of earners. The National Association for Law Placement collects data that can be used to visualize this phenomenon in stark clarity. A huge share of law graduates make between $45,00 and $65,000 — a good wage, but hardly the salary we associate with a “top professional”.
Distribution of salaries for people with a law degree, from NALP.
We tend to think that all law graduates are on track to becoming partners at a law firm when really there are many paths: paralegal, clerk, public defender, judge, legal services for businesses, contract writing, and so on. Computer science graduates also have many options for their professional practice, from web development to embedded systems. As a basic level of programming literacy continues to become an expectation, rather than a “nice to have”, I suspect a similar distribution will emerge in programming jobs.
While there will always be a cohort of programmers making a lot of money to push on the edges of technology, there will be a growing body of middle-class programmers powering the new computer-centric economy. The average salary for web developers will surely decrease over time. That said, I suspect that the number of jobs for “programmers” in general will only continue to grow. As worker supply begins to meet demand, hopefully we will see a healthy boom in a variety of middle-class programming jobs. There will also continue to be a top-professional salary available for those programmers who are redefining what is possible.
Regardless of which cohort of programmers you’re in, a career in technology means continuing your education throughout your life. If you want to stay in the second cohort of programmers you may want to invest in learning how to create the machines, rather than simply operate them.