Code Generation with Neural Networks
The “No-Code” movement is gaining steam. The idea is not a new one, and in many senses, the entire story of personal computing has been about better ways to automatically generate programs. Let me explain.
The first personal computer killer app was a program called VisiCalc. Written by Dan Bricklin, VisiCalc was the industry’s first spreadsheet software. Before it came about, financial analysis on computers was quite an involved task. The spreadsheet changed all this by offering a simple way to write very specialized accounting and forecasting programs in easily understood arithmetic syntax. VisiCalc also provided immediate visual feedback, allowing errors to be corrected quickly. And it was an instant hit. The spreasheet freed users from Fortran programming for simple tasks and transformed a small, inexpensive home computer into a powerful accounting machine; a $1500 computer could perform tasks delegated previously to expensive mainframes!
The first killer app, then, was all about building financial and modeling programs quickly and simply.
But financial computations aren’t the only type of task computers are useful for. They are also great for keeping records, searching and displaying vast document collections or databases. The programming language COBOL ruled the roost when it came to business applications of this type. Until the advent of Visual Basic. In the 1990s, Microsoft launched an easy-to-use visual interface building environment on top of a thoroughly enhanced BASIC kernel. VB, as it became known, allowed interface (UX) generation via a mouse click here and a drag and drop operation there. At the time, Windows user interface libraries were not trivial to code to, but suddenly, all that complexity melted away and designing form-based Windows apps become a “Low-Code” affair. With Visual Basic, you could build business applications quickly and as a result it spawned an entire industry of app developers!
The idea of being able to visually edit content, applications or interfaces was referred to as “What you see is what you get”, or WYSIWYG. In the ’90s, it was part of a wider push toward tools that enabled immediate and quick previews on-screen, so authors and developers could immediately inspect the output they would eventually get on paper, in print or with their final, compiled application. With WYSIWYG, you didn’t have to rely on escape sequences and formatting tags like previous document formatting solutions such as nroff, troff or LaTeX required. Instead, you used a pointing device or keyboard to take control of an on-screen element’s properties, modified them and saw instant feedback reflected on the screen! It was this instant feedback and rapid development ability that made WYSIWYG so useful.
In 1985, Steve Jobs unceremoniously departed Apple Computer and by 1988 was well on to his “next” thing. He was building the NeXT computer workstation with a small team of amazingly talented engineers. While the NeXTcube and later, NeXTstation, were certainly the type of sleek, flashy hardware Jobs was known for, they were much more than that. Their enduring innovation was the excellent software development tools for object oriented design that were bundled with them. In fact, much of that software still survives in the form of Apple’s Interface Builder, which is part of XCode. The visual tools to generate user interfaces, build form based applications and connect them to backend databases became the essential tasks in an era of client-server applications. All of these were converted to near no-code by NeXTSTEP and its development tools.
There were many more examples of rapid application development environments, including Delphi, Paradox, Smalltalk - Alan Kay’s brilliant piece of work from a much earlier era in the 1970s - and many others.
But let us come back to the No-Code movement. As I said, it’s nothing new. Killer software apps have always been about doing things quickly; building tools and getting tasks done with minimal effort. And so, the No-Code movement has gained strength once more. Instead of “Rapid Application Development”, which is what it was referred to through the 90s and early 2000s, the idea has been rebranded, but the goals remain the same: enable the development of specific types of applications without requiring users to write code. Remember Geocities? What was it if not an early precursor of Wix and modern website platforms that allow you to build websites without writing CSS or HTML code?
History doesn’t as much repeat as it rhymes with itself. And true to that adage, a new development that holds significant implications for No-Code tool development is the use of neural networks to automatically generate code. Given a high level specification of a function in plain english, a trained ML model can now automatically produce the necessary source code in a variety of different programming languages. In fact, the new language model, GPT-3 from OpenAI, is being used for exactly this task. A number of readily available examples illustrate well GPT-3’s ability to generate code for Python functions based only on plain english comments describing their functionality. The model has also been used to generate HTML/CSS for laying out web pages based on natural language descriptions of the desired page.
How does this technology work? It seems like magic, but as with all things, it’s just science!
For a deeper dive, I leave you with an insightful and very hands-on discussion of how to build a system of this type. Enjoy the journey!
Further reading: