blog dds

The tools we use have a profound (and devious!) influence on our thinking habits, and, therefore, on our thinking abilities.

— Edsger W. Dijkstra

With a shovel excavator a single operator can effortlessly move 720 tons of earth with a single movement; a VLSI fabrication plant allows a designer to create elaborate sub-micron structures. Without tools the thousands employed in a car factory are nothing, with tools they can assemble a car in 18 effort hours. Sometimes, tools can even subsume the importance of their operators. The violinist Ivry Gitlis, considered one of the most talented musicians of his generation, said of his Stradivarius: "I have a violin that was born in 1713. I don't consider it my violin. Rather, I am its violinist; I am passing though its life." Tools are clearly an important and defining element of any profession and activity: tools help us move boulders and atoms, tools help us reach the Moon and our soul.

The objective of this new IEEE Software column is to explore the interplay between software development and the tools we apply to the problem. Skilled craftsmen set themselves apart from amateurs by the tools they use and the way they employ them. As a professional I feel I am getting a tremendous boost in my productivity by appropriately applying tools to the software construction problems I face every day. I also often find myself developing new tools, both for my personal use and for wider distribution. Column installments will discuss specific software construction activities from the standpoint of the tools we can employ, the tools of our trade. Specific topics I am planning to address include: editing, compiling, documentation, debugging, testing, configuration management, issue tracking, the development environment, tool building, and domain-specific tools. Of course, this is your column as much as it is mine, so I am always open to suggestions for different topics or viewpoints.

Under-spending on Development Tools

So, how do our tools of the trade measure up? Pathetically, by many measures. Although the software industry is large and, dare I say it, mature, the software tool industry is still in its infantile stage. This becomes readily apparent if we consider the cost of the tools we use. A 720 ton-rated shovel excavator is so expensive that the company selling it also provides financing. The cost of the VLSI fabrication plants effectively dictates the product cycles of the manufactured chips. In comparison, the tools we use for software development cost at most a few thousands of dollars per seat. Economists track capital expenditures as a way to judge the economic future of a country or sector. On the radar screen of these statistics the cost of software development tools wouldn't amount to a single blip.

Table 1. Capital expenditures in different industries

Industry	Revenue ($ bn)	Capital Expenditure ($ bn)	CE / R (%)
Semiconductors	430,360	99,577	23.1%
Motor vehicles	1,094,157	90,042	8.2%
Heavy (non building) construction	143,957	6,187	4.3%
Prepackaged software	105,356	3,402	3.2%
Programming services	18,216	438	2.4%

To substantiate the claim of capital under-spending in our industry I used the COMPUSTAT global database to compare the capital expenditures of some industries we software engineers often admire and look to as role models, against our own. Look at the numbers appearing in Table 1. The capital expenditures of the semiconductor industry amount to 23% of its revenue; this is how it succeeds in following Moore's law for more than 30 consecutive years. The robotic factories of the car industry, envisioned by the proponents of software assembly plants, soak up 8% of its revenues. Even the nomadic heavy construction industry—our perennial favorite when we compare software engineering to bridge building—spends on capital equipment nearly a double percentage of its revenues than that spent by our own custom software construction (programming services) firms.

I hear you say that the economies of software are different: software can be duplicated at a zero marginal cost, therefore the low cost of tools reflects the realities of their distribution, rather than their intrinsic value. I only wish this was true, that we are all buying expensively developed tools at rock bottom prices. I can vouch from personal experience that the effort our industry puts into developing software development tools is apparently miniscule. A couple of years ago I developed UMLGraph, a prototype of a declarative UML diagramming tool, and made it available over my web site. I wrote the first version of this tool over a single weekend, yet I regularly receive email from enthusiastic users. This fact definitely does not reflect on my programming brilliance, but says a lot about the state of the art in diagramming software and the amount of cash employers are willing to spend on purchasing diagramming (and conceivably other software development) tools.

What would happen if an established tool vendor with deep pockets decided to build a software development tool by investing the kind of money associated with a chip plant? (Mind you, I recognize the difference between chip production and software design, my argument concerns capital expenditures over the entire product life cycle.) According to Intel financier Arthur Rock, the cost of capital equipment to build semiconductors doubles every four years. Currently a chip plant under construction costs over $2.5 billion. To put this number in perspective, consider it represents about 13,000 software development effort-years. This is almost three times the effort invested in the development of OS/360 (5000 effort-years) and, according to my calculations, almost the same as the effort invested in the development of the Windows NT line, up to and including Windows 2000. Investing this kind of money on a design tool could buy us round trip model-based software development that actually worked under realistic conditions. Investing this kind of money on a compiler could buy us type checking integrated across the presentation, application logic, and database layers, or the ability to generate provably correct and efficient code. We could also have at our hands debuggers that would be able to execute the program forwards and backwards, editors that would let us navigate between diagrams and source code, effortlessly performing sophisticated refactoring operations, and infrastructure to test an application's GUI delivered as part of our IDEs. To get a picture of the lag between what is theoretically possible, and what tools provide in practice, scan the conference proceedings of the last five PLDI and ICSE conferences, and see how few of the results reported there are now available to developers for everyday use.

Tools Underused

As if our under-spending on software development tools was not worrisome enough, a related problem in our profession is our failure to use the most appropriate tools for a given task. Here is my list of the Ten Software Tool Sins.

10. Maintaining the source code's API documentation separately from the source code.
9. Failing to integrate automated unit testing in the development process.
8. Using paper forms, email folders, and Post-it® notes to track pending issues.
7. Painstakingly analyzing the effects of a source code change in cases where the compiler and the language's type system can more reliably do the job.
6. Refusing to learn how existing tools can be made to work together through scripting or a shell interface.
5. Ignoring or (worse) silencing compiler warning messages.
4. Maintaining isolated copies of the source code base for each developer, and performing software configuration management using email attachments or the trendy new technology of USB dongles.
3. Locating definitions of program entities through a mixture of guesswork and sequential scanning through the source code.
2. Adding temporary print statements in the source code instead of using a debugger.
1. Performing mechanical repetitive editing operations by hand.

I often spot mature developer colleagues committing the number one offence in the list by the sound of their keyboard: a repetitive click-clack-clack, click-clack-clack, click-clack-clack typing pattern gives the game away. This sin is inexcusable, as (free) editors with sophisticated text processing capabilities have been available for over 30 years. Other sins, such as the number two in the list, are admittedly a mixture of tool immaturity and developer laziness. The Linux 2.4 kernel contains 65000 printf/printk statements, the FreeBSD kernel another 17000. Many of these statements can be explained by the poor support of most debuggers for embedded and system software development; a shortcoming that is becoming increasingly important as more and more software is developed for embedded devices. In my experience, many other sinful habits can be traced back to our university days. In academia the dirty mechanics of software development are often regarded as a less than respectable activity. Software tools get in the way when teaching the "Introductory Programming" course, and would take valuable time away from discussing lofty theories when teaching "Software Engineering". Students are therefore left on their own, many graduating and still writing their software with Windows Notepad.

The Silver Lining

There is really no need to end this column in a sullen mood. Software is a great lever. The little our industry has invested in tool development has provided us with numerous admirable and sophisticated tools. The many volunteers working on free and open source software projects are further increasing our choices for mature development environments and tools. It is up to us to make the best of what is available, and, why not, contribute back to the community.

^* This piece has been published in the IEEE Software magazine Tools of the Trade column, and should be cited as follows: Diomidis Spinellis. The tools at hand. IEEE Software, 22(1):10–13, January/February 2005. (doi:10.1109/MS.2005.23)

Read and post comments