Maintainability of the FreeBSD System
Last November Ioannis Samoladas and his colleagues published an article in the Communications of the ACM [1] that compared the maintainability of open-source versus-closed source projects. I applied the maintainability index [2] they used on the FreeBSD source repository following the code's maintainability over time, and comparing the maintainability of different modules. Here are the results.
Maintainability Changes Over Time
The following two diagrams illustrate the changes of the FreeBSD kernel maintainability and size over a period of 10 years (1995-2005). The first depicts the kernel maintainability evolution, while the second depicts the maintainability evolution of all the user-space code. Higher maintainability index (MI) figures indicate a more maintainable system. As we would expect, the maintainability decreases over time, but only slightly. This is especially important, if we consider that both the kernel and the user code more than doubled in size over that period. To provide a perspective, Coleman's article [2] describes two competing systems at HP that had a difference in the MI of over 30 points, while a CMU SEI report documents how a reeingineering a US Air Force system doubled its MI.
Maintainability index over time in the FreeBSD kernel.
Maintainability index over time in the FreeBSD user-space code.
Maintainability of Modules
The last diagram depicts the maintainability index of different FreeBSD modules (a module is a collection of files grouped together for administrative purposes; a module typically represents a single command, or a subsystem). In our calculation we have not included modules less than 500 lines of code, to remove strange outliers. Based on the diagram one can argue that for this project an MI cutoff threshold of 40 can be established; modules under this value should be scrutinized.
Maintainability index of different modules in the FreeBSD user-space code.
Here are the modules with the highest and lowest MI.
Least Maintainable Modules
| Module Name | Size (LOC) | MI |
| fore_dnld | 10284 | -2.77781525697723 |
| ipfw | 8730 | 2.19445150187947 |
| ancontrol | 3564 | 7.28187026797856 |
| ee | 18584 | 7.68737418575401 |
| ipcs | 1380 | 9.27186025233512 |
| route | 3328 | 10.6633834274736 |
| ping6 | 5444 | 10.7523490117775 |
| elfdump | 2204 | 13.5413559180982 |
| ip6fw | 2906 | 15.9562542125594 |
| wicontrol | 2478 | 17.5318159363428 |
| moused | 6252 | 19.5567221067852 |
| rpcinfo | 3334 | 20.0572886254166 |
| cdcontrol | 2524 | 20.1761505886906 |
| contrib_pam_modules | 10344 | 21.4155931121246 |
| fdisk_pc98 | 2512 | 22.6477288555626 |
| fdisk | 2792 | 23.0836640302608 |
| ping | 3222 | 23.4929663251796 |
| vmstat | 2078 | 26.7612628539371 |
| chat | 3084 | 26.8113703372858 |
| burncd | 1398 | 27.4184012138496 |
Most Maintainable Modules
| Module Name | Size (LOC) | MI |
| librpcsvc | 1526 | 114.229834520675 |
| libcrypt | 918 | 115.437965055286 |
| libscrypt | 918 | 115.437965055286 |
| libypclnt | 1384 | 115.790647786048 |
| libkern | 9742 | 115.934585498123 |
| sys_libkern | 9742 | 115.934585498123 |
| libc_include | 1616 | 116.122388754993 |
| sys_ia64_include | 15392 | 116.210428306165 |
| sys_sparc64_include | 18160 | 116.278661622061 |
| sys_powerpc_include | 14672 | 117.736894781652 |
| sys_arm_include | 14368 | 117.907041705555 |
| lukemftpd | 966 | 119.875420612704 |
| contrib_openpam | 14902 | 120.021600297376 |
| vi | 828 | 121.217588381786 |
| libc_quad | 4830 | 123.358498814576 |
| kgzldr | 516 | 124.143227794042 |
| libc_sys | 774 | 129.149297651959 |
| libc_alpha | 2266 | 129.78494426648 |
| libc_string | 8584 | 131.45431196217 |
| libc_compat_43 | 1046 | 134.126190286764 |
| libc_i386 | 2084 | 135.901800173643 |
| contrib_gperf | 1418 | 136.301393878737 |
References
[1] Ioannis Samoladas, Ioannis Stamelos, Lefteris Angelis, and Apostolos Oikonomou. Open source software development should strive for even greater code maintainability. Communications of the ACM, 47(10):83–87, 2004. (doi:10.1145/1022594.1022598)
[2] Don Coleman, Dan Ash, Bruce Lowther, and Paul Oman. Using metrics to evaluate software system maintainability. Computer, 27(8):44–49, 1994. (doi:10.1109/2.303623)
Read and post comments
