Thursday, October 16, 2008

Getting better software into manufactured products

Getting better software into manufactured products

The problems of embedded software are rooted in the legacies of hardware development.

March 2006

Software development increasingly dominates the engineering process in consumer and industrial products as small as cell phones, as commonplace as washing machines and automobiles, and as large as farm equipment, mining tools, and airplanes. At companies that once focused largely on the mechanics of hardware, the primary challenge today is developing high-quality, reliable software to embed in these products. Siemens, for example, now employs more software engineers in its high-tech businesses than do large software companies such as Microsoft, Oracle, or SAP. The focus and value in engineered products is shifting from chips to code.

As the focus shifts, companies accustomed to managing the development of their hardware need to learn new processes and metrics for managing the development of software so that they can make the process more productive and the software more reliable. Hardware typically involves much less uncertainty about how the elements of a system work together: something connects or it doesn't. Software development involves shades of gray. Because there are more levels of connectivity and greater integration with other systems, it's often hard to uncover all of the side effects during the testing stage.

Automobiles make a good case study for demonstrating the challenges of this mounting complexity and for devising a road map to improve the quality and reliability of embedded software, which now drives most of the industry's innovations and accounts for a substantial and rising part of each new car's value (Exhibit 1). Embedded-software systems control a wide variety of automotive applications and handle a number of fundamentally different challenges, such as those of suspension control and satellite navigation—all while exchanging information in real time. High-end models from German and Japanese automakers use 65 to 100 electronic control units (ECUs), making each vehicle a real-time computing network requiring extremely high reliability. Unfortunately, embedded software has, at best, a varied record of quality: newspapers regularly report failures and recalls.

Chart: More money, more problems

To improve the reliability of embedded software—and to protect the reputations of automotive brands—carmakers and their suppliers must find ways to improve its quality. This won't be easy to do, but other industries have succeeded by adopting more mature architectures and improved procedures for design, development, and testing. The aerospace industry, for example, is adopting central computing architectures that promote reuse and help companies manage complexity and meet strict reliability and quality standards.1 For some commodity handsets, several players in the mobile-phone industry have developed the Symbian standard interface between the software protocol stack and applications. This approach helps manufacturers collaborate with suppliers and thus speeds up time to market.

Proprietary advantage is crucial in the automotive industry, and organizations such as Autosar (Automotive Open System architecture) and Jaspar (Japan Automotive Software platform and architecture) are having difficulty establishing standards.2 But even without them, individual players and the communities of companies that supply each major automaker can work on improving the reliability of embedded software and, therefore, the quality of the auto itself.

Building better software

Improving software is a responsibility that automakers must share with the component suppliers they rely on for many of their systems. McKinsey is conducting research into the economic impact, on the automotive industry, of complexity and design choices in embedded software, as well as engaging in discussions with leading executives at automakers and their suppliers. This work suggests four key areas where concerted, cooperative efforts could substantially boost the quality of embedded systems and the productivity of their development.

Reduce the complexity of features

Anyone who has flipped through the 500-page instruction manual accompanying a new car knows that automakers add more features every year. The industry should look hard at the value each of them adds to a car, since their complexity has major implications for quality ratings and warranty costs. Our research suggests that productivity in small-scale projects (up to 300 feature points, according to the metric we use to express complexity) is three times higher than it is in large projects involving more than 3,000 features.

Automakers should analyze the way consumers and ratings organizations evaluate features and then design cars to meet these standards without additional complexity. An airbag supplier, for example, recognized that airbags are judged by the number of stars they receive in the New Car Assessment Program (a global standard of safety and quality), so the company focused on software features that would be visible in the test. This move limited the scope of development and allowed the company to beat competitors to market with new designs and avoid needless complexity.

Adopt a more mature software architecture

Good software architectures—the patterns or frameworks that underlie software design—are modular, with a well-organized system of layers, as well as libraries of code that allow developers to reuse components. But in practice, many embedded-software developers rely on architectures that have grown up incrementally and often haphazardly, resulting in what is sometimes called "spaghetti" code. In fact, our research finds that the architecture of embedded software is one of the weakest spots of its development, lagging half a grade behind the development of comparable traditional software.3

This shortcoming must be corrected, since the maturity of software architectures has a big effect on productivity: more mature ones are more modular and standardized, reuse components, and have fewer point-to-point interfaces. Our research has found that moving from one maturity grade to the next can make development efforts 2.5 times more productive (Exhibit 2). In most cases, the improvement comes from abandoning an existing platform and then adopting a more developed architecture as a corporate standard. With further improvements—leveraging standard operating systems, hardware, and integration platforms—developers of embedded software could aspire to the level of productivity that traditional PC software developers now achieve. A telecom-equipment company that switched from its proprietary system to an embedded Linux architecture found that it could develop new features six times faster. A supplier of engine controls invested in a flexible, layered architecture and cut time to market from four months to three weeks.

Chart: Mature platforms are more productive
Understand the economics

Cost pressures can tempt suppliers and automakers to try to save money by selecting less expensive inputs, but this approach often raises warranty costs down the road. In hardware, cheaper components may break or wear down faster. In embedded systems, faults usually emanate from poor design or insufficient testing. When suppliers and OEMs undertake design-to-cost planning, they must learn to consider a broader set of criteria, including the lifetime costs of complex software and less expensive hardware.

One automaker looking to reduce the cost of its components had to analyze whether it would make sense to use less expensive hardware, since this option would require additional software development and could lead to more defects. After factoring in the higher warranty costs likely to result from this added complexity, the company calculated that it would break even only if it produced more than 260,000 units. By using more expensive hardware, it lowered its breakeven point to 120,000 units.

Improve development processes

Many companies that use embedded software—whether in the automotive, telecommunications-equipment, or consumer electronics industries—have a strong engineering tradition. As a result, they often take a hardware-driven approach focused on physical components and tend to have a mechanical view of how pieces should fit together. Such companies need to improve their skills in developing and integrating software systems.

Over the past 20 years, the software industry has professionalized its approach to development by adopting global quality measures such as the Capability Maturity Model or those based on function points. But software development in embedded systems has lagged behind the industry. Fewer software-development tools are available for embedded systems, and those that exist tend to focus on parts of the design cycle rather than the whole. Because embedded development requires expertise in software and in hardware engineering and physics, best practice includes the use of cross-functional teams of experts (feature-based development), development methodologies that apply common models and simulation tools, and "time boxing" (giving developers specific deadlines for deliverables). By instituting cross-functional teams and time-boxed development, one navigation systems company boosted its productivity to more than twice the industry average.

Implementing improvements

As in other industries where products increasingly rely on software, the automotive supply business needs to shift from a hardware-development mentality to one more attuned to the iterative nature of software development. As compared with many other software developers, those who create the automotive industry's embedded software are less likely to follow norms that have boosted productivity elsewhere—for example, measuring both the productivity of the development process (say, by feature points per person, per month) and the quality of the software (defects per feature point). These developers still labor under the hardware-development legacy, so hardware-oriented productivity measures still dominate their thinking. Until companies learn to deploy software-oriented metrics to measure the productivity of the development process, they will struggle to improve their performance and to make the right trade-offs, such as increasing their up-front investment in architecture to raise the quality of the software.

Managers will also have a tough job changing established mind-sets. Within the company, they will have to persuade engineers to adopt metrics that closely monitor productivity—something that few employees rush to embrace. They will also have to make compelling fiscal arguments for trading out established architecture and development processes for more productive internal standards. Examples from other industries can help managers make the case for change, and the job will inevitably become easier as more auto suppliers get up to speed. In the larger web of the automotive supply network, individual players will have to become more familiar with how their systems interoperate with those of their suppliers. Proprietary advantage is important, but they can no longer afford to work within a black box, hoping that automakers will make all the necessary connections. In the absence of strong industry standards, systems integrators will have many new opportunities.

Auto manufacturing is just one industry—albeit a high-profile case—where the potential of embedded software is being squandered and the weakness of current models for developing it has been exposed. If auto and other manufacturers are to protect their reputation for quality, they and their suppliers will have to raise the quality of embedded software and make its development more productive while reducing time to market.

About the Authors

Wolfgang Huhn and Marcus Schaper are members of McKinsey's global IT practice. Wolfgang, a director, leads the industrial-IT practice and is based in Frankfurt; Marcus, an associate principal, specializes in managing the performance of IT, particularly software and applications, and is based in Hamburg.

This article was first published in the Spring 2006 issue of McKinsey on IT.


1For example, the Airbus A380 incorporates the Integrated Modular Avionics architecture, an established open architecture that uses common, standardized software components.

2Autosar is a standards body representing about 35 automakers and suppliers, with the goal of establishing common, open standards for electronic systems in automobiles. Jaspar is a similar standards organization for Japanese automakers and suppliers.

3McKinsey's maturity-assessment model assigns grades from 1 (ad hoc, nonmanaged) to 4 (world class).