The double crash of the Boeing 737 MAX was an apocalyptic event. I will detail later, but I can anticipate that the repetition of the same accident is one of the reasons that allow qualifying it as apocalyptic. Nowadays, it is crystal clear that the same software flaws, i.e., flaws caused by software defects, engendered both accidents. The imputed code is located at least in two different software modules: the Maneuvering Characteristics Augmentation System (MCAS), a flight control software, and the display system software. The role of the MCAS is to prevent the plane from stalling (loss of lift) by maneuvering the trim to push the nose of the aircraft down, based on readings from two AOA (Angle-Of-Attack) sensors placed on each side of the nose. The display system software should activate the AOA Disagree alert when it detects a discrepancy between the readings of the two AOA sensors.

In both accidents:

  • one of the AOA sensors was faulty and sent readings that were incorrect and different from those of the other sensor,
  • the MCAS managed this situation incorrectly by continuing to push down the nose, notwithstanding the aircraft was not losing lift
  • the display system software did not raise the AOA Disagree alert,
  • the pilots, unaware of the MCAS system, the AOA sensors, and the sensor disagreement, were physically prevented by the repeated activation of the MCAS system to recover manual control of the aircraft.

First of all, it is necessary to emphasize that this post is not about the aircraft industry. Anyway, this industry is one of the most reliable: in 2018 there has been one fatal accident (the Lion Air 737 MAX jet crash) for 3 million commercial airline flights. That's one too many, but, on average, taking a commercial flight is safer than crossing the street.

This post is the first of a series about: 

  • failures and flaws caused by software defects, 
  • the types and characteristics of these defects, 
  • the methods and tools that can help to prevent defective software to be deployed in the field, 
  • the physical, economic, institutional, and societal consequences of the occurrence of these failures and flaws in the field (in operation), and 
  • the possible remedies if the failures and flaws occur in operation.

My opinion that is shared by all the simplyTestify team is that the software defects that are at the origin of the 737 MAX catastrophe, their manifestation as flaws in operation, and the consequences of these flaws are not at all specific to the aircraft industry. Unfortunately, they are reproducible in every human activity where the code takes control of the decisional and operational processes. Boeing is a leading aircraft manufacturer, has designed the first wide-body airplane (the 747 Jumbo), has manufactured successful planes for decades and has designed embedded software for more than forty years. If such a prominent body has not been able to prevent such a defective software from being deployed in operation, every organization in all the industries and sectors of human activity is exposed to a severe risk of suffering the same fate. 

We believe that the 737 MAX crash is as important an event for the software industry (as it is for the aeronautics sector) as Three Miles Island, Chernobyl, and Fukushima for the nuclear industry. The case of the 737 MAX is emblematic for the software industry well beyond the aeronautics sector. The lessons that the software profession could and should draw from a thorough analysis and discussion of all the topics mentioned above (causes, consequences, preventive actions, remedies) concerning the 737 MAX double accident are general and can be applied to the design and use of software in all areas of human activity.

The real problem, beyond the horrible disaster, is that, for the time being, analysis and discussion are limited to the aeronautics sector, as if other industries and the software professionals were not involved and could do without it. 

What is puzzling today is that there is a lot of in-depth discussion in institutional bodies and core working groups on a wide range of important ethical, social and political issues related to the digital transformation, such as privacy protection, ethics of AI (beneficence, non-maleficence, autonomy, justice, intelligibility and accountability)[2], lethal autonomous weapon systems. On the other hand, there is no debate on the more fundamental epistemic issue of the software functional reliability, namely the assurance that software does what it is supposed to do and does not do what it is supposed not to do.

If we fail to take a substantial step forward in our ability to ensure the functional reliability of the software we produce, any ethical, legal and political discussion about the intentions we would implement with the software is limited and even seems to be pointless.

This first post is about the consequences of the occurrence in the field of failures and flaws caused by software defects. It is deliberately scaring: our intention is to underline the urgency of opening the discussion. The consequences are categorized as physical injuries, regulation problems, liabilities, customer loss, profit loss, reputational damages, competitive disadvantage, and time-to-market delays.

Physical injuries

Among the consequences, the first and the most important are the physical injuries: two jets crashed, and 346 people died. Moreover, these horrible events are only the starting point of a chain of other adverse circumstances, such as liabilities, customer loss, and reputational damages.

Regulation problems

US congressional committees, US federal investigators, aviation authorities, airlines, and aircraft manufacturers all over the world are looking at the regulatory framework for aircraft and the role of the Federal Aviation Administration (FAA). In particular, are under investigation: 

  • how the FAA allowed Boeing to certify the plane itself partly, and
  • how the US federal government shutdown 2018-19 (35 days) prevented the FAA (that has been shut down too - unbelievable!) to look at and pursue the validation of software updates proposed by Boeing after the first crash, with, as a result, that everything remained unchanged: not only the software but also the authorization to fly for the 737 MAX plane (unbelievable!).

In our opinion, the global regulatory framework and ecosystem are going to be changed in depth. The international role of the FAA, which has been until now a prestigious and unquestioned authority and the subsidiary regulation body for many countries, will certainly be downsized. The global certification process for each new aircraft will be distributed again (at least in the meantime) across authorities and countries and will become costlier and longer than the current one. 

Liabilities

Boeing is facing lawsuits by passengers’ families and airlines. Airline customers are also suing Boeing and the carriers together. For example, Southwest customers have sued the carrier and Boeing claiming that the two companies have agreed to hide a fatal design defect on Boeing's 737 MAX. Ongoing federal investigations can bring other lawsuits, against Boeing and the US FAA too. The liability cost for Boeing but also the carriers will certainly be heavy and especially difficult to foresee. Moreover, there is a direct correlation between liabilities and reputational damages.

Customer loss

During the recent visit to Paris by Xi Jin Ping, the China Aviation Supplies Holding Company (a 737 MAX operator) suddenly bought 300 Airbus planes.  Customer loss does not concern only Boeing, but also the airlines that operate the 737 MAX. Southwest Airlines, which uses more 737 Max aircraft than its American competitors, is canceling 150 of its 4,000 daily flights. On the other hand, Delta Airlines, which does not fly the Boeing 737 MAX, has benefited from customer spillover from its domestic competitors. Delta increased its capacity by 4.9% in the first half of the year, for which the company expected growth of 3%. Probably, other companies that decided not to operate the 737 MAX will enjoy an unexpected growth. Of course, there is a strong correlation between customer loss and profit loss. 

Profit loss

It is too early to get figures but, independently from other considerations, the costs for Boeing and the 737 MAX operators will be in the billions. Only for the first half of the year: 

  • 37% fewer planes have been delivered, 
  • no return-to-service date for the grounded 737 MAX has been planned, 
  • the delivery of the 5000 737 MAX placed orders has been suspended sine die, 
  • a lot of unfulfilled orders have been canceled, and
  • production means have been 20% reduced. 

Financial experts estimate that this situation will affect Boeing’s financial results at least until 2021. Note that each 737 MAX comes with a united cost of $121.6 million. Current convergent estimations by several analysts total the cost for Boeing to $10 billion, which corresponds to the expense in research and development of an all-new aircraft with the improved features that motivated the design of the 737 MAX. The top operators of the 737 MAX aircraft (Southwestern, GECAS, American Airlines, Air Canada...) will be in trouble because of the canceled flights, the grounding of the already acquired jets, and the undelivered orders. American Airlines has declared that Boeing 737 MAX jet grounding cost it $185 million in the second quarter. Chinese airlines are seeking compensation from Boeing for the grounding of its 737 MAX jet that is on track to result in losses of more than $500 million for the carriers. 

Reputational damages

According to a Business Insider survey, 53% of American adults say they do not want to fly a Boeing 737 MAX even after repair. Consumer advocate - and former presidential candidate - Ralph Nader, who lost a relative in the Ethiopian 737 MAX accident, said the Boeing 737 MAX is defective and should never fly again. The specialized media raise the question of Boeing's survival, and the usual "too big to fail" consideration (prefiguration of a US government rescue plan) does not benefit the brand.

Competitive disadvantage 

It seems reasonable to think that Airbus, the principal and still unique competitor of Boeing on the specific “737 MAX / A320 Neo” segment, even if it has been sober and measured in its public communication on the jet crashes (which, from another point of view, is not reassuring either), will try to take commercial advantage of the situation. In December 2018, after the first 737 MAX crash, Saudi airline Flyadeal signed an agreement with Boeing worth up to $6 billion for 30 737 MAX, with an option for 20 others. A week ago, the airline placed the order with Airbus, with a new agreement for the same number of A320 Neo aircraft. Note that Boeing launched the 737 MAX in production to compete directly with the Airbus A-321 Neo. The temporary or permanent unavailability of the 737 MAX puts Boeing at a disadvantage compared to its competitor.

Time-to-market delays

Time-to-market delay is probably the harshest consequence and a real nightmare for the business. Indeed, the point is less the delay than the fact that, four months after the crash, the date at which the 737 MAX will be allowed to fly again, if any (with Ralph Nader working hard to ground it permanently), is not known. The ballet of the announced return-to-service dates since the Ethiopian jet crash has been a little pathetic, in a way scary (short-term dates are continuously replaced by future dates that are always short-term), certainly not reassuring, and in any case disrupting for airlines that continue to cancel flights, try to manage grounded planes, and wait for orders that cannot be delivered. Moreover, the suitable and praiseworthy appeal by Boeing to all the airlines and aviation authorities to participate actively to detect, debug, and fix the software defects has resulted in the discovery of 

  • other MCAS errors, 
  • other bugs not directly related to the jet crashes, in different software modules such as the Auto-pilot, and
  • even flaws that require hardware fixes. 

The mechanical consequence of these ongoing in-depth investigations is the continuous extension of the delay. The stunning outcome could be that the 737 MAX, once cleaned up from software defects and hardware flaws will probably be - as promised by the Boeing CEO - the safest aircraft in the history of commercial aviation, but perhaps too late.

Conclusion

The list of adverse consequences of the 737 MAX software failures involved in the double jet crash is frightening. How is it possible that the deployment in operation of a few lines of code had engendered the chain of catastrophic effects enumerated above?  In the next post of the series, we will discuss the types and characteristics of the 737 MAX software defects.

References 

[1] Somer, J. (2017, September). The Coming Software Apocalypse. The Atlantic.

[2] Floridi, L., & Cowls, J. (2019). A Unified Framework of Five Principles for AI in Society. Harvard Data Science Review.