Boeing's Misstep Is Indicative of Broader Misconceptions of Risk

In an announcement made to the chagrin of its credibility, Boeing Co (BA) revealed that the 787 Dreamliner's maiden flight would once again be delayed, due primarily to issues surrounding the proper attachment of the aircraft's wing to its body. This revelation prompted a round of excoriating commentary, accompanied by the necessary downward revisions to earnings estimates, from several equity analysts.

Although management will bear the brunt of the fallout from the Dreamliner misstep, we would not characterize the embarrassing event as being indicative of Boeing's inability to manage a new product from conception to delivery, but rather as an over reliance on computer driven models - a condition that parallels many aspects of the current US economic predicament.

The specifics of Boeing's troubles were outlined in a WSJ article from yesterday's paper, which cited the difficulties associated with certain carbon-fiber composite parts as the Company's primary stumbling block. The composite parts, desirable partly because of their light weight, pose an interesting problem for airline manufacturers: Advanced computer models display an inability to accurately predict the behavior of these composites under actual flight conditions. While the laws of physics and the elemental composition of the metals lend themselves to calculable outcomes under laboratory conditions, there is something about the introduction of real wind, velocity, acceleration etc. that defies the computer's ability to conduct its "stress tests".

In his new book Shopclass as Soulcraft: An Inquiry Into the Value of Work , Matthew Crawford addresses this phenomenon by referencing some wisdom he once received from his mathematically inclined father. The gist of this wisdom was that a shoelace knot, by all mathematical accounts, could always be untied by tugging at a single end of the knot. Even at a young age, Crawford was able to identify the inherent shortfalls of his father's statement; namely that while the conclusion might be valid under perfect conditions, it largely ignored the possibility that a shoelace might be wet, or compromised in some other way as to negate its "easy removal" property.

Unfortunately, such blind reliance on mathematical assumptions is not limited to shoelaces. As evidenced by Boeing's costly delays, airline manufacturers have come to rely heavily on computer models whose predictions apparently fail to capture real world variables. Furthermore, there is an argument to be made that an over reliance on computer driven and mathematical models, in an attempt to capture the elusive concept of "risk", has contributed to the current state of financial markets and the economy.

We will first clarify our position on this matter by stating that we are not categorically opposed to the use of quantitative models as a risk measurement tool, provided that the user of the said tool has sufficient respect for the model's fallibility. Additionally, we acknowledge the argument that, in the midst of pursuing financial gains, individuals or organizations will often choose to ignore sobering evidence, and in some cases manipulate the means by which said evidence is generated altogether. However, we would observe that the point at which computer based and statistical models are the most vulnerable, i.e. most susceptible to providing an incorrect prediction, is the point at which the model meets reality. Boeing's moment of realization arrived when its composite parts were introduced to climatic and environmental variables that simply do not exist in the vacuum of computer models.

For those involved in the securitization of sub prime mortgages, and in particular those who either constructed or trusted the models which purported to predict default / delinquency / foreclosure rates across a pool of mortgages, the fact that something went wrong has been apparent for some time now. The historical data that guided mortgage performance projections was collected during both economic expansions and recessions, but never during an expansion that was accompanied by a housing boom of such colossal proportions. As speculative purchases proliferated, human nature introduced a wild-card to the equation that the models had not foreseen: Many homes were purchased with the intent to make a quick profit, and many of those buyers were ready, willing and able to stop servicing their mortgages when it became apparent that those profits would not materialize.

The widespread development of this mindset was the point at which the computer models were faced with a reality that did not conform to their view of the world. Models based upon the assumption that the risk and return of a financial time series is normally distributed are potentially best suited for markets dominated by professionals, having a predictable and uniform set of goals and responses.

However far removed a mortgage may be from the original loan underwriting decision, it can never escape the fact that an individual must continue to remit payment on a monthly basis. Whether it be shoelaces, composite material used in airline construction, or the performance of speculative mortgages, the important thing to acknowledge, and respect, is that aspects of this world will continue to defy prediction via statistical and computer based assignments of probability.

Disclosure: No position in any securities mentioned. The author does not stand to benefit in any way from the sale of Matthew B Crawford's book.

Comments

[molson360:]

Engineering is an art we all rely on and should be valued by society as such. Time to start teaching our kids we cant all be MBAs and Lawyers. The computer modeling has significantly scaled back the number of engineers working on any given problem be it structal analyisis of tall buildings or airplane wing mechanics. Good for the bottom line in the short term however for each brain lost is missed chance at uncovering a problem while still on paper and not at 35,000 FT.

Jun 25 10:42 AM

[Cyarg:]

Computer simulations can only deal with known data or events with some anticipation of events. With ground breaking technology there will be unanticipated events, remember the Comet please. Boeing is doing it right by testing before flying and making corrections just as it did with the 707. The Comet ended up being first for too many events as it turned out. Read your history along with your short-term bottom lines

Jun 25 11:44 AM

[Carneades:]

That's a good point about computer simulations. Isn't it though, the point which I was ultimately making?
I had written a paragraph on the corporate responsibility angle, but decided that it strayed from the primary topic a bit too much. It is interesting that, at a time when a lack of corporate responsibility is cited as a major reason for our current problems, an airline maker is lambasted for delays that, while negative to the bottom line in the short run, will ultimately contribute to the safety of flyers.


On Jun 25 11:44 AM Cyarg wrote:

> Computer simulations can only deal with known data or events with
> some anticipation of events. With ground breaking technology there
> will be unanticipated events, remember the Comet please. Boeing
> is doing it right by testing before flying and making corrections
> just as it did with the 707. The Comet ended up being first for
> too many events as it turned out. Read your history along with your
> short-term bottom lines

Jun 25 11:59 AM

[donzelion:]

Mathematical modeling of very simple, physical properties (e.g., atmospheric conditions) becomes very complex, unreliable, and indeterminate over long periods of time. It's the height of arrogance to presume human financial behavior is simpler to model than something as simple as rainfall at a specific location 365 days in advance, and the depth of folly to believe comparable claims in other contexts.

That said, engineering is an altogether different beast. With aluminum, steel, fabric, and what-not, you have decades of experience to supplement the models - but it's still a process of testing, testing, re-testing, and testing once more.

Jun 25 01:29 PM

[history rep...:]

computer simulation is not limited to "known data". models with very simple rules can yield very complex, surprising, unknown outcomes.

the relevant point is that every simulation is based on an approximate model. even if one could simulate every atom in boeing's airplane, it would be necessary to use approximations for the forces between the atoms. the actual simulations are of course not atomistic and rely on approximate laws of material properties. while "linear" laws for small stresses and strains are well known, nonlinear behavior is poorly understood. the simulation can't be better than the underlying model. this is a longwinded way to say the 1'st commandment of simulation, "garbage in, garbage out".

most simulators i know are acutely aware of the limits of their model. others, especially those who had nothing to do with making up the model but were handed a canned program to run, can fall into the trap of taking it much more seriously than is warranted.

Jun 25 03:05 PM

[notsosmart:]

& so it is with climate change.

Jun 25 03:57 PM

[Tom o'Bedlam:]

WHAT A BUNCH OF BULL.

You must have read the dust jacket of The Black Swan, and smoked something and thought it might apply to Boeing.

They are trying something new, and it was really stupid to set themselves a tight schedule. Period.

Jun 25 10:33 PM

[User 437111:]

There is a huge difference between financial simulations and engineering simulations. The engineering simulations are at least constrained by the laws of physics. Financial simulations are trend analyses based upon past trends, something that completely fails when you have a huge bubble pop. The problem with modeling composites is they are not homogeneous materials like metals, but their strength all depends on fabric orientations. The 787 is one of the most tested planes in a wind tunnel ever with scale models. They dont just pop stuff onto a plane like some redneck mechanic

Jun 26 11:35 AM

[Concordius:]

The article is right on the money but some readers prognostications are not. This is not a simple fix for the 787: Reinforcement of the weak points will undoubtedly change the entire stress pattern in the wing-fuselage connection. It is therefore imperative that pre-flight testing be begun again from scratch, on several specimens.
That composite structures would defy computer analysis should have been obvious. Fastening separate composite structures together or to structures made of conventional materials adds another set of problems: How many vertical stabilizers need to snap off their connectors in flight before the industry realizes that stuctures that can not "give a little" plastically under extreme loading are too brittle for flight applications?

Jun 26 11:37 AM

[BeachRider:]

There has been a lot of conjecture about the nature of the change needed for 787. The FAA will certainly 'assist' BA in examining the body-of-evidence for completing the testing. BA should be nervous that the FAA will find some unacceptable shortcut. There is also a scent of schadenfreude in some of these comments, given the low level of actual detail available in the press, right now.

Jun 28 09:10 PM