Toyota Case: Single Bit Flip That Killed翻译

Toyota Case: SingleBit Flip That Killed

Toyota Case: Single Bit Flip That Killed

Junko Yoshida

10/25/2013 03:35 PMEDT

 

MADISON, Wis. —Could bad code kill a person? It could, and it apparently did.

 

The Bookout vToyota Motor Corp. case, which blamed sudden acceleration in a Toyota Camry fora wrongful death, touches the issue directly.

 

This case -- oneof several hundred contending that Toyota's vehicles inadvertently (非故意地) accelerated -- was the first in which a jury 陪审团 heard the plaintiffs' attorneys supporting their argument withextensive testimony 证词 fromembedded systems experts. That testimony focused on Toyota's electronicthrottle control system -- specifically, its source code.

 

The plaintiffs'attorneys(原告律师) closedtheir argument by saying that the electronics throttle control system causedthe sudden acceleration of a 2005 Camry in a September 2007 accident thatkilled one woman and seriously injured another on an Oklahoma highway off-ramp.It wasn't loose floor mats, a sticky pedal, or driver error.

 

An Oklahomajudge announced that a settlement to avoid punitive damages had been reachedThursday evening. This was announced shortly after an Oklahoma County juryfound Toyota liable 有责任的for the crash and awarded $1.5 million of compensation to JeanBookout, the driver, who was injured in the crash, and $1.5 million to thefamily of Barbara Schwarz, who died.

 

During thetrial, embedded systems experts who reviewed Toyota's electronic throttlesource code testified that they found Toyota's source code defective 有缺陷的, and that it contains bugs -- including bugs that can causeunintended acceleration.

 

"We'vedemonstrated how as little as a single bit flip 蹦跳 can causethe driver to lose control of the engine speed in real cars due to softwaremalfunction 失灵 that is notreliably detected by any fail-safe," Michael Barr, CTO and co-founder ofBarr Group, told us in an exclusive interview. Barr served as an expert witnessin this case.

 

A core group of seven experts, including four from BarrGroup, analyzed the Toyota case. Their analysis ultimately resulted in Barr's800-plus-page report.

 

In Toyota's ownview, though, the automaker had been already exonerated使免罪 when the National Highway Traffic Safety Administration closed itsprobe of Toyota models in February 2011. The NHTSA decision came after NASAinvestigated Toyota's electronic throttle control system and found noelectronic causes of unintended 无意识的；非计划中的accelerationduring a 10-month review.

 

But not everyonein the embedded systems industry thinks NASA had enough time to come up with acomplete report. Perhaps more significantly, in its report, NASA itself did notrule out the possibility of software having caused unintended acceleration.

 

The group ofseven experts was given the task of picking up where the NASA investigationleft off.

 

"We did afew things that NASA apparently did not have time to do," Barr said. Forone thing, by looking within the real-time operating system, the expertsidentified "unprotected critical variables." They obtained andreviewed the source code for the "sub-CPU," and they "uncoveredgaps and defects in the throttle fail safes."

 

Further, theteam ran simulations in the Green Hills Simulator. "This confirmed taskscan die without the watchdog resetting the processor." His group alsoindependently checked worst-case stack depth. "We found many big mistakesin the Toyota analysis that NASA relied on."

 

The expertsdemonstrated that "the defects we found were linked to unintendedacceleration through vehicle testing," Barr said. "We also obtainedand reviewed the source code for the black box and found that it can recordfalse information about the driver's actions in the final seconds before acrash."

 

It's importantto note Barr Group testimony led to a billion-dollar economic-loss settlementby Toyota last December. Because of that settlement, details of the technicaldiscoveries made back then by the experts were not made public until theOklahoma trial. The economic-loss settlement resolved hundreds of lawsuitsclaiming vehicles depreciated after the company issued recalls related tofaulty acceleration. Toyota still faces lawsuits claiming injury or deathrelated to the recalls.

 

Task X death

Now that theexperts' testimony and findings have been made public through the Oklahomatrial, let's get into details. What defects were found in Toyota's electronicthrottle control systems?

 

Barr said thatthe 2005 Camry L4 source code and in-vehicle tests by the experts confirmedthat some critical variables are not protected from corruption, and sources ofmemory corruption are present. He believes that Toyota's engineers sought toprotect numerous variables against software- and hardware-cause corruptions,but they failed to mirror several key critical variables, and they made nohardware protection available against bit flips.

 

Stack overflowand software bugs led to memory corruption, he said. And it turns out that thecrux of the issue was these memory corruptions, which acted "likericocheting bullets."

 

Barr explains the issue this way:

 

Memorycorruption as little as one bit flip can cause a task to die. This can happenby hardware single-event upsets -- i.e., bit flip -- or via one of the manysoftware bugs, such as buffer overflows and race conditions, we identified inthe code.

There are tensof millions of combinations of untested task death, any of which could happenin any possible vehicle/software state. Too many to test them all. But vehicletests we have done in 2005 and 2008 Camrys show that even just the death ofTask X by itself can cause loss of throttle control by the driver -- even ascombustion continues to power the engine. In a nutshell, the fail safes Toyotadid install have gaps in them and are inadequate to detect all of the ways UAcan occur via software.

Just to clarify,the "tasks" are equivalent to apps running on smartphones or PCs. Allsoftware malfunctions from time to time -- we often have to reboot ourmachines. The 2005 Camry L4 has a set of dozens of apps (or tasks). Becausethey are all meant to be running always, the death of one could have direconsequences.

 

When asked ifthe whole case for unintended acceleration could be pinned on the task X death,Barr replied, "The task X death in combination with other taskdeaths." There are dozens of tasks and 16 million different ways thosetasks can die. The experts group was able to demonstrate at least one way forthe software to cause unintended acceleration, but there are so many other waysthat could have happened.

 

Barr also saidmore than half the dozens of tasks' deaths studied by the experts in theirexperiments "were not detected by any fail safe."

 

What's next for NHTSA

After theOklahoma trial, what steps should the NHTSA be taking? Barr made somesuggestions:

 

NHTSA needs toget Toyota to make its existing cars safe and also needs to step up on softwareregulation and oversight. For example, FAA and FDA both have guidelines forsafety-critical software design (e.g., DO-178) within the systems they oversee.NHTSA has nothing.

Also, NHTSArecently mandated the presence and certain features of black boxes in all UScars, but that rule does not go far enough. We observed that Toyota's black boxcan malfunction during unintended acceleration specifically, and this willcause the black box to falsely report no braking. NHTSA's rules need to addressthis, e.g., by being more specific about where and how the black box gets itsdata, so that it does not have a common failure point with the engine computer.

译文：

糟糕的代码可以杀死一个人吗？是的，而且事实已经证明了。

丰田汽车的例子：

丰田汽车公司的丰田凯美瑞，它的突然加速可以致人死亡。

         原判：

原告律师的证词集中在丰田的电子节气门控制系统 - 特别是，它的源代码。

原告律师结束他们的论点时说道，2005年凯美瑞的电子油门控制系统造成的突然加速在2007年9月一名女子丧生并致使一人驶出匝道重伤。

俄克拉何马县陪审团发现丰田应该负责，向两个家庭各赔偿150万美元。此消息公布后不久，周四晚上，为避免惩罚性赔偿，奥克拉荷马州的法官宣布，已经达成和解。

         后续调查：

在庭审中，审查电子油门的源代码的嵌入式系统专家作证说，他们发现丰田的源代码是有缺陷的，而且包含可能会会导致意外加速的bug。

“我们已经证明一个bit的跳转可以造成软件故障，从而导致司机在真正的汽车中失去对发动机的控制，而这种故障是无法被可靠的检测到的。”迈克尔·Barr，首席技术官、巴尔集团的创始人之一，作为专家证人，在一篇独家专访中这样说道。

在丰田自己看来，国家公路交通安全局是在美国航空航天局调查丰田汽车的电子油门控制系统之后——10个月的审查，并没有发现电子意外加速的原因，才宣布汽车制造商无罪的。但并非所有人都这么认为。美国航空航天局没有足够的时间得出一份完整的报告。更重要的是，在其报告中并没有排除软件造成意外加速的可能性。

 

因此，7位专家组成的小组继续美国航空航天局的调查。

Barr说：“我们做了几件美国宇航局显然没有时间做的事。”其中的一件事：专家通过查看实时操作系统找到了“未受保护的关键变量。”他们获得并审阅了“子CPU的源代码，”他们揭露了电子气节门的安全缺陷。

此外，团队运行了模拟器，发现了美国宇航局那份报告中很多的重大错误。他们还发现了在软件故障之前的几秒钟，系统记录的用户动作信息是错的。

最终结果：

巴尔集团的调查去年12月导致丰田损失数十亿美元，而且丰田公司还在不断召回车辆，并且仍面临很多诉讼。

 

Task X death：

现在，专家的证词和调查结果已经公布。让我们看看丰田的电子节气门控制系统中发现的缺陷。

Barr说， 2005年凯美瑞L4源代码在车辆测试由专家确认一些关键变量数据被损坏，并且内存数据也被损坏。他认为，丰田的工程师想防止软件和硬件对众多变量的损坏，但他们没能对几个关键变量作镜像，而且他们没有防止硬件的位翻转。

他还说，堆栈溢出和软件bug导致了内存损坏。实验显现，问题的关键就是这些像“来来回回弹射的子弹”一样的内存损坏。

 

Barr explains the issue this way：

仅仅是一个位的翻转导致的内存损坏也会造成一个任务的终止。内存损坏可以由多个原因引起，如：硬件原因（位反转）、软件的许多bug（缓存泄漏、代码中定义的竞态条件等）。

太多的未经测试的任务（程序）会造成终止，其中的任何一个都有可能发生在汽车/软件状态中。全部去测试他们实在是太多了。但是在我们2005—2008年对丰田凯美瑞的测试中显示：仅仅是随意一个task x都会造成驾驶员对电子气节门的失控，而这时发动机还在加速。总之，丰田的安全故障来源于软件的缺陷，并且不是软件测试可以全部检测到的。

澄清一点，这里我们说的task(任务)是不等价于运行在PC和智能手机上的app的。一次又一次的软件失灵，我们只需要重启我们的机器。凯美瑞L4有许多重要的 需要同时运行的task，任何一个task的终止都会造成可怕的后果。

那么是不是意外的加速全都是因为task x的终止呢？Barr回答说，是因为task x和其他的task共同终止造成的。有许多个task，并且有1600万种方式会造成task终止。研究小组只举出了至少这样一个造成意外加速的例子，其实还有很多可能发生的。

Barr说在实验中超过一半的task终止都没有被安全故障检测到。

 

What's nextfor NHTSA：

Oklahoma 审判之后, 国家公路交通安全管理局下一步应该采取什么样的措施呢？Barr给了一些建议：

国家公路交通安全管理局（NHTSA）应该让丰田汽车公司保证现有的车辆安全，并且应该规范化软件，更加重视软件。例如：联邦航空局（FAA）和食品及药物管理局（FDA）都有对生命攸关系统中软件的指导方针。但是国家公路交通安全管理局（NHTSA）却什么都没有。

虽然国家公路交通安全管理局（NHTSA）现在强制在美国汽车中要出现某些特性的黑匣子，但是这个规则不能长期有效。我们发现丰田汽车的黑匣子在意外的加速时是失灵的，这造成了黑匣子的错误报告并且不会刹车。国家公路交通安全管理局（NHTSA）需要去处理这种情况，比如可以通过详细定义黑匣子是从哪、怎样获得数据的方式去避免发动机系统的常见故障。