Safety Evaluation of Automotive Electronics Using VirtualPrototypes: State of the Art and Research Challenges

J.-H. Oetjens1, N. Bannow1, M. Becker2, O. Bringmann3,4, A. Burger4, M. Chaari7,S. Chakraborty5, R. Drechsler6, W. Ecker7, K. Grüttner8, Th. Kruse7, C. Kuznik2,

H. M. Le6, A. Mauderer1, W. Müller2, D. Müller-Gritschneder5, F. Poppen8, H. Post1,S. Reiter4, W. Rosenstiel3,4, S. Roth1, U. Schlichtmann5, A. von Schwerin9,

B.-A. Tabacaru7, A. Viehl41Robert Bosch GmbH, 2Universität Paderborn, 3Universität Tübingen,

4FZI Forschungszentrum Informatik, 5Technische Universität München,6University of Bremen,7Infineon Technologies AG, 8OFFIS – Institute for Information Technology, 9Siemens AG

Paper contact: Jan-Hendrik.Oetjens@de.bosch.com, wolfgang.ecker@infineon.com

ABSTRACTIntelligent automotive electronics significantly improved driv-ing safety in the last decades. With the increasing complexityof automotive systems, dependability of the electronic com-ponents themselves and of their interaction must be assuredto avoid any risk to driving safety due to unexpected failurescaused by internal or external faults.

Additionally, Virtual Prototypes (VPs) have been acceptedin many areas of system development processes in the automo-tive industry as platforms for SW development, verification,and design space exploration. We believe that VPs will sig-nificantly contribute to the analysis of safety conditions forautomotive electronics. This paper shows the advantagesof such a methodology based on today’s industrial needs,presents the current state of the art in this field, and outlinesupcoming research challenges that need to be addressed tomake this vision a reality.

Categories and Subject DescriptorsB.8 [Performance and Reliability]: Reliability, Testing,and Fault Tolerance; C.4 [Performance of Systems]: Re-liability, availability and serviceability

General TermsVerification, Reliability

KeywordsSafety, Reliability, Virtual Prototypes, Embedded Systems,Automotive, Robustness

1. INTRODUCTIONWith the increasing number and acceptance of complex

electronic-based embedded systems in daily life, e.g. in

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are notmade or distributed for profit or commercial advantage and that copies bearthis notice and the full citation on the first page. Copyrights for componentsof this work owned by others than ACM must be honored. Abstracting withcredit is permitted. To copy otherwise, or republish, to post on servers or toredistribute to lists, requires prior specific permission and/or a fee. Requestpermissions from Permissions@acm.org.DAC ’14, June 01 - 05 2014, San Francisco, CA, USACopyright 2014 ACM 978-1-4503-2730-5/14/06 ...$15.00http://dx.doi.org/10.1145/2593069.2602976.

Figure 1: Combined Active and Passive Safety (CAPS)Photo: Bosch

medical aids, industrial production and cars, their risk offailure increases, which may threaten human lives or physicalconditions. Therefore, the evaluation of safety is a crucialtopic in several areas, especially for automotive electronics.Of course, several functions could be implemented withoutelectronic components however they help to increase energyefficiency (e.g. highly sophisticated motor control), com-fort (e.g. adaptive cruise control), and safety (e.g. airbags).So, year after year, more and more sophisticated electroniccomponents based on semiconductors make their way intocars making them even more efficient and safer. In so doing,new functions are often realized by the interaction of severalelectronic components, e.g., in systems such as CombinedActive and Passive Safety (CAPS), which is illustrated inFig. 1. CAPS links the data from environment sensors withthe airbag control to better protect the passengers by esti-mating direction and strength of a collision. As such it mustbe, for instance, absolutely guaranteed that the failure ofany system component does not trigger the airbag in normaloperation.

In recent years, virtual prototypes (VPs) have been ac-cepted in many areas of system development processes inthe automotive industry and other industrial sectors. In theautomotive industry, VPs are most frequently applied as plat-forms for software development, in design space exploration

and system verification. Meanwhile, we can find dedicatedframeworks for Virtual ECU (Electronic Control Unit) designand testing based on the automotive standard AUTOSAR(AUTomotive Open System ARchitecture) [1]. Moreover,we are expecting that VPs will become more important forsafety analysis and safety assurance of future automotivesystems as a result of the wide spectrum of dependabilitychallenges [2].

In this paper, we present an overview on how the correctand safe functionality of complex systems can already beexamined in an early design phase using Virtual Prototypes.A major challenge is to ensure the consistency between theVP and the real system. Here, the worst case for an incon-sistency between both is that a fault is not modeled in theVP, but impacts the real system.

On the other hand, there are many scenarios that canbe hard to apply to real physical systems. Erroneous datain arbitrary components, such as registers or memory cells,and errors in hardware components, such as stuck-at errorsor disconnected wires between two subcomponents of anASIC are difficult to insert into a real hardware prototype.Also, insertion of faults into physical HW may destroy theprototype or lead to a system state that may cause safetyissues for the operator. Additionally, in a VP it is much easierto observe the impact of the error on the system and trackthe error propagation. Even more importantly, VPs can helpfind unexpected error effects early in the design process. So,measures to keep the overall system safe can be includedat an early phase therefore avoiding costly redesigns. Asa consequence, VPs have many advantages when analyzingsafety. The current state of the art in this field is outlinedin Section 2. A large research effort is required to makethe vision of early safety assurance based on VPs a reality.The related, unresolved research challenges for applying themethodology on future complex automotive electronics arediscussed in Section 3, before we close the paper with aconclusion.

2. STATE OF THE ARTSafety and reliability of automotive and embedded systems

in general have been the focus of many research activities.This section outlines some important work in these andrelated areas.

2.1 System-Level Safety EvaluationThere are several established methods for evaluating sys-

tem safety. Fault Tree Analysis (FTA) and Failure ModeEffects & Diagnostic Analysis (FMEDA) are two examplesfor well known dependability analysis methods. Tradition-ally, these methods have to be executed by system experts,as the identification of failures, their logical interaction andthe estimation of failure rates are based on the engineer’sexperience and knowledge. To reduce the complexity of theseevaluations as well as the strong reliance on the engineer’ssystem knowledge, different approaches were developed toautomate various steps [3–6]. Pinello et al. [3] propose anapproach based on Fault Tolerant Data Flow (FTDF) graphsto create Fault Trees (FT). The Fault Propagation and Trans-formation Calculus (FPTC) [4] allows the determination ofthe system failure behavior based on information about thefailure behavior of components and their interconnections.All approaches require that the engineer has to specify anintermediate model that contains crucial information, e.g.the dependency between system components, such as dataflow or control flow dependency.

Another very challenging task for system experts, whichis necessary for the intermediate format, is the estimation of

the failure propagation of system components, especially incombination with failure tolerance.In the area of system verification, simulation runs have beenestablished for the validation of functional and non-functionalrequirements. System-level simulations are used to verifyfunctional, timing or power requirements like [7]. Executablemodels of the system under test promise a huge potentialfor safety evaluations. They cover inherently the internalcomponent dependencies and the provided error tolerance.

When using system simulation runs to overcome the chal-lenges of traditional safety evaluation methods, new chal-lenges emerge as well. In [8] an approach to implicitly supportthe FTA with an error effect simulation is shown. To providea generalized and comprehensive methodology, additionalchallenges have to be solved, which are beyond the scope ofthe presented framework. Failure behavioral patterns, whichare currently mostly informally specified, have to be trans-ferred into an executable format that can be used duringsimulation to stimulate errors. Both the simulation and thefailure specification have to fit in system models at differentlevels of abstraction, particularly with the abstract systemlevel. Additionally, methods for creating FTs from the simu-lation results or for guiding the error simulation to handleanalysis work have to be developed.

2.2 RTL and Gate-Level AnalysisIntensive research was conducted to analyze the reliability

of integrated circuits (ICs). Errors can be injected as bit valueflips in memory cells or registers during logic simulation at thegate or register transfer level (RTL) [9]. Monte-Carlo-basedtechniques can be used to model different fault scenarios.Based on the fault model, statistically varying error locationsand occurrence times - as well as stimulus [10–12] - aregenerated.

These RTL and gate level analysis methods require fin-ished RTL implementation of the investigated IC, whichmay result in expensive redesigns if weaknesses are found atthis late design phase. Additionally, simulation at the gateand RTL is usually too slow, so that acceleration techniquesare required. A prototype of the IC can be emulated onan FPGA, for instance. The code can be instrumented byerror injection capabilities [9, 11, 13] or the bit-stream canbe adapted [14,15] to inject errors in the IC during FPGAemulation. Simulation-based error injection can acceleratethe evaluation by using higher levels of abstraction, e.g.,micro-architectural level [12].

2.3 System-Level TestbenchesAs stated in Section 2.1, simulations promise a huge po-

tential for safety evaluations. A complete simulation envi-ronment consists of an executable model of the DUT and atesbench for stimulation. Design reuse and interoperabilitybecame mandatory in system design to keep design produc-tivity in pace with Moore’s law. Consequently, reuse andinteroperability had to become available for system-leveltestbenches alike and resulted into concepts such as TLM(Transaction Level Modeling) for modeling communications[16] and UVM (Universal Verification Methodology) for test-bench infrastructures [17]. TLM imitates communicationapart from specific hardware implementations across a va-riety of abstraction levels: cycle-accurate, approximatelytimed, loosely-timed, untimed, and token-level. The commu-nication architecture is kept undefined and remains open fordesign space exploration experiments, usually an applicationarea of VPs. Moreover, the different communication abstrac-tion levels allow significant speed-up for system-level modelssimulation, a crucial advantage on early safety assurance of

large VPs. Meanwhile, TLM-2.0 is widely accepted as anindustry standard and part of the SystemC standard IEEE1666-2011 [18]. Nevertheless, the concept of TLM is genericand can be found as part of the UVM, a methodology andlibrary for system level testbenches based on SystemVerilog.UVM separates IP design from IP verification, and makestestbenches and test cases reusable by employing standard-ized components, such as agents, sequencers, drivers, andmonitors. In fact, UVM components utilize TLM interfacesfor communication and make use of UVM agents to interactwith the DUT by translating sequences of TLM messagesto wire toggles by means of a translating driver, which actsas a transactor. UVM monitors are responsible for the op-posite direction of communication, as they interpret wiretoggles to create TLM messages. Together with the UVMenvironment and a concept called UVM factory, these giveUVM testbenches high reconfiguration and reuse potentialfor system-level safety evaluation.

2.4 Testbench QualificationTestbenches and virtual experimentation environments are

fundamental tools for early safety evaluations of automo-tive systems. For instance, virtual ECU frameworks arefrequently applied to provide early integration and testing ofAUTOSAR software stacks according to automotive softwaredesign processes, such as the V-Model flow. However, thesignificance of the results obtained depends on the pertinenceof test cases being stimulated by the verification environ-ment. Mutation analysis utilizes fault injection to qualify atestbench with respect to its ability to reveal faults in termsof fault activation, error propagation, and failure detection.For this, artificial faults, referred to as mutations, are deliber-ately seeded into the DUT (Design Under Test), i.e., modelsof HW/SW components. A mutant is generated by applyingone mutation operator to a single place of the DUT.

Such mutation operators were originally1 introduced torepresent typical programmer failures [19]. These failuresare modeled as simple syntactic program faults due to theCompetent Programmer Hypothesis [19], which assumes thatprogrammers are competent and write programs that arenearly correct. Further, another hypothesis called CouplingEffect [19] attempts to establish the effectiveness of mutationtesting. It states that if a set of test data is able to kill (i.e.,to monitor/detect) most of the mutants that are generatedwith simple fault seeding, then they will also be capableof revealing the real, more complex bugs in the program.Thus, the mutation score, i.e. the fraction of killed mutantswith respect to the overall number of mutants, provides anadvanced metric to assess a testbench’s quality comparedwith coverage based metrics. Mutation testing results canbe applied for automatic test pattern generation in case themutation score turns out to be insufficient [20].

The application of mutation analysis for testbench qualifi-cation has been investigated for various domains, abstractionsand languages [21], such as software (Fortran, C/C++, Java,bytecode, assembler/binary [22], Simulink [23]) and hardware(VHDL [24], SystemC/TLM [25], IP-XACT [26]). As such,various fault models are considered that cover faults relatedto specification/integration, programmer failures, and designtool failures. In [27] the authors consider fault injection for

1Mutation analysis originally addresses testbench qualifica-tion with respect to design faults. However, testbenches canbe also qualified in order to reveal software errors/failures be-ing caused by runtime faults in the digital/analog hardwareand the physical environment. As such, mutation analysiscan be also investigated for qualifying stress test environ-ments.

dependability assessment of AUTOSAR [1] models. How-ever, they do not investigate fault injection for testbenchqualification.

As the number of mutants to cover can be very large,current research mainly addresses techniques to improvemutation-based testing efficiency in order to keep up withthe ever-increasing system complexity. To achieve this, sev-eral approaches have been recently investigated, such asmutation schema [21], model abstraction [28], hardware ac-celeration [29], software emulation [30], and directed testgeneration [20,22].

3. RESEARCH CHALLENGES3.1 Overview

While using Virtual Prototypes for early safety evaluationis a very promising approach, major challenges still must beresolved to make it a reality. The first obstacle is to bridgethe big gap of abstraction levels between errors injected inVPs and real-world faults, which are mostly of a physicalnature. The next obstacle is the complexity of simulatingstress tests on VPs to analyze the impact of faults on sys-tem safety. Functional verification using VPs is relativelywell-understood today and has become a mature technology.Still, managing the complexity of functional verification forincreasingly complex systems is a formidable task. On theother hand, safety evaluation requires comparing the systemfunctionality in the presence of faults to the correct one.Given the seemingly endless possibilities of fault manifesta-tion, safety evaluation clearly adds another dimension to thecomplexity problem.

In the following, we envision a potential safety evaluationapproach to tackle these challenges. The approach combinestwo well-established methods with a novel approach: (i)Mission Profiles from the automotive domain, (ii) UVM-based testbenches from the hardware domain and (iii) erroreffect simulation. The following sections discuss each of thesethree pillars in more detail and outline concrete researchquestions.

3.2 Mission Profile-Compliant VerificationEarly system verification and validation is a mandatory

necessity in order to efficiently guarantee reliability androbustness of a system. Especially in the automotive industry,a high degree of reliability and robustness has to be ensured.Therefore, novel methods have to be employed to recognizeas early as possible any malfunctions of the system and itscomponents due to faults. Consequently, these methods mustuse parts of the system specification or design experiencesin order to define and simulate errors efficiently at an earlystage of development. As such, a novel approach to specifyrequirements and scenarios, known as Mission Profiles [31,32],has been developed in the automotive sector.

A Mission Profile (MP) defines the application-specificcontext refined for a system or a component to ensure robust-ness and reliability. The Mission Profile requirements areexpressed as a set of relevant environmental stresses, func-tional loads and operating conditions regarding a component.After a formalization process, the Mission Profiles are passeddown from the OEM to the semiconductor manufacturer.Therefore, Mission Profiles are a promising approach for rec-ognizing malfunction of a system or its components as theyconsider all system components along the entire supply chainand are available in an early stage of the design flow.

The different requirements and relevant environmentalstresses within the Mission Profiles can be used to derivetargeted functional fault/error descriptions. Additionally,

Figure 2: System Validation with Mission Profiles

the Mission Profile specifies the normal operating states ofthe system as well as operating states that describe a possiblemalfunction or a special use case, for instance the high loadfor the servo motor when steering against a curbstone. Nev-ertheless the derivation of functional fault/error descriptionsfrom Mission Profiles is a very challenging task and currentlynot yet solved.

Hence, it is necessary to thoroughly analyze the differentenvironmental stresses and functional loads to derive feasibleapplication- and system-specific fault/error descriptions. Anenvironmental stress, e.g., could describe vibration loadsfor components according to their specific mounting point.Based on this vibration load, a probability of errors dueto wiring, such as open load or short to ground, should bederived.

Based on this functional fault/error description, a stressorshould be generated, which stimulates fault/errors in theerror effect simulation. The error effect simulation is pre-sented in Section 2.1 more closely. How the stressor can beintegrated in the testbench is outlined in the next section.

As shown in Fig. 2 Mission Profiles are early availablein the design flow. Therefore, the derivation of functionalfault/error descriptions and the resulting stressors can bedone at every system level - from the OEM down to thesemiconductor manufacturer.

3.3 UVM Testbenches for Fault AnalysisUVM, as it was introduced in Section 2.3, is a well estab-

lished standard for testbenches in electronic systems design.However, not all of the requirements are addressed:

Currently, standard UVM is implemented in SystemVer-ilog. However, many other languages are applied for elec-tronic systems design such as MATLAB/Simulink, VHDLand SystemC, to name just a few. Ideally, when choosing onelanguage best suited for an individual application, it is prefer-able to always adhere to the same verification methodologyand testbench library, for instance with C-based implementa-tions. As such, the works still under development in [33,34]introduce a SystemC UVM implementation which is an initiallanguage reference definition and associated proof-of-conceptimplementation to the Accellera Systems Initiative. Alterna-tively, SVM (System Verification Methodology) [35,36] hasbeen introduced as a SystemC implementation of a significantUVM subset. Creating a unique verification methodology forall design languages is not sufficient to be able to integrate ver-ification environments from a mix of components. Therefore,the Accellera Multi-Language (ML) working group is headingfor testbench and test case reuse across multi-language veri-fication platforms, e.g. a UVM-MATLAB/Simulink environ-

ment hosting a UVM-SystemVerilog testbench, instantiatinga UVM-SystemC agent to stimulate a VHDL DUT. Systememulation platforms, like Zebu, Palladium, and Veloce, canbe seen as a special case for this. Like simulated components,emulated system parts should fit under the umbrella of multi-language based design and verification reuse. It should beof no relevance for analysis if a part of a DUT is executedby simulation or emulation. Zebu, e.g., offers transactors forsimulation to emulate TLM communication. Digital basedmethodologies have to be extended towards AMS (AnalogueMixed Signal) designs. Li et al. [37] target this by includingSystemC-AMS in their work.

Correctness analysis (verification, validation) focuses onbug detection in specifications and designs of systems. How-ever, there is a need for extending the methodologies forfault/error analysis of automotive and other systems. Veri-fied by design, a deployed system in the field is, nevertheless,vulnerable to external and internal defects. Fault models forASIC fabrication tests are available (stuck-at, short, open,...), but comparable fault/error models are missing at higherlevels of abstraction.

In Section 3.2 it was described, how mission profiles canbe used to derive formal fault/error descriptions. Thesefault/error descriptions are included into the stressor, thatwill become an additional component of the testbench forfault/error evaluation.

One of the main challenges is how to inject the faults/errors.For external faults, the stimuli, which are provided to theDUT by the testbench, need to be modified. For internalfaults, errors need to be injected into the DUT, but thedesign should not be changed. For this, we propose to addinjectors into the DUT and testbench. These provide aninterface to change the stimuli in the testbench or modifythe state or state transitions at different positions in theDUT. The stressor uses these injectors to inject faults/errorsaccording to its formal fault/error description. Additionalto this, methodologies for fault/error classification and fault-error-failure analysis are required at the monitoring side ofthe testbench.

3.4 Stress Tests and Error Effect SimulationAnalyzing the impact of errors on system safety in the

automotive domain poses major challenges, which can beaddressed by simulating stress tests using VPs. VPs coverthe system with ECUs with the integrated software, digitaland analog hardware components and the interconnectionnetwork. Stress tests based on VPs allow error effects toalready be investigated in an early design phase, by enablingwhat-if analysis of the system when errors are present. Re-peated stress tests enable a quantitative evaluation, e.g. todetermine the safety integrity level. A stress test runs theerror effect simulation in a closed loop, as shown in Fig. 3.The stressor introduces different errors according to its errorscenarios via the injectors for each simulation as was outlinedin Sec. 3.3.

Specific characteristics of automotive systems pose somemajor challenges that must be resolved when applying suchstress tests on VPs to assure automotive safety. MissionProfiles in the automotive domain may require investigatingmany seconds of real time. This directly translates to thesimulation of a vast amount of instructions of the embed-ded cores to emulate the application software. Additionally,automotive applications typically require the execution ofseveral concurrent tasks that exhibit hard and soft real-timeconstraints. To evaluate the effect of errors, the paradigm”The right value at the wrong time can still be an error.”holds. Thus, it must not only be investigated whether an

Figure 3: Error Effect Simulation applying Stress Tests

error may lead to a wrong value, but also if the error mayinduce additional delay, e.g., by applying some error correc-tion that may cause deadline violations. The verification ofreal-time requirements requires simulating time and concur-rency. Usually, current reliability-aware emulators such asVarEMU [38, 39] only take time in number of instructionsinto account. In contrast, SystemC includes a simulationkernel for event-driven simulation. It enables the evaluationof the system under development, which becomes the DUT,along the design process using abstract models in early de-sign phases to evaluate and to verify coarse system aspects.The model is refined as the design process progresses, get-ting more accurate in mirroring the behavior of the physicalsystem. When software prototypes become available, theycan be integrated and tested in combination with the simu-lated system parts. With this simulation platform, the erroreffect simulation can easily evaluate a huge set of systemalternatives, allowing a quantitative safety assessment. Still,synchronization poses an extreme overhead in SystemC andapproaches are required that increase simulation performanceof the simulation of concurrent processes, e.g., by temporaldecoupling or by raising the abstraction level with the guar-antee that the error effect is simulated correctly in terms offunctionality and time.

Raising the abstraction level creates another major chal-lenge, as faults that lead to possible errors are usually lowlevel technology-based effects. In [40], it was shown that errorinjection at high level of abstraction may result in differentresults than injecting errors at the gate level, but simulationat low level is too time consuming, especially if many errorscenarios are to be investigated. Information on the faultmust be propagated to higher levels of abstraction, requiringcross-layer analysis. The purpose of such analysis is to derivethe fault models for the high-level stressors, which shouldideally capture the effects resulting from low-level faults tothe full extent, or at least provide adequate approximations.Furthermore, efficient error effect simulation requires avoid-ing the error-prone process of manual error injection. Thus,these fault models should be available in a formalized formto enable automatic configuration/generation of the errorinjectors inside the UVM-based testbenches as mentionedin Section 3.3. Considering that errors affect various differ-ent domains, e.g., digital hardware, analog hardware andsoftware, deriving a unified formalism is a very challeng-ing task. One of the promising directions is to adopt UMLand the Object Constraint Language (OCL) for error model

specification.While approaches such as temporal decoupling reduce the

overhead of single simulation runs, the number of simulationruns has to be optimized as well. Given the huge number ofpossible locations of errors, system states and alternatives ofa complex automotive system, it is not feasible to simulate allthe possibilities in the available time. Therefore, intelligentcoverage models are required to measure the completenessof the error effect simulation as shown in Fig. 3. It has tobe investigated how coverage models can be systematicallyderived from safety requirements and Mission Profiles. Then,the strategy of error injection and stimuli generation shouldbe geared towards coverage closure. Maximal reuse of exist-ing functional verification tests should be one of the maintargets in the design process. This is very challenging inautomotive systems design. Stress tests need to be executedfor highly protected systems, which feature a range of failsafemeasures and redundancy at several levels to assure drivingsafety. Standard Monte-Carlo techniques may fail to identifythe critical error effects leading to system failure becausefailure probabilities are extremely low. Thus, the probabilityof exposing a weakness of the system not covered by thesafety and protection measures in a ”lucky guess scenario” isextremely low, or may require a vast amount of simulation.Thus, a systematic approach is required that stresses the sys-tem at its possible weak spots to find error scenarios that leadto safety-critical failures. Identifying the weak spots has tobe conducted by analysis of error propagation, error masking,and error recovery by protection mechanisms in all systemdomains (e.g. digital HW, analog HW, SW) with respect tothe system structure. For errors that are hard to propagate,formal approaches such as symbolic execution [41, 42] mightbe necessary to generate stimuli to bypass the protectionmechanisms.

4. CONCLUSION AND OUTLOOKProgress in the application of Virtual Prototypes for safety

assurance in the automotive and other domains with safety-critical applications is mainly driven by the needs of industry.In combination with other standards, such as SystemC andUVM, VPs can provide an ideal platform for error effectsimulation, for the identification of weak spots of a system inan early design phase. In this paper, the state of the art andmost recent research challenges, which need to be resolvedto efficiently apply this methodology on automotive systems,have been outlined. In the upcoming years, we expect thatthe research community wil invest a huge amount of effortto overcome these challenges. Brought to the industriallevel, this methodology will certainly increase the safety ofever increasingly complex automotive systems. This level ofsecurity is certainly required to address future verificationchallenges for automotive systems designs such as thoserequired for autonomous driving.

ACKNOWLEDGMENTThis work has been partially supported by the German Fed-eral Ministry of Education and Research (BMBF) in theproject EffektiV under grant 01IS13022.

5. REFERENCES[1] AUtomotive Open System ARchitecture (AUTOSAR)

Development Partnership Website. http://www.autosar.org/.[2] Georg Georgakos, Ulf Schlichtmann, Reinhard Schneider, and

Samarjit Chakraborty. Reliability challenges for electricvehicles: from devices to architecture and systems software. In50th Annual Design Automation Conference (DAC), page 98,2013.

[3] C. Pinello, L.P. Carloni, and A.L. Sangiovanni-Vincentelli.Fault-tolerant deployment of embedded software forcost-sensitive real-time feedback-control applications. InDesign, Automation and Test in Europe Conference (DATE),pages 1164–1169 Vol.2, 2004.

[4] Malcolm Wallace. Modular architectural representation andanalysis of fault propagation and transformation. Electron.Notes Theor. Comput. Sci., 141(3):53–71, December 2005.

[5] Xiaocheng Ge, Richard F. Paige, and John A. Mcdermid.Probabilistic failure propagation and transformation analysis.In 28th International Conference on Computer Safety,Reliability, and Security (SAFECOMP), pages 215–228, 2009.

[6] Bernhard Kaiser, Peter Liggesmeyer, and Oliver Mackel. A newcomponent concept for fault trees. In 8th Australian Workshopon Safety Critical Systems and Software (SCS) - Volume 33,pages 37–46, 2003.

[7] J. Zimmermann, S. Stattelmann, A. Viehl, O. Bringmann, andW. Rosenstiel. Model-driven virtual prototyping for real-timesimulation of distributed embedded systems. In 7th IEEEInternational Symposium on Industrial Embedded Systems(SIES), pages 201–210, 2012.

[8] S. Reiter, M. Pressler, A. Viehl, O. Bringmann, andW. Rosenstiel. Reliability assessment of safety-relevantautomotive systems in a model-based design flow. In 18th Asiaand South Pacific Design Automation Conference(ASP-DAC), pages 417–422, 2013.

[9] Ningfang Song, Jiaomei Qin, Xiong Pan, and Yan Deng. Faultinjection methodology and tools. In International Conferenceon Electronics and Optoelectronics (ICEOE), volume 1, pagesV1–47–V1–50, 2011.

[10] Giacinto P. Saggese, Nicholas J. Wang, Zbigniew T.Kalbarczyk, Sanjay J. Patel, and Ravishankar K. Iyer. Anexperimental study of soft errors in microprocessors. IEEEMicro, 25(6):30–39, November 2005.

[11] M. Rebaudengo, M. Sonza Reorda, and M. Violante. Anaccurate analysis of the effects of soft errors in the instructionand data caches of a pipelined microprocessor. In Conferenceon Design, Automation and Test in Europe (DATE), 2003.

[12] M.L. Li, P. Ramachandran, U.R. Karpuzcu, S. Hari, and S.V.Adve. Accurate microarchitecture-level fault modeling forstudying hardware faults. In IEEE 15th InternationalSymposium on High Performance Computer Architecture(HPCA), pages 105–116, 2009.

[13] D. May and W. Stechele. An fpga-based probability-aware faultsimulator. In International Conference on EmbeddedComputer Systems (SAMOS), pages 302–309, 2012.

[14] T. Schweizer, D. Peterson, J.M. Kuhn, T. Kuhn, andW. Rosenstiel. A fast and accurate fpga-based fault injectionsystem. In IEEE 21st Annual International Symposium onField-Programmable Custom Computing Machines (FCCM),pages 236–236, 2013.

[15] D. Peterson, O. Bringmann, T Schweizer, and W. Rosenstiel.Stml: Bridging the gap between fpga design and hdl circuitdescription. In International Conference onField-Programmable Technology (ICFPT), 2013.

[16] Lukai Cai and Daniel Gajski. Transaction level modeling: Anoverview. In 1st IEEE/ACM/IFIP International Conferenceon Hardware/Software Codesign and System Synthesisv(CODES+ISSS), pages 19–24, 2003.

[17] Accellera Systems Initiative. Universal VerificationMethodology (UVM), May 2012.

[18] IEEE Computer Society. IEEE 1666-2011 Standard SystemCLanguage Reference Manual, 2011.

[19] R. A. DeMillo, R. J. Lipton, and F. G. Sayward. Hints on testdata selection: Help for the practicing programmer. Computer,11:34–41, April 1978.

[20] Richard A. DeMillo and A. Jefferson Offutt. Constraint-basedautomatic test data generation. IEEE Transactions onSoftware Engineering, 17(9):900–910, September 1991.

[21] Yue Jia and Mark Harman. An analysis and survey of thedevelopment of mutation testing. IEEE Transactions onSoftware Engineering, 2010.

[22] Markus Becker, Daniel Baldin, Christoph Kuznik, Mabel MaryJoy, Tao Xie, and Wolfgang Mueller. Xemu: An efficient qemubased binary mutation testing framework for embeddedsoftware. In Tenth ACM International Conference onEmbedded Software (EMSOFT), pages 33–42, 2012.

[23] C. Berger R. Rana, M. Staron and F. Torner J. Hansson,M. Nilsson. Increasing efficiency of iso 26262 verification andvalidation by combining fault injection and mutation testingwith model based development. 8th International JointConference on Software Technologies (ICSOFT-EA), July2013.

[24] Synopsys. CERTITUDE Functional Qualification System.[25] V. Guarnieri, N. Bombieri, G. Pravadelli, F. Fummi,

H. Hantson, J. Raik, M. Jenihhin, and R. Ubar. Mutationanalysis for systemc designs at tlm. In 12th Latin AmericanTest Workshop (LATW), pages 1 –6, 2011.

[26] Tao Xie, W. Mueller, and F. Letombe. IP-XACT based systemlevel mutation testing. In IEEE International High LevelDesign Validation and Test Workshop (HLDVT), nov. 2011.

[27] Thorsten Piper, Stefan Winter, Paul Manns, and Neeraj Suri.Instrumenting autosar for dependability assessment: Aguidance framework. 43rd Annual IEEE/IFIP InternationalConference on Dependable Systems and Networks (DSN),0:1–12, 2012.

[28] Nicola Bombieri, Franco Fummi, and Valerio Guarnieri.Accelerating RTL Fault Simulation through RTL-to-TLMAbstraction. In European Test Symposium, pages 117–122,2011.

[29] Nicola Bombieri, Franco Fummi, and Valerio Guarnieri.Fast-gp: An rtl functional verification framework based on faultsimulation on gp-gpus. In Conference on Design, Automationand Test in Europe (DATE), pages 562–565, 2012.

[30] Markus Becker, Christoph Kuznik, Mabel Mary Joy, Tao Xie,and Wolfgang Mueller. Binary mutation testing throughdynamic translation. In 42nd Annual IEEE/IFIPInternational Conference on Dependable Systems andNetworks (DSN), pages 1–12, 2012.

[31] T. Nirmaier, A. Burger, M. Harrant, A. Viehl, O. Bringmann,W. Rosenstiel, and G. Pelz. Mission profile aware robustnessassessment of automotive power devices. In Conference onDesign, Automation and Test in Europe (DATE), 2014.

[32] ZVEI. Handbook for Robustness Validation of AutomotiveElectrical/Electronic Modules. ZVEI - ZentralverbandElektrotechnik- und Elektronikindustrie e. V., June 2013.

[33] Y. Li, M.-M. Louerat, F. Pecheux, R. Iskander, P. Cuenot, M.Barnasconi, T. Vortler, K. Einwich. Virtual Prototyping,Verification and Validation Framework for Automotive UsingSystemC, SystemC-AMS and SystemC-UVM. Embedded RealTime Software and Systems (ERTS2), 2014.

[34] M. Barnasconi, F. Pecheux, T. Vortler, K. Einwich. Advancingsystem-level verification using UVM in SystemC. Design andVerification Conference (DVCon), 2014.

[35] Marcio F.S. Oliveira, Christoph Kuznik, Hoang M. Le, DanielGroße, Finn Haedicke, Wolfgang Mueller, Rolf Drechsler,Wolfgang Ecker, and Volkan Esen. The System VerificationMethodology for Advanced TLM Verification. In InternationalConference on Hardware/Software Codesign and SystemSynthesis (CODES+ISSS), 2012.

[36] Marcio F. S. Oliveira, Christoph Kuznik, Wolfgang Mueller,Wolfgang Ecker, and Volkan Esen. A SystemC Library forAdvanced TLM Verification. In Design and VerificationConference (DVCON), 2012.

[37] Yao Li, Ramy Iskander, Farakh Javid, and Marie-MinerveLouerat. A Design and Verification Methodology forMixed-Signal Systems Using SystemC-AMS. In Jan Haase,editor, Models, Methods, and Tools for Complex Chip Design,volume 265 of Lecture Notes in Electrical Engineering, pages89–108. Springer International Publishing, 2014.

[38] Ankur Sharma, Joseph Sloan, Lucas F Wanner, Salma HElmalaki, Mani B Srivastava, and Puneet Gupta. Towardsanalyzing and improving robustness of software applications tointermittent and permanent faults in hardware. In IEEE 31stInternational Conference on Computer Design (ICCD), pages435–438, 2013.

[39] Lucas Wanner, Salma Elmalaki, Liangzhen Lai, Puneet Gupta,and Mani Srivastava. Varemu: An emulation testbed forvariability-aware software. In IEEE International Conferenceon Hardware/Software Codesign and System Synthesis(CODES+ ISSS), pages 1–10, 2013.

[40] Hyungmin Cho, Shahrzad Mirkhani, Chen-Yong Cher, Jacob AAbraham, and Subhasish Mitra. Quantitative evaluation of softerror injection techniques for robust system design. In 50thAnnual Design Automation Conference (DAC), page 101,2013.

[41] Cristian Cadar, Daniel Dunbar, and Dawson Engler. Klee:Unassisted and automatic generation of high-coverage tests forcomplex systems programs. In 8th USENIX Conference onOperating Systems Design and Implementation (OSDI), pages209–224, 2008.

[42] H. M. Le, D. Große, V. Herdt, and R. Drechsler. VerifyingSystemC using an intermediate verification language andsymbolic simulation. In Design Automation Conference(DAC), page 116, 2013.