44
Industrial-Grade DevOps Balancing Speed with Extreme Quality Frank Buschmann w/ Joachim Fröhlich, Lars Gelbke, Fei Li, Marc Zeller, Peter Zimmerer Siemens AG, Corporate Technology

Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

  • Upload
    others

  • View
    2

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Industrial-Grade DevOpsBalancing Speed with Extreme Quality

Frank Buschmann w/ Joachim Fröhlich, Lars Gelbke, Fei Li, Marc Zeller, Peter ZimmererSiemens AG, Corporate Technology

Page 2: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 2 Frank Buschmann

DevOps is state-of-practice in enterprise IT software engineeringLeaders in DevOps experience measurable benefits

Ñ Market attractiveproduct quality1

§ 18% increase in revenue§ 19% improvement in

product quality and performance

Ñ Short time-to-market1

§ 21% reduction of time to marketfor product innovations

Ñ Minimal cost ofdowntime2

§ 60% decrease of deploymentdowntime costs per year

Ñ IT performance2

§ On demand product deployment§ 3x lower deployment failure rate,

24x faster recovery from failure§ >2000x shorter lead times for

product changes

1) CA Technologies: DevOps: Enterprise Organizations‘ Newest Best Practice, July 2015 2) Puppet 2016 State of DevOps Report

Page 3: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 3 Frank Buschmann

The Digitalization of the industry requires an adoption of DevOpsIt is the “how” to achieving business agility

Source: Siemens AG

Page 4: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 4 Frank Buschmann

The Digitalization of the industry requires an adoption of DevOpsIt is the “how” to achieving business agility

Digitalization @ SiemensDigital Services

Vertical Software

Digitally EnhancedElectrification andAutomation

110010101001110110 8 % growth p.a.

Source: Siemens AG

Page 5: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 5 Frank Buschmann

Yet the adoption of DevOps in industry is challengingStringent industry-specific constraints and requirements apply

Source: Siemens AG

Page 6: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 6 Frank Buschmann

Yet the adoption of DevOps in industry is challengingStringent industry-specific constraints and requirements apply

Multi-level DevEcosystems

Ops EnvironmentComplexity

SolutionBusinesses

RegulatedDomains

SystemsEngineering

IT Security OperationalQuality

Source: Siemens AG

Page 7: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 7 Frank Buschmann

Yet the adoption of DevOps in industry is challengingStringent industry-specific constraints and requirements apply

Multi-level DevEcosystems

Ops EnvironmentComplexity

SolutionBusinesses

RegulatedDomains

SystemsEngineering

IT Security OperationalQuality

Source: Siemens AG

Page 8: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 8 Frank Buschmann

Ops environment complexity complicates continuous deploymentOperations environments are often polyglot, critical infrastructures

ÑMultiple IT, OT and field networksÑRestricted network accessÑMulti-platform execution environmentsÑ Federated cloud to edge deploymentsÑ Flexible deployment demands

Source: Siemens AG

ÑWhat delivery pipeline architectures enable continuousdeployment into industry operations environments?

ÑWhat technologies support flexible deployment offeatures to multi-platform execution environments?

Source: Siemens AG

Page 9: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 9 Frank Buschmann

Release BinRepository

Depl

oym

ent

VPN

Execution Stack

Depl

oym

ent

Asse

tMgm

t.

Execution Stack

Push

Pull

Depl

oym

ent

Execution Stack

VPN

Execution Stack

BinRepository

BinRepository

DEV OPS

Delivery pipelines extend to the fieldCentrally managed, federated on-premise deployment infrastructure

ÑDedicated (restricted) DevOps managementnetworks per (restricted) operations network

ÑSecure connection of DevOps managementnetworks with the delivery pipeline via VPN

ÑDedicated on-premise deployment executionstacks per DevOpsmanagement network

ÑManagement of on-premise deploymentexecution stacks as deployable services

ÑProvisioning of on-premise deploymentexecution stacks using containerization

Page 10: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 10 Frank Buschmann

Diverse deployment scenariosNo two operations networks are likely to be the same

ÑOn-premise deployment service directlyaccesses the main repository via Internet

ÑOn-premise deployment service runs onedge device and updates it via feature pull

BinRepository

Depl

oym

ent

VPN

Execution Stack

VPN

Direct Connection (for open Ops networks)

ÑRepository mirror with the latest feature versionsÑAsset management to orchestrate devices for updateÑDeployment service updates devices via feature push

Push Update

ÑRepository mirror with the latest product featuresÑDeployment service runs on edge device and

updates it via feature pull

PullUpdate Pull

Depl

oym

ent

Execution Stack

Execution Stack

BinRepository

Depl

oym

ent

Asse

tMgm

t.

Execution Stack

Push

BinRepository

Page 11: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 11 Frank Buschmann

(WIN)

(ARM)

ReleaseDeployto Test

Build(x64)

VersionCtrl.

BinRepository

Delivery Pipeline

Delivery pipelines support multi-platform build, test, and releaseParallelization of build, integration and test in virtual, emulated environment

ÑVersioning and release management strategies for products with flexible deployment demandsÑParallel build stage with dedicated build infrastructure per product execution platformÑ Features w/ deployment flexibility packed for all execution environments on which they can runÑCommon integration and test stage based on virtualization and emulationÑRepository with images for all deployment platforms available to the deployment service

Page 12: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 12 Frank Buschmann

Ñ Leveraging modern orchestration and provision technologies simplify the creation andmanagement of industrial DevOps environmentsÑ Most state-of-the-art technology need (some)

adaption to fulfill industry-specific needs

Ñ Ops environment complexity enforces tailored solutionsÑ Single enterprise standard does not workÑ Orchestration requires proper toolingÑ Secure networking required to bridge dev and ops

Ñ “Environment as asset” is a key mindset change!Ñ Suggests role and practice extensions in

existing scaled agile frameworks, e.g., SAFe

Environment orchestration in industrial contextAlways different, always specific, always something new

Page 13: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 13 Frank Buschmann

ÑWhat enhancements to agile practices help addressregulatory compliance in development fast enough?

ÑWhat methods and technologies support continuousdelivery in the face of regulations and certification?

Regulated domains challenge agility and cycle timeRegulatory compliance like functional safety is often mandatory

ÑAgility and continuous delivery create highvolatility in ensuring regulatory compliance

ÑCurrent practice is time and effort intenseÑCross-cuts multiple engineering disciplinesÑDevOps environments are subject of certification

Source: Siemens AG

Page 14: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 14 Frank Buschmann

Definition Design Realization Test andVerification Deployment OperationsRelease

Feedback

Safety assurance lifecycle in a nutshellStrict compliance to (domain-specific) norms is mandatory

Hazard andRisk Analysis

SafetyRequirements Safety Tests

SafetyAnalysis Safety Case

ÑRegulatory compliancerequires dedicatedactivities in addition to“normal” development

Certification

Page 15: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 15 Frank Buschmann

Methods and technologies for safety assurance are emergingResearch driven by funded projects on EU and national level*

ÑComponent Fault Trees (CFT)ÑSafety Concept Trees (SCT)ÑConditional Safety

Certificates (ConSerts)ÑComponent-based Failure

Mode and Effect Analysis(FMEAexpress)

ÑDigital DependabilityIdentities (DDI)

*) EU H2020 project DEIS (Dependability Engineering Innovation for Cyber Physical Systems), national BMBF project CrESt (Collaborative Embedded Systems)

Definition Design Realization Test andVerification Deployment OperationsRelease

Feedback

Hazard andRisk Analysis

SafetyRequirements Safety Tests

SafetyAnalysis Safety Case

Certification

«Goal»G1

Side collis ion wit h c ross ing vehic lesavoided

«Goal»G3

.. .

«G oal»G2

{Ego Vehic le} does not cros s a r edtraff ic light

«Strategy»St1

A rgument over criticals cenarios of traffic lightphas es

«Context »C1

Cr itic al sc enarios of tr af fic lightphas es are red to green andgr een to red

«G oal»G5

{Ego Vehicle} does not cros s a redtr af fic light during sc enar io "greento red"

«Goal»G4

{Ego V ehicle} does not c ross ared t raffic light during sc enario"red to gr een"

«Context »C2

Sc enario red to green:[x ]

«Context»C3

Sc enario green to red:[x ]

«Context »C0

Safety goal: {A void overrunninga red traffic light, when TL As ystem has vehic le c ont rol}

TL Shif t PlanCam eraim age

«block»TLA Syst em::Ego V ehicle:: TLA Fu nction

TL Shif t PlanCam eraim age

Cam era imagedoes not containTL state change

TL Cur rent ShiftPlan wrong

V alue

«CFT»TLA Function

Cam era imagedoes not containTL state change

TL Cur rent ShiftPlan wrong

V alue

Came ra im agedoes n ot cont ainTL st ate chan ge

No TL Stat e changedetected (Camer a)

«CFT Instance»(Camer a-based TL State

Dete ction)

Came ra im agedoes n ot cont ainTL st ate chan ge

No TL Stat e changedetected (Camer a)

Nominaldeceleration

T oo_Low

Acce ler ationvector Too_High

«CFT I nstance»( Longitu dinal Contr ol)

Nominaldeceleration

T oo_Low

Acce ler ationvector Too_High

A cceleration vectorToo_High

D istance to TLToo_High TL Curr ent Shift

Plan wrongV alue

«CFT Instance»( Opt im al Tr aje ctor y C omputation)

A cceleration vectorToo_High

D istance to TLToo_High TL Curr ent Shift

Plan wrongV alueDistance to TL

Too_High

« CFT I nst ance»(Se lf -localization)

Distance to TLToo_High

TL Curr entShift Plan

wr ongValue

Em ergencydecelerat io n

Too_Low

No TL StateCha nge de tect ed

(Camera)

D istance to TLT oo_High

«CFT Instance»(Side Collis ion Avoidance)

TL Curr entShift Plan

wr ongValue

Em ergencydecelerat io n

Too_Low

No TL StateCha nge de tect ed

(Camera)

D istance to TLT oo_High

Nominalaccel. _Too

High

Emergencydecelera tion_Too

Low

Overrun a red tr affic light, when TLAsystem has vehicle control

«CFT Instance»( Accelerat ion Coor din ator)

Nominalaccel. _Too

High

Emergencydecelera tion_Too

Low

Overrun a red tr affic light, when TLAsystem has vehicle control

DDI

Page 16: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 16 Frank Buschmann

ÑAgile practices are availablebut rarely practiced

ÑThough tool supported, activitiesare mostly manual

ÑCompilation of safety case istypically tedious anderror-prone paperwork

ÑCertification by authoritiesnot integrated withdevelopment

Non-agile culture and limited automation are core challengesNegative impact on agility and cycle time

Definition Design Realization Test andVerification Deployment OperationsRelease

Feedback

Hazard andRisk Analysis

SafetyRequirements Safety Tests

SafetyAnalysis Safety Case

Certification

Hazard andRisk Analysis

SafetyRequirements Safety Tests

SafetyAnalysis Safety Case

Certification

ManualManual

Paperwork

LimitedAutomation

SeparatedDev

Authorities

Manual

Page 17: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 17 Frank Buschmann

Automation of delivery process is a viable first stepImproves traceability and reduces pre-certification efforts

ÑAutomatic generation of safety tests from component fault trees (CFTs)Ñ Integration of safety tests with test automation in delivery pipelineÑAdaptive test and simulation environments with hardware in the loop for test executionÑAutomatic generation of safety cases using digital dependability identifiers (DDIs)

Hazard andRisk Analysis

SafetyRequirements Safety Tests

SafetyAnalysis Safety Case

Certification

Automation

Automation

Model-Based

Automation

Digital110010101001110110

Page 18: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 18 Frank Buschmann

Component Fault Trees (CFTs)*Extend classic fault trees with a component concept

Extension of classic fault trees with acomponent conceptÑFocus of are failure modes of an

encapsulated system componentÑFailures visible at the inport / outport

of a component are modeled usingInput / Output Failure Modes

Divide-and-conquer strategy for systemsÑModular, hierarchical decomposition of

system fault treesÑPotential reuse of CFTs in other systems

Legend:

*) Höfig, K., Joanni, A., Zeller, M., Montrone, F., Rothfelder, M., Amarnath, R., Munk, P., Nordmann, A. (2018). Model-based Reliability and Safety: Reducing thecomplexity of safety analyses using component fault trees, Proceedings of the 2018 Annual Reliability and Maintainability Symposium (RAMS)

Page 19: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 19 Frank Buschmann

Automated Test Case Generation from CFTsDerive test cases from the failure behavior specification (safety tests)

ÑShow that in case of failure the implemented behavior is compliant to the specified behaviorÑUse of test results to verify if safety mechanisms are correctly implementedÑProvide inputs and expected outputs

for fault injection tests

Testinputs

Expected testoutputs

a Ù b Ù c f

*) Zeller, M., Höfig, K. (2015). CONFeTTI – Component Fault Tree-Based Testing, Proceedings of the 25th European Safety and Reliability Conference (ESREL

Page 20: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 20 Frank Buschmann

Digital Dependability Identity (DDI)*Ensures interoperability of safety information across organizations

ÑContains all information that uniquely describes the safety characteristics of a systemcomponent and its interaction

ÑSupports encapsulation of model-based descriptions of a system component’s safety artifactsÑProvides standardized, machine-readable interfaces to access the safety artifactsÑComponent DDIs can be

connected to describesafety characteristicsof an entire system

*) Schneider, D.; Trapp, M.; Papadopoulos, Y.; Armengaud, E.; Zeller, M. Höfig, K. WAP: Digital Dependability Identities 2015 IEEE International Symposium onSoftware Reliability Engineering (ISSRE), 2015, 324-329

Page 21: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 21 Frank Buschmann

Composition of safety cases from DDIsDefinition and execution of modular, hierarchical safety cases

Ñ Continuous and defined extension along the system’s lifecycleÑ Basis for automatic execution of formal safety testsÑ Support for reliable and traceable certification by authorities

Integration atdesign time

Integration atconfiguration time

Integration atruntime

DDI

Safety-relevantartifacts

Page 22: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 22 Frank Buschmann

Ñ Component Fault Trees (CFT), Conditional Safety Certificates (ConSerts) and DigitalDependability Identities (DDI) are promising technologies to automate the delivery processÑ Design and implementation of adaptive test

and simulation environments is a challenge

Ñ Automated delivery process is important, but notsufficient to practice DevOps in regulated domains

Ñ Additional methods and technologies required,e.g., for rapid design, change / impact analysis

Ñ Agile practices available, but practices do not automatically create culture!

Ñ Certification of tools is a challenge by itself, especially if DevOps environments evolve

Regulated domains challenge agility and cycle timeFirst steps towards DevOps possible, a long and windy road is still ahead

Source: Siemens AG

Page 23: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 23 Frank Buschmann

Source: Siemens AG

Operational quality demands side-effect-free deploymentCorrect system behavior, availability and performance must not degrade

ÑMany systems run 24/7, some with 99.999%availability – especially in Operations and Control

ÑSystem correctness and (real-time) performancemust be guaranteed when needed

ÑActual configuration may differ from testconfiguration, e.g., hardware change,exact position of machine

ÑWhat product designs enable continuous deploymentin run and test-in-production without side effects?

ÑWhat runtime architectures ensure a side-effect-freedeployment of features in run?

Page 24: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 24 Frank Buschmann

Microservice

SensorProxy

Input

ControlLogic

ActuatorProxy

Output

Test Scripts and Data

Driv

er

Driv

er

1..n 1..n

Product features must be prepared for continuous deploymentDesign for independence

ÑRealize product features as independent, self-contained MicroservicesÑDefine and block dedicated time and resource budgets for testing in production

Page 25: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 25 Frank Buschmann

Microservice

SensorProxy

Input

ControlLogic

ActuatorProxy

Output

TestProbe

TestController

TestInterpreter

Test Scripts and Data

Driv

er

Driv

er

1..n 1..n

Product features must be prepared for continuous deploymentDesign for testability

Ñ Integrate a permanent TEST PROBE* to dynamically configure and execute tests in production

*) F. Li, J. Fröhlich, D. Schall, M. Lachenmayr, C. Stückjürgen, S. Meixner, F. Buschmann: Microservice Patterns for Industrial Edge Applications, EuroPLoP 2018

Page 26: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 26 Frank Buschmann

Microservice

SensorProxy

Input

ControlLogic

ActuatorProxy

Output

TestProbe

TestController

TestInterpreter

Test Scripts and Data

AuthorityDetermination

PrimaryActivation

Driv

er

Driv

er

1..n 1..n

Product features must be prepared for continuous deploymentDesign for availability

ÑUse AUTHORITY DETERMINATION* paired with PRIMARY ACTIVATION* for seamless version switchand fall back in case of deployment failure

*) Publication in preparation

Page 27: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 27 Frank Buschmann

Microservice

SensorProxy

Input

ControlLogic

ActuatorProxy

Output

TestProbe

TestController

TestInterpreter

Test Scripts and Data

AuthorityDetermination

PrimaryActivation

Driv

er

Driv

er

Connector

1..n 1..n

Product features must be prepared for continuous deploymentDesign for availability

Ñ Integrate CONNECTORS* to separate domain logic and infrastructure domains from one another

*) F. Li, J. Fröhlich, D. Schall, M. Lachenmayr, C. Stückjürgen, S. Meixner, F. Buschmann: Microservice Patterns for Industrial Edge Applications, EuroPLoP 2018

Page 28: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 28 Frank Buschmann

Product features must be prepared for continuous deploymentDesign for independence, testability and availability

ÑRealize product features as independent,self-contained Microservices

ÑDefine and block dedicated time and resourcebudgets for testing

Ñ Integrate a permanent TEST PROBE to dynamicallyconfigure and execute tests in production

ÑUse AUTHORITY DETERMINATION paired withPRIMARY ACTIVATION for seamless version switchand fall back in case of deployment failure

Ñ Integrate CONNECTORS to separate domain logicand infrastructure domains from one another

Ensure all timing and resourcerequirements are met!

Microservice

SensorProxy

Input

ControlLogic

ActuatorProxy

Output

TestProbe

TestController

TestInterpreter

Test Scripts and Data

AuthorityDetermination

PrimaryActivation

Driv

er

Driv

er

Connector

1..n 1..n

Page 29: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 29 Frank Buschmann

Microservice

SensorProxy

Input

ControlLogic

ActuatorProxy

Output

TestProbe

TestController

TestInterpreter

Test Scripts and Data

AuthorityDetermination

PrimaryActivation

Driv

er

Driv

er

Connector

1..n 1..n

Product features must be prepared for continuous deploymentDesign for independence, testability and availability

ÑDesign based on “Microservice Patternsfor Industrial Edge Applications”

ÑPaper accepted for EuroPLoP 2018*ÑPatterns integrate well with existing

Microservice pattern space, e.g. Microservice.io

How can we enable field tests of an industrial microservice to access internalprocess data, possibly distributed across several, internal components,without impairing the real-time behavior of the microservice under test?

Build a TEST PROBE permanently into each microservice. Encapsulatetestable process data. Make the TEST PROBE programmable, so that testprograms can exactly control read and write accesses to testable processdata. Let microservices treat TEST PROBES as ordinary components withreserved and limited system resources, so that test programs cannotaccidentally change the microservice behavior

*) F. Li, J. Fröhlich, D. Schall, M. Lachenmayr, C. Stückjürgen, S. Meixner, F. Buschmann: Microservice Patterns for Industrial Edge Applications, EuroPLoP 2018

Page 30: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 30 Frank Buschmann

ÑDeploy new microservice version, connect it with the availability infrastructure of the currentversion, let it receive input but “mute” is output

ÑExecute pre-activation tests on the new microservice version via its TEST PROBE

Deployment architecture must support zero downtimeDesign for test in production

Microservice V1.0

Free resources

Coordination

Robot

Robotcontroller

InputOutput

Microservice V1.0

Coordination

Robot

Robotcontroller

InputOutput

……………… Test

Microservice V2.0

Ensure the device has dedicated resourcebudgets for continuous deployment

Page 31: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 31 Frank Buschmann

ÑNotify the AUTHORITY DETERMINATION infrastructure to determine the new primaryÑ Let the PRIMARY ACTIVATION infrastructure activate the new microservice version when

seamless switch-over is possible

Deployment architecture must support zero downtimeDesign for seamless activation

Microservice V1.0

Coordination

Robot

Robotcontroller

InputOutput

……………… Test

Microservice V2.0

Microservice V1.0

Coordination

Robot

Robotcontroller

InputOutput

……………… Test

Microservice V2.0

Page 32: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 32 Frank Buschmann

Free resources

Coordination

Robot

Robotcontroller

InputOutput

Microservice V2.0

………………

ÑContinuously test the new microservice version in production via its TEST PROBE

Ñ Fall back to old microservice version in case of problems, otherwise uninstall it

Deployment architecture must support zero downtimeDesign for fail-over

Microservice V1.0

Coordination

Robot

Robotcontroller

InputOutput

……………… Test

Microservice V2.0

Page 33: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 33 Frank Buschmann

ÑDeploy new microservice version to device,let it receive input but “mute” is output

ÑExecute pre-activation tests on the newmicroservice version via its TEST PROBE

ÑNotify the AUTHORITY DETERMINATIONinfrastructure to determine the new primary

Ñ Let the PRIMARY ACTIVATION infrastructureactivate the new microservice version whenseamless switch-over is possible

ÑContinuously test the new microserviceversion in production via its TEST PROBE

Ñ Fall back to old microservice version in caseof problems, otherwise uninstall it

Deployment architecture must support zero downtimeDesign for test in production, seamless activation and fail-over

Ensure the device has dedicated resourcebudgets for continuous deployment

Microservice V1.0

Free resources

Coordination

Robot

Robotcontroller

InputOutput

Microservice V1.0

Coordination

Robot

Robotcontroller

InputOutput

……………… Test

Microservice V2.0

Microservice V1.0

Coordination

Robot

Robotcontroller

InputOutput

……………… Test

Microservice V2.0

Free resources

Coordination

Robot

Robotcontroller

InputOutput

Microservice V2.0

………………

Page 34: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 34 Frank Buschmann

Operational quality demands side-effect-free deploymentProof-of-Concept up and running

DevOpspipeline

Controldashboard

Demonstrator

Gatekeeper

Demonstrator set-up Expected behavior Lab set-up

Lana

T-Rux

3) Deploy in operation4) Monitor5) Testing in operation

ControlSystem

3)

4)

Activated

µService2

µService2a

Feature

Test Activate ActivatedLana

Page 35: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 35 Frank Buschmann

Ñ Design for continuous deployment adopts and enhances patterns for fault-tolerant systems

Ñ Design concepts are feasible for independent microservices that implement aself-contained control programÑ PRIMARY ACTIVATION is a sensitivity point

regarding correctness and availability

Ñ Additional design concepts necessary forÑ Control programs that are federated across

multiple (distributed) microservicesÑ Assurance that deployment is side-effect-free

to other microservices

Ñ Applicability of common test-in-production practices like chaos monkey uncertain

Operational quality demands side-effect-free deploymentKey aspects addressed, several research questions remain

Source: Siemens AG

Page 36: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 36 Frank Buschmann

First adoptions of DevOps in industry target at the industrial IoTCloud to edge analytics applications are in focus

Source: Siemens AG

Siemens Industrial Edge offers users the possibility of executing a range of descriptive, diagnostic,predictive and prescriptive analytical applications. This allows cloud connectivity (data to cloud) to beused in combination with Edge Apps from Siemens, third party providers or end users themselves inan integrated hardware and software ecosystem (Edge App to Device) for automation components.

Page 37: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 37 Frank Buschmann

First adoptions of DevOps in industry target at the industrial IoTKey aspects are development efficiency and delivery automation

ÑDelivery Pipeline and App ManagementÑ Multi-staged pipeline with defined quality gates

Ñ Orchestration of apps federated across cloud and edge

Ñ Provisioning of runtimes for all target ops environments

Ñ App management infrastructure

ÑAdaptive Development WorkspaceÑ Centrally managed, locally adaptable by partners

Ñ Set-up and configuration < 30 min

Ñ Project templates to get developers productive < 1 day

Ñ Connectivity to delivery pipeline and edge devices

Page 38: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 38 Frank Buschmann

DevOps proof of concepts for industrial operations systems in workMOM and SCADA are the target

Manufacturing Operations Management

Supervisory Control and Data Acquisition

Source: Siemens AG

Page 39: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 39 Frank Buschmann

DevOps proof of concepts for industrial operations systems in workDeployment flexibility and operational quality in focus

ÑDelivery pipeline for flexible deploymentÑ Automatic orchestration of federated, distributed

applications: cloud, on-premise, cloud to edge

Ñ Automatic provisioning of runtime images for alltarget ops environments

Ñ Firmware updates

Ñ App and firmware management

ÑOperational quality during deploymentÑ Reliability

Ñ IT Security

Ñ Availability (on-premise and edge)

PublicCloud

cloud nativecontainer

PrivateCloud

cloud nativecontainer

OnPremise

container

Edgecontainer container

Field

DeployBuild &Test

Page 40: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 40 Frank Buschmann

Research on industrial-grade DevOps ongoingTough industry challenges in focus

Source: Siemens AG

Operational Quality

IT Security / Regulated Domains

Continuous Testing ofSoftware-intense Systems

Agile Culture andAutomation

Systems Engineering PLM and ALM Integrationand Automation

Page 41: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 41 Frank Buschmann

Continuous testing of software-intense industrial systemsAdaptive test and simulation environment to balance speed and quality

Enabler for shift-left and shift-right in software intense-systems, e.g., industry automation

ÑModel-based Digital Twin of the software to get immediate and early feedback on its quality inindustrial environments

ÑVirtual integration anddeployment as digital sandbox

ÑX-in-the-loop to step-wisefrontload physical realityinto software development Virtual

components

Adaptive Test- and Simulation EnvironmentEvaluation Emulation Prove

Entire system

Specific systemaspects Virtual

components

HiL / SiL / MiL

Real components Installed livesystem

Virtual system asDigital Twin

Real subsystems

Production data

*) concept plus prototype

ExecutableModels

VirtualIntegration

VirtualCommissioning

RealSystem

Page 42: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 42 Frank Buschmann

Research in industrial-grade DevOps is well foundedCore technologies developed over a longer time period

Höfig, K., Joanni, A., Zeller, M., Montrone, F., Rothfelder, M., Amarnath, R., Munk, P., Nordmann, A. (2018). Model-based Reliability and Safety: Reducing thecomplexity of safety analyses using component fault trees, Proceedings of the 2018 Annual Reliability and Maintainability Symposium (RAMS)

Möhrle, F., Zeller, M., Höfig, K., Rothfelder, M., Liggesmeyer., P. (2016). Automating Compositional Safety Analysis Using a Failure Type Taxonomy for ComponentFault Trees, Proceedings of the 26th European Safety and Reliability Conference (ESREL), pp. 1380-1387

Zeller, M., Höfig, K. (2015). CONFeTTI – Component Fault Tree-Based Testing, Proceedings of the 25th European Safety and Reliability Conference (ESREL), pp4011-4017

Armengaud, E.; Macher, G.; Massoner, A.; Frager, S.; Adler, R.; Schneider, D.; Longo, S.; Melis, M.; Groppo, R.; Villa, F.; O’Leary, P.; Bambury, K.; Finnegan, A.;Zeller, M.; Höfig, K.; Papadopoulos, Y.; Hawkins, R. & Kelly, T. DEIS: Dependability engineering innovation for automotive CPS 21st International Forum onAdvanced Microsystems for Automotive Applications, 2017

Schneider, D.; Trapp, M.; Papadopoulos, Y.; Armengaud, E.; Zeller, M. & Höfig, K. WAP: Digital Dependability Identities 2015 IEEE International Symposium onSoftware Reliability Engineering (ISSRE), 2015, 324-329

Höfig, K.; Zeller, M. & Grunske, L. metaFMEA-A Framework for Reusable FMEAs Model-Based Safety and Assessment, Springer International Publishing, 2014,8822, 110-122

Fröhlich, J., Stückjürgen, C. 2017: Reliable Inspection of an Autonomous System At System Runtime with Built-in Data Probes. ISSRE Workshops, Industry Track,Toulouse, IEEE Computer Society 2017, ISBN

Fröhlich, J., Frtunikj, J., Rothbauer, S., Stückjürgen, C. 2016: Testing Safety Properties of Cyber-Physical Systems with Non-Intrusive Fault Injection – An IndustrialCase Study. SafeCOMP 2016, Trondheim, Springer, LNCS 9923.

Fröhlich, J., and Schwarzinger, M. 2006: Improve Component-Based Programs with Connectors.Joint Modular Language Conference, Oxford, Springer, LNCS 4228

F. Li, J. Fröhlich, D. Schall, M. Lachenmayr, C. Stückjürgen, S. Meixner, F. Buschmann: MicroservicePatterns for Industrial Edge Applications, EuroPLoP 2018

S E L E C T E DP U B L I C A T I O N S

Page 43: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Unrestricted © Siemens AG 201830.05.2018Page 43 Frank Buschmann

Industrial-grade DevOps is key for the Digitalization of IndustryHelps mastering the digital transformation at speed and scale

Defines the “How” of Digitalization

Source: Siemens AG

Industry-specific challengesto master

Successful adoption forIndustrial IoT

Research ongoing forIndustrial Operations and Control

Page 44: Siemens On-stage PowerPoint-Template · Ecosystems Ops Environment Complexity Solution Businesses Regulated Domains Systems Engineering IT Security Operational ... Operations environments

Industrial-Grade DevOpsBalancing Speed with Extreme Quality

Frank Buschmann w/ Joachim Fröhlich, Lars Gelbke, Fei Li, Marc Zeller, Peter ZimmererSiemens AG, Corporate Technology