VORSTELLUNG
Konrad Pfeilsticker Geschäftsführer
• Seit 2013 bei der NovaTec als Performance Engineer
• Projekterfahrung im Automotive & Finance Bereich
• Freie / Open Source Alternativen für APM
• Freie / Open Source Alternativen für Lasttests
• Engagement im inspectIT Open-Source Projekt Mario Mann
Consultant
Mario Mann
@mawtourde
http://www.linkin.com/mario-mann
3
The NovaTec APM Team
Führende unabhängige APM Beratung § Aktuell 15 APM Berater § Kunden: mittelständische bis große Konzerne
(Automobil, Finanzen, Versicherungen)
Unbefangenheit durch Multi-Partner Ansatz § Partnerschaft mit den führenden APM
Software-Anbietern § Zusammenarbeit mit neuen Innovatoren
Fördern Open-Source im APM-Bereich § Eigenes Open-Source APM Werkzeug § Bewertung und Kombination existierender
Open-Source Werkzeuge
4
Ellen DeGeneres‘ Oscars selfie crashes Twitter
[Abbildung: Ellen DeGeneres]
Zusammenfassung § 2.6 Millionen Re-Tweets inerhalb 2 Stunden § 20 Minuten nicht verfügbar
[Abbildung : Twitter.com]
5
AGENDA
• Application Performance Management
• inspectIT – The OpenSource APM solution
• Gathering and Visualizing Timeseries Data
• Application Performance Testing & Diagnosis
• WEB Performance Analysis
6
What is APM about?
APM
Data Collection
Data Storage and
Processing
Data Presentation
Data Interpretation
and Use
7
Application Performance Management
… umfasst den gesamten Software-Lebenszyklus
… deckt alle
Systemebenen ab
… erfordert Zusammenarbeit unterschiedlicher Bereiche
… ist ein Querschnittsthema!
Anforderungen
Entwurf
Entwicklung Test
Betrieb Netzwerk
Anwendung
System
Clients
Entwicklung
Betrieb Business
8
Load Testing
Web Perf. Testing
Performance Modelling
Monitoring in Production Synthetic
Monitoring
Real User Monitoring
Root Cause Diagnosis
Regression Testing
APM
Application Performance Management
Requirements
Design
Development Testing
Operations
Network
App
System
Clients
Development
Operations Business
9
Umfangreiche Funktionalität
Ausgereift
Bei großen Unternehmen im Einsatz
Commercial APM tools
Gartner‘s Magic Quadrant for Application Performance Monitoring Suites (12/2015), Cameron Haight, Will Cappelli, Federico De Silva http://www.dynatrace.com/en/gartner-magic-quadrant-application-performance-monitoring-2015.html
Partner
10
Lizenzkosten
Vendor Lock-In
Typisches Modell: x $ pro Agent à Skalierung?
Weakness of commercial APM tools
Microservices Internet of Things
[Abbildung: http://blog.wso2.com] [Abbildung: Christian Hinkelmann, http://nahverkehrhamburg.de]
Flexibilität Interoperabilität Nachhaltigkeit
Mobile Revolution
[Abbildung: https://uxmag.com]
§ Anpassung an eigene Bedürfnisse
§ Fehlerbehebung
§ Datenquellen
§ Analysewerkzeuge
§ Andere APM Werkzeuge
§ Produkt eingestellt
§ Strategiewechsel des APM-Herstellers
11
Open Source APM tools
Load Testing
Performance Modeling
Web Performance Analysis
System & Resources Monitoring
Real User Monitoring
JRat JMemProf
Low-Level Performance Profiling
Monitoring & Application Deep Dive
12
What is inspectIT?
pen Source
8 |2
Plattf rm
Pareto-Prinzip • Fokus auf Hauptfunktionalität • Erfahrung aus APM Projekten
Plattform-Prinzip • Integration mit Werkzeugen • Erweiterbarkeit
Die Open-Source APM Lösung • Entwicklung seit 2005 • Open Source seit 2015
13
What is APM about?
Estimate Business Impact Monitor Application Health Identify anomalies and performance issues
Prod. Environment Real Users
Request
Dashboards
Monitoring Tool
Problem Diagnosis
14
Gathering and Visualizing Timeseries Data
Time Series Databases
Graphing Tools Data Collectors
Custom Code …
Persist data Query & Visualize
17
The Flaw of Averages
The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty Sam L. Savage, with illustrations by Jeff Danziger – http://flawofaverages.com
Gil Tene – https://www.youtube.com/watch?v=lJ8ydIuPFeU
20
Missing Business Context
CPU utilization over time
How critical is this peak in CPU utilization?
Asking the right questions: à How are the users affected? à Which business transactions are affected? à Are any time-critical batch jobs affected?
11pm 0am 1am 2am 3am 4am 5am 6am
100 %
50 %
0%
23
§ Deep Dive into request execution flow necessary
§ Level of Detail vs. Measurement Overhead à Do not instrument frequently called methods (e.g. Getters, Setters, String class, etc.) à Instrument system and component boundaries (entry and exit points)
Problem Diagnosis
24
What is Performance Testing About?
Identify & Diagnose Performance Problems
Evaluate Application Behaviour under Load
Test Environment Monitoring Tool
Test Scalability of Application
§ load § e-2-e response times § throughput
§ detailed invocations § system resource data § deep dive information
fLoad Generator
Agent
Agent
Agent
Request
Response
25
0 1 2 3
1 2 3 4 5 6 7
Res
pons
e Ti
me
[s]
Execution Nr.
e-2-e response times
load data
detailed measurement
data
System Under Test Load Driver
Regression Tests
http://blog.novatec-gmbh.de/automated-performance-regression-testing/
26
End User Experience?
[Image | http://cdn2.hubspot.net/hub/2224117/file-4141070752-jpg/blog-files/three-e1409846588737.jpg]
Satisfied Tolerating Frustrated
Performance User Actions Users
§ Performance issues § Functional Errors § Network (Local ISPs, Mobile network carriers) § Third party content providers
§ First user action § Last user action § Did the visit convert? § Did the visit bounce?
§ Device § Resolution § Browser Versions § Geolocation
What is Web Performance Analysis about?
27
Real User Testing
Synthetic Monitoring
§ Continuous external testing § Known testing nodes § Different Locations (ISP, Networks) § No baseline traffic required § Competitor Benchmark § Availability testing (incl. 3rd Party)
e.g., every 30 mins from different nodes
Real User Monitoring
§ Real End Users § Real Interaction § Real traffic
External testing with all major § browsers, § operating systems, § mobile devices and real world data
QoS Validation
QoS Expectation QoS Optimization
Anomaly
Testing and Monitoring Web Performance
28
Conclusion
Monitoring & Application Deep Dive
Load Testing
Performance Modeling
Web Performance Analysis
System & Resources Monitoring
Real User Monitoring
JRat JMemProf
Low-Level Performance Profiling
Open-Source tools sufficiently
address APM needs!
There are lots of them for different purposes!
Combine them!