Application Performance Management

with Open Source Tooling

VORSTELLUNG

Konrad Pfeilsticker Geschäftsführer

•  Seit 2013 bei der NovaTec als Performance Engineer

•  Projekterfahrung im Automotive & Finance Bereich

•  Freie / Open Source Alternativen für APM

•  Freie / Open Source Alternativen für Lasttests

•  Engagement im inspectIT Open-Source Projekt Mario Mann

Consultant

Mario Mann

@mawtourde

http://www.linkin.com/mario-mann

mario.mann@novatec-gmbh.de

3

The NovaTec APM Team

Führende unabhängige APM Beratung §  Aktuell 15 APM Berater §  Kunden: mittelständische bis große Konzerne

(Automobil, Finanzen, Versicherungen)

Unbefangenheit durch Multi-Partner Ansatz §  Partnerschaft mit den führenden APM

Software-Anbietern §  Zusammenarbeit mit neuen Innovatoren

Fördern Open-Source im APM-Bereich §  Eigenes Open-Source APM Werkzeug §  Bewertung und Kombination existierender

Open-Source Werkzeuge

4

Ellen DeGeneres‘ Oscars selfie crashes Twitter

[Abbildung: Ellen DeGeneres]

Zusammenfassung §  2.6 Millionen Re-Tweets inerhalb 2 Stunden §  20 Minuten nicht verfügbar

[Abbildung : Twitter.com]

5

AGENDA

•  Application Performance Management

•  inspectIT – The OpenSource APM solution

•  Gathering and Visualizing Timeseries Data

•  Application Performance Testing & Diagnosis

•  WEB Performance Analysis

6

What is APM about?

APM

Data Collection

Data Storage and

Processing

Data Presentation

Data Interpretation

and Use

7

Application Performance Management

… umfasst den gesamten Software-Lebenszyklus

… deckt alle

Systemebenen ab

… erfordert Zusammenarbeit unterschiedlicher Bereiche

… ist ein Querschnittsthema!

Anforderungen

Entwurf

Entwicklung Test

Betrieb Netzwerk

Anwendung

System

Clients

Entwicklung

Betrieb Business

8

Load Testing

Web Perf. Testing

Performance Modelling

Monitoring in Production Synthetic

Monitoring

Real User Monitoring

Root Cause Diagnosis

Regression Testing

APM

Application Performance Management

Requirements

Design

Development Testing

Operations

Network

App

System

Clients

Development

Operations Business

9

Umfangreiche Funktionalität

Ausgereift

Bei großen Unternehmen im Einsatz

Commercial APM tools

Gartner‘s Magic Quadrant for Application Performance Monitoring Suites (12/2015), Cameron Haight, Will Cappelli, Federico De Silva http://www.dynatrace.com/en/gartner-magic-quadrant-application-performance-monitoring-2015.html

Partner

10

Lizenzkosten

Vendor Lock-In

Typisches Modell: x $ pro Agent à Skalierung?

Weakness of commercial APM tools

Microservices Internet of Things

[Abbildung: http://blog.wso2.com] [Abbildung: Christian Hinkelmann, http://nahverkehrhamburg.de]

Flexibilität Interoperabilität Nachhaltigkeit

Mobile Revolution

[Abbildung: https://uxmag.com]

§  Anpassung an eigene Bedürfnisse

§  Fehlerbehebung

§  Datenquellen

§  Analysewerkzeuge

§  Andere APM Werkzeuge

§  Produkt eingestellt

§  Strategiewechsel des APM-Herstellers

11

Open Source APM tools

Load Testing

Performance Modeling

Web Performance Analysis

System & Resources Monitoring

Real User Monitoring

JRat JMemProf

Low-Level Performance Profiling

Monitoring & Application Deep Dive

12

What is inspectIT?

pen Source

8 |2

Plattf rm

Pareto-Prinzip •  Fokus auf Hauptfunktionalität •  Erfahrung aus APM Projekten

Plattform-Prinzip •  Integration mit Werkzeugen •  Erweiterbarkeit

Die Open-Source APM Lösung •  Entwicklung seit 2005 •  Open Source seit 2015

13

What is APM about?

Estimate Business Impact Monitor Application Health Identify anomalies and performance issues

Prod. Environment Real Users

Request

Dashboards

Monitoring Tool

Problem Diagnosis

14

Gathering and Visualizing Timeseries Data

Time Series Databases

Graphing Tools Data Collectors

Custom Code …

Persist data Query & Visualize

15

Dashboards

16

Dashboards

17

The Flaw of Averages

The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty Sam L. Savage, with illustrations by Jeff Danziger – http://flawofaverages.com

Gil Tene – https://www.youtube.com/watch?v=lJ8ydIuPFeU

18

Dashboards

19

Anomaly Detection & Alerting

20

Missing Business Context

CPU utilization over time

How critical is this peak in CPU utilization?

Asking the right questions: à  How are the users affected? à  Which business transactions are affected? à  Are any time-critical batch jobs affected?

11pm 0am 1am 2am 3am 4am 5am 6am

100 %

50 %

0%

21

Defining Business Context

22

Defining Business Context Hawkular BTM

23

§  Deep Dive into request execution flow necessary

§  Level of Detail vs. Measurement Overhead à Do not instrument frequently called methods (e.g. Getters, Setters, String class, etc.) à Instrument system and component boundaries (entry and exit points)

Problem Diagnosis

24

What is Performance Testing About?

Identify & Diagnose Performance Problems

Evaluate Application Behaviour under Load

Test Environment Monitoring Tool

Test Scalability of Application

§  load §  e-2-e response times §  throughput

§  detailed invocations §  system resource data §  deep dive information

fLoad Generator

Agent

Agent

Agent

Request

Response

25

0 1 2 3

1 2 3 4 5 6 7

Res

pons

e Ti

me

[s]

Execution Nr.

e-2-e response times

load data

detailed measurement

data

System Under Test Load Driver

Regression Tests

http://blog.novatec-gmbh.de/automated-performance-regression-testing/

26

End User Experience?

[Image | http://cdn2.hubspot.net/hub/2224117/file-4141070752-jpg/blog-files/three-e1409846588737.jpg]

Satisfied Tolerating Frustrated

Performance User Actions Users

§  Performance issues §  Functional Errors §  Network (Local ISPs, Mobile network carriers) §  Third party content providers

§  First user action §  Last user action §  Did the visit convert? §  Did the visit bounce?

§  Device §  Resolution §  Browser Versions §  Geolocation

What is Web Performance Analysis about?

27

Real User Testing

Synthetic Monitoring

§  Continuous external testing §  Known testing nodes §  Different Locations (ISP, Networks) §  No baseline traffic required §  Competitor Benchmark §  Availability testing (incl. 3rd Party)

e.g., every 30 mins from different nodes

Real User Monitoring

§  Real End Users §  Real Interaction §  Real traffic

External testing with all major §  browsers, §  operating systems, §  mobile devices and real world data

QoS Validation

QoS Expectation QoS Optimization

Anomaly

Testing and Monitoring Web Performance

28

Conclusion

Monitoring & Application Deep Dive

Load Testing

Performance Modeling

Web Performance Analysis

System & Resources Monitoring

Real User Monitoring

JRat JMemProf

Low-Level Performance Profiling

Open-Source tools sufficiently

address APM needs!

There are lots of them for different purposes!

Combine them!

29

Sponsored by...

likwid

30

ASK ME ANYTHING

Visit us at NovaTec exhibition stand 42 + 44

Thank you!