Embed Size (px)
Citation preview
MongoDB, Neo4j & Co. Einführung NoSQL-Datenbanken_
Tobias Trelle@tobiastrelle
Senior IT Consultant @codecentric
Software Architect
Java/JEE/RDBMS background
Committer Spring Data
@tobiastrelle
ttrelle
2
NoSQL
3
4
What‘s My Name Again?
It’s not about using no SQL any more but about thinking beyond SQL for certain problems!
NoSQL
5
Why NoSQL?
NoSQL databases help to solve the 3 problems of
BigData (Gartner, 2009):
• Volume (Menge an Daten)
• Variety (verschiedene Strukturen)
• Velocity (Geschwindigkeit)6
The Trouble with Scaling
Vertical vs.Horizontal
Scalability
7
Vertical Scalability
8
RAMCPU
Storage
Vertical Scalability
9
RAMCPU
Storage
Vertical Scalability
10
RAMCPU
Storage
Vertical Scalability
Size
Cost
„BIG“ DATABASES SCALE
VERTICALLY
BUT AT WHATPRICE?
11
Horizontal Scalability
RAMCPU
Storage
12
Horizontal Scalability
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
13
Horizontal Scalability
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
14
Google Data Center
15
Vertical vs. Horizontal Scalability
NOSQL DATABASES SCALE HORIZONTALLY
NEED AN
EXAMPLE?
MESSAGING SYSTEM OF (POWERED BY )
- 100 NODE CLUSTER- 87 BILLION R/W-OPERATIONS / DAY- 9+ BILLION MESSAGES / DAY- 4 PB COMPRESSED DATA / DAY- 45% GROWTH / MONTH SOURCE: NICOLAS SPIEGELBERG @
BERLIN BUZZWORDS, JUNE 2012
QUITE IMPRESSIVE
NUMBERS, EH?
16
For a distributed computer system it is impossible to simultaneously provide all these three guarantees!
Partition
Tolerance
failure of single nodes doesn‘t effect
the overall system
CAP Theorem a.k.a Brewer’s Theorem (2000)
Consisteny all nodes
see the same data at the same time
Availability
a guarantee that every
request receives a response
PICK 2!
17
Partion
Tolerance
Visual Guide to (No)SQL systems
Consisten
cy
Availability
18
CAP – Netzwerk-Partition
19
ACID vs. BASE
Atomicity
Consistency
Isolation
Durability1983
RDBMS
20
ACID vs. BASE
ACID is a good idea, but it is not an unbreakable law! 2
1
ACID vs. BASE
Basically Available
Soft State
Eventually consistent
2008
NoSQL
22
Overview of NoSQL Categories
11 1 1
1 11 11
11
Column StoresKey-Value StoresK V
K V
K V
Graph Databases
23
Document Stores
KKK
V
V
K
V
V
MongoDB
25
MongoDBhumongous, engl. = gigantisch, riesig
26
MongoDB > FaktenVersion 3.0.5Kategorie: DokumentenorientiertAbfragen: • Query-by-Example + reichhaltige
Operatoren• Aggregationen / MapReduce• Geodaten, Volltextsuche
Indizes: Primär-Index + frei wählbare, auch zusammengesetzte Sekundär-Indizes
Replikation: Master/Slave mit automatischen FailoverSharding: Auto-Sharding anhand wählbarem SchlüsselTX: Atomarität auf einem DatensatzAPI: • Binäres TCP/IP-basiertes Protokoll
• Treiber für viele Sprachen27
MongoDB > FaktenLizenz: APGL (DB), Apache V2 (Treiber)Hersteller: MongoDB Inc. (vormals 10gen)Web: http://www.mongodb.orgGithub: https://github.com/mongodb
28
MongoDB > Dokument
Dokument := Geordnete Menge von Key-Value-PaarenKey := UTF8-StringValue := Einfacher Datentyp | Array | Dokument
Einfacher Datentyp := UTF8-String| Integer32| Integer64| Double| Date| Boolean
…29
MongoDB > Dokument{"_id" : ObjectId("53e3663ccb3bd259f9252f67"),"typ" : ["gastro", "kultur"],"name" : "Unperfekthaus","tags" : "uph unperfekt perfekt haus essen","desc" : "Im Unperfekthaus bekommen Künstler & Gründer ... ","adresse" : { "str" : "Friedrich-Ebert-Straße 18",
"plz" : 45127,"ort" : "Essen"
},"location" : {"type" : "Point",
"coordinates" : [ 7.0075, 51.45902 ]}
}
30
MongoDB > Server
mongodServer
Database
Collection
local admin productstest
pois system.indexes system.profile
Doc 1Doc 2
Doc 3Index
...
...
Query Scope
31
MongoDB > APIs
MongoDB-Server
Browser REST-Client
Treiber
Anwendung
HTTP(optional)
REST(optional)
TCP/IP
verwendet
32
MongoDB > Java APIs
Object / DocumentMapper
Driver
MongoDB-Server
Java-Treiber
Spring Data MongoDB Morphia Jongo ...
TCP / IP
33
MongoDB > Tools > Mongo Shell
34
MongoDB > Tools > Robomongo
35
Neo4J
36
Neo4jGraph-Datenbank
37
Neo4j > FaktenVersion 2.2.5Kategorie: Graphen-DatenbankAbfragen: Query Languages: Cypher, Gremlin:
• Graph Traversal• Aggregation, Projektion, …
Indizes: Primär-Index + frei wählbare, auch zusammengesetzte Sekundär-Indizes
Replikation: Master/Slave mit automatischen FailoverSharding: Keine disjunkte Verteilung, aber sog. Cache
ShardingTX: ACID, mit JDBC auch JTA
38
Neo4j > FaktenAPI: • Java-API (Embedded)
• REST / HTTP• JDBC
Lizenz: GPL v3 (Community Edtion)Hersteller: Neo Technology Inc.Web: http://www.neo4j.org/Github: https://github.com/neo4j
39
Neo4j > Graph
Graph := ( V, E ) mit
V := Menge Knoten (Vertices)E := Teilmenge von (V x V) (Edges)
Neo4j-Graph := Gerichteter Graph,Knoten und Beziehungen können
Eigenschaftenhaben sog. Property Graph
40
Neo4j > Graph
name: "Joe"age: 35
name: "Neo4j" name: "MongoDB"
:Person
:Database :Database
:Graph :Document
:KNOWSskill: 10
since: 2007
:KNOWSskill: 5
41
Neo4j > Server
Neo4jServer
Database Graph
Doc 2Doc 3Index
KnotenDoc 2Doc 3Beziehungen
Doc 2Doc 3Labels
Constraints
42
Neo4j > APIs
Neo4j
(embedded)
JVM
Java API Anwendung
Neo4j
REST APITransactionalHTTP EndpointJDBC
43
Neo4j > Tools > Web-Interface
44
Redis
45
RedisKey-Value-Datenbank
46
Redis > FaktenVersion 3.0Kategorie: Key-Value-DatenbankAbfragen: Key-only, Pub/Sub
Indizes: Zugriff per se nur über Key
Replikation: Master/Slave mit autom. FailoverSharding: Hash-basierte, disjunkte Verteilung der
DatenTX: ACID mit opt. Locking auf Menge von KeysAPI: • Protokoll: ASCII over TCP/IP
• Treiber für viele Sprachen47
Redis > FaktenLizenz: Three clause BSD licenseHersteller: Salvatore Sanfilippo (Pivotal) and Pieter
NoordhuisWeb: http://www.redis.io/Github: https://github.com/antirez/redis
48
Redis > Key-ValueKey := StringValue := Simple Value | List | Set | Hash | ZSet
Simple Value := String | Integer
List := SV1, SV2, …
Set := SV1, SV2, … (eindeutig)
Hash := (K1, SV1, K2, SV2, …)
ZSet := (score_1, V1, score2, V2, …) (nach score sortierbar)
49
Redis >Beispiel
Simple a:1
l:12
"hello Redis"
"hello"List
"Redis" "hello"
Set s:1 "hello" "Redis"
Hash h:1 name "Redis"
typ "key-value"
version 2.8
ZSet z:1 42 "/"
23 "/inbox"
2 "/inbox/joe"
50
Redis > Server
redis-serverServer
Database
Keyspace
db0 db1 db15...
Key 1
Key 2
...
Value 1
Value 2
...
51
Redis > APIs
High LevelFrameworks
Low LevelJava Clients
Redis Server
JRedis
Spring Data Redis
Jedis Lettuce
ASCII over TCP / IP
52
Redis > Tools > redis-cli
53
Redis > Tools > Redis Desktop Manager
54
