Wir sind aber nicht Twitter

Wir sind aber nicht TwitterWas alternative Datenbanken jenseits von Skalierung bieten.

Frank Celler

Developer Week 2013Nürnberg

www.arangodb.org (c) [email protected]

Mittwoch, 26. Juni 13

http://www.arangodb.org


Die NoSQL Bewegung






http://www.strozzi.it/cgi-bin/CSA/tw7/I/en_US/nosql/Home%20Page

NoSQLA Relational Database Management System

NoSQL is a fast, portable, relational database management system without arbitrary limits, (other than memory and processor speed) that runs under, and interacts with, the UNIX1 Operating System. It uses the "Operator-Stream Paradigm" described in "Unix Review", March, 1991, page 24, entitled "A 4GL Language". There are a number of "operators" that each perform a unique function on the data. The "stream" is supplied by the UNIX Input/Output redirection mechanism. Therefore each operator processes some data and then passes it along to the next operator via the UNIX pipe function. This is very efficient as UNIX pipes are implemented in memory. NoSQL is compliant with the "Relational Model".

Carlo Strozzi, 1998







Traffic, Traffic, Traffic





http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html

Dynamo: Amazon’s Highly Available Key-value StoreGiuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall and Werner Vogels

Amazon.com

Abstract

Reliability at massive scale is one of the biggest challenges we face at Amazon.com, one of the largest e-commerce operations in the world; even the slightest outage has significant financial consequences and impacts customer trust. The Amazon.com platform, which provides services for many web sites worldwide, is implemented on top of an infrastructure of tens of thousands of servers and network components located in many datacenters around the world. At this scale, small and large components fail continuously and the way persistent state is managed in the face of these failures drives the reliability and scalability of the software systems.

This paper presents the design and implementation of Dynamo, a highly available key-value storage system that some of Amazon’s core services use to provide an “always-on” experience. To achieve this level of availability, Dynamo sacrifices consistency under certain failure scenarios. It makes extensive use of object versioning and application-assisted conflict resolution in a manner that provides a novel interface for developers to use.

Werner Vogels, 2007






CAP


ACID

BASE

Was ist neu?





NOSQL meetupThursday, June 11, 2009 from 10:00 AM to 5:00 PM (PDT)

San Francisco, CA

IntroductionThis meetup is about "open source, distributed, non relational databases".

Have you run into limitations with traditional relational databases? Don't mind trading a query language for scalability? Or perhaps you just like shiny new things to try out? Either way this meetup is for you.

Join us in figuring out why these newfangled Dynamo clones and BigTables have become so popular lately. We have gathered presenters from the most interesting projects around to give us all an introduction to the field.

http://nosql.eventbrite.com/






Dokument Datenbanken






Warum NoSQL?Reasons to use NoSQL databases

This is interesting. If you group together the sections that deal with ‘ease of application development’ – which are ‘simpler data model’, ‘schema less’ and ‘rapid development’ – then these come to a total of 17. This far outweighs any other result. It would seem that most people use NoSQL databases because they’re easier to develop application on.

This likely accounts for why you come across Open Source NoSQL databases on the web, but not in large organisations doing serious work – for a large and complex system ease of development is not a primary concern. These organisations are instead concerned with ‘doing things properly’ – i.e. Enterprise class solutions.

Adam Fowler, 2013

http://adamfowlerml.wordpress.com/2013/01/04/why-use-a-nosql-database-and-why-not/






• relationales Gebot: Daten sollen normalisiert gespeichert werden

• Redundanzen auflösen

• Konsistenz und Integrität der Daten erhöhen


RDBMS Normalisierung




mailto:[email protected]


• Eine „Person“ hat

• Vor und Zuname

• eine oder mehrere Adressen, Email-Adressen, Telefonnummern

• Liste für Hobbies, Skills, Tags


RDBMS Normalisierung







PersonPersonPerson

AddressAddressAddress

HobbyHobbyHobby

emailemailemail

n-to-mn-to-mn-to-m

n-to-mn-to-mn-to-m

n-to-mn-to-mn-to-m

Relationale Welt







Relationale Welt

• normalisierte Daten sind meist auf mehrere Tabellen verteilt

• zur Abfrage benötigt man dann Joins

• es können Beziehungen (references) und Konsistenzbedingungen (constraints) definiert werden

• diese werden von der Datenbank erzwungen







{ name: { first: „Frank“, last: „Celler“ },

address: { home: { street: „...“, city: „...“ } },

email: { work: „[email protected]“ },

hobbies: [ „C++“, „MRuby“, „JavaScript“ ] }

Aggregate aka Dokument










• Dokument

• einfache Key / Value Paare

• Listen und Unterobjekte

• vielfach als JSON

• Schema-Freiheit

• keine objektrelationale Unverträglichkeit








• es ist "erwünscht", ein Objekt als zusammengehöriges Aggregat zu speichern (keine Aufteilung der Objekteigenschaften auf verschiedene "Tabellen" etc.)

• durch die Aggregatorientierung ist das Lesen und Speichern von kompletten Objekten einfach und schnell








• Schema ist in gespeicherten Daten inhärent vorhanden

• verschiedene Versionen eines "Schemas" können parallel existieren:

• { name: "..." }

• { name: { first: "...", last: "..." } }

• { firstName: "...", lastName: "..." }


Schema-Freiheit






• kein Migrationszwang: man kann selbst entscheiden, ob (und wann) "alte" Daten migriert werden

• wenn keine Migration alter Daten erfolgt, muss die Applikation mit verschiedenen Versionen umgehen können

• mehr Kontrolle = mehr Verantwortung


Schema-Freiheit






• auf die Attribute von Dokumenten kann auch einzeln zugegriffen werden

• teilweise können auch Indizes auf die einzelnen Attribute angelegt werden

• zur Zeit keine einheitliche Abfragesprache, noch nicht einmal einheitliches Paradigma

• selten Joins notwendig


Abfragen






Graph Datenbanken





(c) Geek & Poke



Graph Datenbanken

• ein "Graph" ist die Gesamtheit von

• Objekten (Knoten, auch "vertices") sowie

• Beziehungen (Kanten, auch "edges")





Type: inproceedingTitle: Finite Size Effects

Type: proceedingTitle: Neural Modeling

Type: personName: Anthony C. C.

Coolen

Label: written

Label: publishedPages: 99-120

Type: personName: Snchez-Andrs

Label: edited

Just Documents

Graph Datenbanken




• Hierarchien, Klassifikationen (Bäume)

• soziale Beziehungen (Netzwerke)

• geographische Daten (Navigation, Routen)

• die Objekte selbst und die direkten und indirekten Beziehungen ("Pfade") sind abfragbar


Einsatzmöglichkeiten






• Graph-Abfragen sind oft komplex

• Traversals: in der Datenbank ausgeführte Programme

• Gremlin (Datenbank-übergreifende Traversal-Skriptsprache)

• Datenbank-eigene Abfragemöglichkeiten(z. B. Abfragesprache "Cypher" in Neo4j)


Abfragen






Multi-Modell Datenbanken






• Knoten und Kanten sind Dokumente

• Abfragen wie analog Dokument-Datenbanken

• Range, Geo, Full-Text

• Beziehung als Graph

• Traversieren des Graphen

Multi-Modell Datenbanken




ArangoDB.explain()


{

"type": "NoSQL database","model": [ "document", "graph", "key-value" ],

"openSource": true,"license“: "apache","version": 1.3,

"builtWith": [ "C", "C++", "js" ],"Javascript": [ "client side", "server side" ],"uses": [ "V8" ]

}




NoSQL Landschaft





• in den letzten Jahren sind viele neue, nicht-relationale Datenbanken entstanden

• diese lösen einige der genannten Probleme (Skalierung, Schemas, dynamische Daten, impedance mismatch)

• teilweise durch weniger Zusicherungen und Garantien (a.k.a. Delegation von Aufgaben an die Applikation)


NoSQL Datenbanken






(c) 451 Research31




www.arangodb.org (c) [email protected] 32

Analytic Processing DBsTransaction Processing DBsManaging the evolving state of an IT system

Map/Reduce

Graphs

Extensibility

Key/Value

Column-Stores

Complex Queries

StructuredData

MassivelyDistributed

Documents






JavaScript in der Datenbank





JavaScript Everywhere

• im Browser

• Application-Layer (Node.JS)

• Datenbank







JavaScript in a Database

• MVC im Browser und/oder NODE.JS

• Script-Sprache in der Datenbank

• Transaktionen als Functionen

• Graph-Traversal

• no-backend, just API







db._create("accounts"); db.accounts.save({ _key: "john", amount: 423 });db.accounts.save({ _key: "fred", amount: 197 });

db._executeTransaction({ collections: { write: "accounts" }, params: { user1: "fred", user2: "john", amount: 10 }, action: function (params) { var db = require("internal").db; var account1 = db.accounts.document(params['user1']); var account2 = db.accounts.document(params['user2']); var amount = params['amount'];

if (account1.amount < amount) { throw "account of user '" + user1 + "' does not have enough money!"; }

db.accounts.update(account1, { amount : account1.amount - amount }); db.accounts.update(account2, { amount : account2.amount + amount });

/* will commit the transaction and return the value true */ return true; }});


Transactions




db._executeTransaction({ collections: { write: "accounts" }, params: { user1: "fred", user2: "john", amount: 10 }, action: function (params) { var db = require("internal").db; var account1 = db.accounts.document(params['user1']); var account2 = db.accounts.document(params['user2']); var amount = params['amount'];

if (account1.amount < amount) { throw "account of user '" + user1 + "' does not have enough money!"; }

db.accounts.update(account1, { amount : account1.amount - amount }); db.accounts.update(account2, { amount : account2.amount + amount });


Transactions




function traverse (graph, start, depth, maxlen) { var config = { datasource: traversal.collectionDatasourceFactory(graph._edges), strategy: Traverser.BREADTH_FIRST, expander: traversal.anyExpander, filter: traversal.maxDepthFilter, maxDepth: depth, uniqueness: { edges: Traverser.UNIQUE_GLOBAL, vertices: Traverser.UNIQUE_NONE }, visitor: coauthorVisitor(maxlen) };

var traverser = new Traverser(config);

var result = { positions: {}, vertices: [], links: [], minYear: 0, maxYear: 0 }; var first = graph._vertices.firstExample({ name: start });

if (first !== null) { traverser.traverse(result, first); }

return { start: start, depth: depth, minYear: result.minYear, maxYear: result.maxYear, vertices: result.vertices, links: result.links };}


Graph Traversal




function traverse (graph, start, depth, maxlen) {

var config = {

datasource: traversal.collectionDatasourceFactory(graph._edges),

strategy: Traverser.BREADTH_FIRST,

expander: traversal.anyExpander,

filter: traversal.maxDepthFilter,

maxDepth: depth,

uniqueness: { edges: Traverser.UNIQUE_GLOBAL,

vertices: Traverser.UNIQUE_NONE },

visitor: coauthorVisitor(maxlen)

};


Graph Traversal





No-Backend




To Be Continued

• Fork me on github

• Google Group: ArangoDB

• Twitter: @fceller & @arangodb

• www.arangodb.org


Stay in Touch:




Technology

Wir sind aber nicht Twitter