40
www.ima-zlw-ifu.rwth-aachen.de AI in 45 minutes – How Artificial Intelligence Shapes our Future of Production Aachener ERP-Tage 2016 Planung und Regelung 4.0 – Das Zusammenwachsen von ERP und MES June 16 th , 2016 Univ.-Prof. Dr. rer. nat. Sabina Jeschke IMA/ZLW & IfU Faculty of Mechanical Engineering RWTH Aachen University

AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

  • Upload
    others

  • View
    10

  • Download
    0

Embed Size (px)

Citation preview

Page 1: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

www.ima-zlw-ifu.rwth-aachen.de

AI in 45 minutes –How Artificial Intelligence Shapes our Future of Production

Aachener ERP-Tage 2016Planung und Regelung 4.0 –Das Zusammenwachsen von ERP und MES

June 16th, 2016

Univ.-Prof. Dr. rer. nat. Sabina Jeschke

IMA/ZLW & IfU

Faculty of Mechanical Engineering

RWTH Aachen University

Page 2: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

2

11.05.2016

S. Jeschke

Outline

I. Introduction

The rise of AI… and its relation to 4.0

Entering the scene: intelligent self-learning systems

II. The basics of machine learning and their applications

Data-driven methods: supervised and unsupervised learning

Trial-and-error driven methods: neuroevolution

Probabilistic engines

Deep learning – a powerful tool for “both sides”

Where the story goes: AlphaGo and other stories

Machines getting creative

III. The brain projects

To be or not to be …a bird!

The death of Moore’s law

The limitations of the Von Neumann architecture

Neuromorphic computing

IV. Summary and Outlook

The concept of cognitive computing

The embodiment theory and its implications for your “colleague the robot”

The END!

Page 3: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

3

11.05.2016

S. Jeschke

… leading to the 4th industrial (r)evolution...

Breakthroughs - A new era of artificial intelligence

Communication technologybandwidth and computational power

Embedded systemsminiaturization

Semantic technologiesinformation integration

Watson 2011

Google Car2012

Systems of “human-like” complexity

Artificial intelligencebehavior and decision support

Page 4: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

4

11.05.2016

S. Jeschke

Communication technologybandwidth and computational power

Embedded systemsminiaturization

Semantic technologiesinformation integration

… leading to the 4th industrial (r)evolution...

Breakthroughs - Everybody and everything is networked

Artificial intelligencebehavior and decision support

Team Robotics

Swarm Robotics

Smart GridSmart

Factory

Car2Infra-structure

Page 5: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

5

11.05.2016

S. Jeschke

The fourth industrial (r)evolution

“Information Revolution”

Everybody and everything is networked. - Big Data & Cyber-Physical Systems

Power revolutionCentralized electric power infrastructure; mass production by division of labor

1st industrial revolutionMechanical production systematically using the power of water and steam

today

Digital revolutionDigital computing and communication technology, enhancing systems’ intelligence

Information revolutionEverybody and everything is networked – networked information as a “huge brain”

around 1750 around 1900 around 1970

Weidmüller, Vission 2020 - Industrial Revolution 4.0Intelligently networked, self-controlling manufacturing systems)

“Internet of Things & Services, M2M or Cyber Physical Systems are much more than just buzzwords for the outlook of connecting 50 billions devices by 2015.”

Dr. Stefan Ferber, Bosch (2011)

Vision of Wireless Next Generation System (WiNGS) Lab at the University of Texas at San Antonio, Dr. Kelley

„local“ to „global“

„local“ to „global“

Page 6: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

6

11.05.2016

S. Jeschke

Communication technologybandwidth and computational power

Embedded systemsminiaturization

Semantic technologiesinformation integration

… towards a networked world

And how do these systems work?

Power revolutionCentralized electric power infrastructure; mass production by division of labor

1st industrial revolutionMechanical production systematically using the power of water and steam

today

Digital revolutionDigital computing and communication technology, enhancing systems’ intelligence

Information revolutionEverybody and everything is networked – networked information as a “huge brain”

around 1750 around 1900 around 1970

Towards intelligent and (partly-)autonomous systems AND systems of systems

?? Steering -Controlling ??

Page 7: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

7

11.05.2016

S. Jeschke

Outline

I. Introduction

The rise of AI… and its relation to 4.0

Entering the scene: intelligent self-learning systems

II. The basics of machine learning and their applications

Data-driven methods: supervised and unsupervised learning

Trial-and-error driven methods: neuroevolution

Probabilistic engines

Deep learning – a powerful tool for “both sides”

Where the story goes: AlphaGo and other stories

Machines getting creative

III. The brain projects

To be or not to be …a bird!

The death of Moore’s law

The limitations of the Von Neumann architecture

Neuromorphic computing

IV. Summary and Outlook

The concept of cognitive computing

The embodiment theory and its implications for your “colleague the robot”

The END!

Page 8: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

8

11.05.2016

S. Jeschke

Towards machine learning

Machines and learning

?Can machines learn?Can they learn to predict future states and to do tasks optimized and in the right way? And if so, how can they do it?

This is what this talk is about!

! Let us take a look into a first example of data-driven learning!

How do machines learn?

A –Learning by observations and explanations

Data-driven learning

B –Learning by doing

Trial-and-error learning

! Later…

Page 9: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

9

11.05.2016

S. Jeschke

Data-driven learning - supervised

A first example – learning from guided observations

! Do you remember your childhood heroes – “The Mario Brothers” by Nintendo?

So let us write down our observations (and gather some training data)

pos_x on_ground action status

563 yes jump (B) alive (1)

571 yes jump (A) alive (1)

580 yes walk right dead (0)

582 no jump (A) dead (0)

… … … …

walk right

jump (B)

We want to learn general rules how to survive in this situation - by using data –and visualize it in a decision tree

pos_xon_groundyes

no

> 560 and < 575action

jump (B)

jump (A)confidence (c) = 50%

c = 100%

c = 75%c = 75%

Page 10: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

10

11.05.2016

S. Jeschke

Data-driven learning - supervised

Supervised learning down-to-earth

? Can we predict the result of a HPDC (high-pressure die casting) process –by using historical data? - YES WE CAN!

… in cooperation with

We extended the prediction model by integrating mechanical vibration (using solid-borne sound sensors) weather data.

Acoustic measurementsFourier transformation &

feature extraction

Weather dataTemporal correlation of

weather (and circumstances)

Extended modelk-nearest clustering and

random forest tree

HPDC process Historic dataProcess and quality data

Prediction modelModelling and

training

Visualization of predictionInline and web-based

(result NIO|IO with reason)

IONIO (Outbreak)

NIO (Blowhole)

NIO (Cold shot)

Page 11: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

11

11.05.2016

S. Jeschke

Data-driven learning - unsupervised

A second example – what if we do not tell what is right

? What if we do not know if an observation belongs to a specific category?Or, if an observation is good or bad?

Finding the hidden structure in data!“Although it may seem somewhat mysterious to imagine what a machine could possibly learn given that it doesn't get any feedback from its environment, it is possible to find patterns in image data using probabilistic techniques.”

Zoubin Ghahramani, Professor of Information Engineering at the University of Cambridge, Machine Learning

Batch of unlabeled picturesCleansing, preprocessing and

hierarchical clustering

Unsupervised. Human factor is reduced to modeling. (however a certain bias survives…)

Page 12: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

12

11.05.2016

S. Jeschke

Data-driven learning - unsupervised

Unsupervised learning “down-to-earth”

! Finding hidden relations in our data, we were not aware of, e.g. understanding failures or bad quality of products and processes

Data about chemical compositions of steel (identified as low quality - example)

Searching for hidden relations in databy applying subgroup mining

Sulfur (S) > 0.04% and heat treatment fragilestructure

Phosphorus (P) > 0.04% reduced plasticity Chrome (Cr) > 16%, Molybdenum (Mo) > 13%,

Nickel (Ni) > 56% no findings …

[Ruiz, 2014]

… in cooperation with

Page 13: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

13

11.05.2016

S. Jeschke

Learning by doing – reinforcement learning

The next step: Using rewards to learn actions

? Remember Mario: What if the machine could learn, how to solve a level? Why not use a some kind of intelligent trial-and-error?

Reinforcement learning (R-learning) is inspired by behaviorist psychology –maximizing the expected return by applying a sequence of actions at a current state.

can be applied to broad variety of problems

[SethBling, 2015]

Neuroevolution of augmenting topologies (NEAT)

Genetic algorithms on top of neural networks

At each state the system decides what action to do

Actions are rewarded if Mario do not die in return

Level progress by evolving neural networks

[Stanley, 2002]

Human factor is “very small” reduced to very general, mainly

formal specifications of the neural network…

However, human still influences the underlying representation model

Page 14: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

14

11.05.2016

S. Jeschke

Learning by doing

Reinforcement learning “down-to-earth”

[MiorSoft (reexre), 2014][TU Delft, 2012]

… for learning and optimization of motions

… as “pro-training” for human-machine interaction

[Intelligent AutonomousSystems, 2015]

Should Google have crashed 10.000 cars before coming up with first „ok-solutions“ for autonomous driving?

Avoiding “nonsense solutions” by using simulation environments

[UC Berkeley, 2015]

… for learning and executing complete assembly tasks

[UC Berkeley, 2015]

! Obviously: Super-Mario can easily be extended towards intralogistics scenarios…

Coupling to embodiment theory

Page 15: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

15

11.05.2016

S. Jeschke

“Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection” [Levine et.al., Google, 02/2016]

Learning by doing

The “Kindergarten for robots”

! Transferring human/biological learning processes into all areas -combined with the power of cooperation, “collective learning”

A new approach:object manipulation by “trial-and-error” approach is goal-centric (not insight-oriented!) two components:

1. a grasp success predictor, which uses a deep convolutional neural network (CNN) to determine the success potential a given motion

2. a continuous servoing mechanism, that uses the CNN to continuously update the robot’s motor commands (feedback loop)

trained using a dataset of over 800,000 grasps collected using a cluster of 14 similar (but not

identical !!) robotic manipulators

object manipulation “up to today”

humans and animals: fast feedback loop between perception and action; even very complex manipulation tasks can be performed without advance planning

robotic manipulation: relies heavily on advance planning and analysis;with relatively simple feedback, such as trajectory following (results often slow and unstable, non-adaptive)

Page 16: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

16

11.05.2016

S. Jeschke

Deep learning

The age of deep learning (deep neural networks)

!“Today, computers are beginning to be able to generate human-like insights into data…. Underlying … is the application of large artificial neural networks to machine learning, often referred to as deep learning.” [Cognitive Labs, 2016]

Deep Q-Networks (also "deep reinforcement learning“, Q refers to the mathematical action-prediction-function behind the scenes….): Learning directly from high-dimensional sensory input

[nature, 2015]

AI starts to develop strategies to beat the game Signs of “body cousciousness” …

[Minh, 2015]

Human factor practically zero.

Page 17: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

17

11.05.2016

S. Jeschke

Deep learning

Deep learning “down-to-earth”

! … a variety of practical applications

Central part of “cognitive computing”

Handwriting recognitionAnomaly recognition

Natural language processingAutomated translation

Face/picture/object recognitionImportant feature for autonomous driving etc.

Page 18: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

18

11.05.2016

S. Jeschke

Linguisticpreprocessing

Generation of possiblecandidates

Evaluation of candidates

Probabilistic engines

The new probabilistic engines

? Back to Watson: how is this guy running the (Jeopardy!) show??

DeepQA architecture Purely based on natural language processing (NLP) Approx. 100 different AI/linguistic methods come into play Without any specific semantic representation (“as-is”)

90 IBM-Power-750 servers For each: a 3.5 GHz POWER7

processor, with 8 cores, and 4 threads per core

In total: 2.880 POWER7 threads 16 terabytes of RAM

Page 19: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

19

11.05.2016

S. Jeschke

Probabilistic engines

Probabilistic engines “down-to-earth”

! Watson: from playing Jeopardy! towards becoming some kind of a “medical doctor”…

today, only 20% of the medical knowledge is evidence based (basis of individualized medicine)

also, amount of medical information is doubling every 5 years: physicians can’t read all the journals

Data: all types, 1. structured data from electronic medical record

databases and 2. unstructured text from physician notes and

published literature

How can we deal with these challenges?

Goal of Watson: help physicians in diagnosing and treating patients by analyzing large data

acting as a huge preprocessor for all kind of medical information

potential to transform health care into individual medicine

currently tested by several clinics, e.g. Mayo, MD Anderson, Cleveland, and Sloan-Kettering

“IBM's Watson is better at diagnosing cancer than human doctors”

Example “p53”: Watson identified possible treatments for protein p53 linked to many cancers

Example “Google Flu” (another engine): already now, doctors integrate the results of GoogleFlu (spreading and direction of contagious illnesses) as it is much faster and more precise as the results of the best medical centers in the world)

Page 20: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

20

11.05.2016

S. Jeschke

Deep learning

Where the Story Goes: AlphaGo

!

Go originated in China more than 2,500 years ago. Confucius wrote about it. As simple as the rules are, Go is a game of profound complexity. This complexity is what makes Go hard for computers to play, and an irresistible challenge to artificial intelligence (AI) researchers. [adapted from Hassabis, 2016]

Bringing it all together!

The problem: 2.57×10210 possible positions - that is more than the number of atoms in the universe, and more than a googol times (10100) larger than chess.

Training set30 million moves recorded fromgames played by humans experts

Creating deep neural networks12 network layers with millions ofneuron-like connections

Predicting the human move(57% of time)

Dat

a-d

rive

n le

arn

ing

Re

info

rce

me

nt

lear

nin

g

Learning non-human strategiesAlphaGo designed by Google DeepMind, played against itself in thousands of games and evolved its neural networks; Monte Carlo tree search

! Achieving one of the grand challenges of AI

March 2016:Beating Lee Se-dol (World Champion)AlphaGo won 4 games to 1.(5 years before time)

[Has

sab

is, 2

01

6]

Page 21: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

21

11.05.2016

S. Jeschke

!“Creativity is a phenomenon whereby something new … is formed. The created item may be intangible (such as an idea, a scientific theory, a musical composition or a joke) or a physical object (such as an invention, a literary work or a painting).” [adapted from Wikipedia, last visited 5/3/2016]

Deep learning

Microsoft Visual Storytelling (SIS): machines becoming creative

Visual-Storytelling by Microsoftbased on deep neural networks (convolutional neural

networks)

DII (descriptions for images in isolation): Traditional storytelling software

SIS (stories for images in sequence): new approach towards storytelling, including

Based on SIND - Sequential Image Narrative Dataset: 81,743 unique photos in 20,211 sequences, aligned to both descriptive (caption) and story language.

[Margaret Mitchell / Microsoft, 04/2016, together with colleagues from Facebook]

Page 22: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

22

11.05.2016

S. Jeschke

!“Creativity is a phenomenon whereby something new … is formed. The created item may be intangible (such as an idea, a scientific theory, a musical composition or a joke) or a physical object (such as an invention, a literary work or a painting).” [adapted from Wikipedia, last visited 5/3/2016]

Van Gogh’s Starry Nightinterpreted by Google DeepDream

based on deep neural networks

“Do Androids Dream of Electric Sheep?”

(science fiction novel by American writer Philip K. Dick, published in 1968)

Computational creativity (artificial creativity) … is a multidisciplinary endeavour that is located at the intersection of the fields of artificial intelligence, cognitive psychology, philosophy, and the arts. [adapted from Wikipedia, last visited 5/3/2016]

„Can machines be creative?“ by Iamus, a

computer cluster composing classical

music by genetic algorithms, concert for

Turings 100th birthday [youtube]

Deep learning

Google DeepDream: machines becoming creative

Page 23: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

23

11.05.2016

S. Jeschke

Outline

I. Introduction

The rise of AI… and its relation to 4.0

Entering the scene: intelligent self-learning systems

II. The basics of machine learning and their applications

Data-driven methods: supervised and unsupervised learning

Trial-and-error driven methods: neuroevolution

Probabilistic engines

Deep learning – a powerful tool for “both sides”

Where the story goes: AlphaGo and other stories

Machines getting creative

III. The brain projects

To be or not to be …a bird!

The death of Moore’s law

The limitations of the Von Neumann architecture

Neuromorphic computing

IV. Summary and Outlook

The concept of cognitive computing

The embodiment theory and its implications for your “colleague the robot”

The END!

Page 24: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

24

11.05.2016

S. Jeschke

Summary of part I

What a zoo! Get me out of here… :)

? Relax. Order is half of life.

A –Learning by observations and explanations

Data-driven learning

B –Learning by doing

Trial-and-error learning

WANT BETTER RESULT? – Just shake it!!

C–Using “biological brain structures”

Machine learningNeuromorphic

computing

neuroevolution

genetic algorithmsneural networks

deep learning

SARSA

Q-LearningProbabilistic engines

supervised…

un-supervised…

reinforcement learning

k-nearest clustering

random forest trees

decision trees

Monte Carlo tree search

Page 25: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

25

11.05.2016

S. Jeschke

neuromorphiccomputing

machinelearning

The “second way”

About the connection between birds and AI

“You don’t need to be a bird to fly.”

!

In traditional machine learning, “general purpose computers” are considered.

These computers have no strong similarities of biological brain structures.

Even if they work less effective than biological brains (so far), they have a enormous energy consumption.

!

In the “brain projects”, specialized computer architectures are developed, driven by biological paradigms.

These architectures are more efficient for certain tasks, but do not follow the “general purpose idea” any longer.

Hardware and software become strongly coupled. Thus, experimental changes become more complicated.

… however, it‘s ok to be a bird to fly!

Page 26: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

26

11.05.2016

S. Jeschke

Limits of general purpose computers and machine learning

The limitations of the Moore’s law

Gordon Moore (born 1929) co-founder and Chairman Emeritus

of Intel Corporation before: director of R&D at Fairchild

Semiconductor education: Berkeley, Caltech, Hopkins

Moore's law(from a speech given by Moore, in 1965)

… the prediction that the number of transistors that can be placed on the same volume and for constant costs will double every 18 months!

(above: the current definition; the original used a doubling speed of one year)

5 – the magicboundary…

Page 27: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

27

11.05.2016

S. Jeschke

Limits of general purpose computers and machine learning

Von Neumann architecture

John von Neumann (1903 – 1957) a Hungarian-American mathematician,

physicist (quantum mechanics), inventor, computer scientist, …

Manhattan project ENIAC project: “Great brain”, first pure

electronic computer, 1946 turing complete

!

Turing completeness:

colloquial: “a general-purpose computer can do what a turing machine can do”

And by Church’s lemma: “a turing machine can compute ‘every algorithm’”

thus, Turing completeness means: can compute everything as long as the algorithm is known

Von Neumann architecture is turing -complete

The von Neumann bottleneck:

The shared single bus between the program memory and data memory

limits the data transfer rate between CPU and memory

Resulting in an intellectual bottleneck: “only one thing at a time thinking”

Memory wall: CPU speed rises faster than the speed of transfer and in the memory

Page 28: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

28

11.05.2016

S. Jeschke

Towards new approaches in data sciences

What is neuromorphic computing?

“Neuromorphic engineering, also known as neuromorphic computing, … describing the use of very-large-scale integration (VLSI) systems containing electronic analog circuits to mimic neuro-biological architectures present in the nervous system.

Neuromorphic engineering is an interdisciplinary subject … to design artificial neural systems, such as vision systems, head-eye systems, auditory processors, and autonomous robots, whose physical architecture and design principles are based on those of biological nervous systems.

Page 29: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

29

11.05.2016

S. Jeschke

Towards new approaches in data sciences

The “Brain projects”

! Brain Initiative

White House BRAIN Initiative (Brain Research through Advancing Innovative Neurotechnologies)

Established: 2013 by Obama Timeline: about 10 years

Directors: Cornelia Bargmann and William Newsome

Web: http://www.braininitiative.nih.gov/

total costs: private-public mix, about 100 mio $ in 2014, …

!

Human Brain Project (HBP)

EC FET Flagship

Established : 2013 Timeline: about 10 years

Director: Henry Markram Coordinator: EPFL Web: www.humanbrainproject.eu Partners: 100 all over Europe, in Germany:

Heidelberg, Jülich, …

total costs: 1.19 billion €, about half of it provided by the EC

? Worldwide, already a strong competition has been started.The biggest research activities in this field are … :

Page 30: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

30

11.05.2016

S. Jeschke

Towards new approaches in data sciences

Examples for neuromorphic chips

Intel, sensor image processing architecture

SpiNNaker, component of the Human Brain Project

!

SpiNNaker: Spiking Neural Network Architecture

Spiking: inculdes time and temporal coding novel computer architecture goal: to use 1 mio. ARM processors (currently

0.5 mio) in a massively parallel computing platform based on spiking neural networks

“One Million Chips Mimic One Percent Of The Brain”

By Steve Furber, Univ. of Manchester, one of the world’s best microprocessor designers

!

Intel Reveals Spin-based Neuromorphic Chip Design with up to 300 times lower energy computation

Involves the combined use of spintronics and memristors (memory resistors, not constant but depend on the history of current). In a cross-bar switch lattice, lateral spin valves act as neurons, and memristors act as synapses.

[a paper published by Intel in June 2012, Sharad et.al, 2012: arXiv:1206.3227]

Page 31: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

31

11.05.2016

S. Jeschke

Towards new approaches in data sciences

Timeline – results so far and the expectations in the future

? After Moore’s law: What might be the replacement?A pipeline of new technologies to prolong Moore’s magic

“For some time, making transistors smaller has no longer been making them more energy-efficient; as a result, the operating speed of high-end chips has been on a plateau since the mid-2000s.”

[economist, 2016]

!

The short history of transistors

• 2004 strained silicon• 2007 metal oxides used to beat the

effects of tunneling• 2012 finFET transistors• 2020 “gate-all-around“ transistors

Beyond changes in the design of transistors, more exotic solutions may be needed? For example materials beyond silicon. But in the end, we just “stave off the need for something radical”.

[Greg Yeric, ARM transistor designer, 2016]

Page 32: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

32

11.05.2016

S. Jeschke

Is AI at its end because Moore is not holding much longer?

NO. New technologies will take over, however there may be a gap in time…

NO, because additionally, the future may lay in more and more powerful algorithms instead of pure computational speed and power.

Specialized chipsneuromorphic computing

New materials nanotubes, cadmiumtellurid,

graphene, molybdenite, …

Quantum computingmake direct use of quantum-

mechanical phenomenaNew geometries3D architectures, …

Towards new approaches in data sciences

In a nutshell, the alternatives to the beaten paths

Page 33: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

33

11.05.2016

S. Jeschke

Outline

I. Introduction

The rise of AI… and its relation to 4.0

Entering the scene: intelligent self-learning systems

II. The basics of machine learning and their applications

Data-driven methods: supervised and unsupervised learning

Trial-and-error driven methods: neuroevolution

Probabilistic engines

Deep learning – a powerful tool for “both sides”

Where the story goes: AlphaGo and other stories

Machines getting creative

III. The brain projects

To be or not to be …a bird!

The death of Moore’s law

The limitations of the Von Neumann architecture

Neuromorphic computing

IV. Summary and Outlook

The concept of cognitive computing

The embodiment theory and its implications for your “colleague the robot”

The END!

Page 34: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

34

11.05.2016

S. Jeschke

Summary

The “new AI”: Cognitive Computing

“Cognitive computing (CC) makes a new class of problems computable. It addresses complex situations that are characterized by ambiguity and uncertainty; in other words it handles human kinds of problems. …To do this, systems often need to weigh conflicting evidence and suggest an answer that is “best” rather than “right”.Cognitive computing systems make context computable.”

“Cognitive computing systems [are] a category of technologies that uses natural language processing and machine learning to enable people and machines to interact more naturally […]. These systems will learn and interact to provide expert assistance to scientists, engineers, lawyers, and other professionals in a fraction of the time it now takes.”

“Cognitive computing is the simulation of human thought processes in a computerized model…. involves self-learning systems that use data mining, pattern recognition and natural language processing to mimic the way the human brain works.”

Page 35: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

35

11.05.2016

S. Jeschke

Shadow Dexterous Hand

Summary

From embodiment … to humanoids

Zykov V., Mytilinaios E., Adams B., Lipson H. (2005) "Self-reproducing machines", Nature Vol. 435 No. 7038, pp. 163-164Bongard J., et al., Resilient Machines Through Continuous Self-Modeling, Science 314, 2006Lipson H. (2005) "Evolutionary Design and Evolutionary Robotics", Biomimetics, CRC Press (Bar Cohen, Ed.) pp. 129-155

Robonaut 2- NASA

The Bongard robot – learning through embodiment [Bongard, 2006; Lipson, 2007]

Embodiment theory:„intelligence needs a body“

The existence of a body (incl. sensors and actuators)are basic prerequisites to build experience and finally the development of intelligence.

Embodiment theory:„different bodies = different intelligences“

… leading to humanoids / humanoid components

Asimo Honda

KIT, Dillmann, SFB 588

Atlas 2016 – Boston Dynamics

Page 36: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

36

11.05.2016

S. Jeschke

Summary

When do you start embracing artificial intelligence?

!Waiting for Google to take over? –Google is addressing fields as gaming, mobility, language …Because there, they do get the data they need.They do not have the data for production lines. – So far….

[Blomberg, 2016]

!

Production engineering is still somewhat hesitating and waiting,

but you have the data and we have the domain experts (from production engineering as well as data science),

so - let’s get started!

Page 37: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

37

11.05.2016

S. Jeschke

Summary

… in five steps!

4.0: Revolution of (distributed) artificial

intelligence

Innovation –a question of culture

AI today

4th Industrial Revolution

Cognitive Computing and Embodiment

AI tomorrow

Page 38: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

www.ima-zlw-ifu.rwth-aachen.de

Thank you!

Univ.-Prof. Dr. rer. nat. Sabina JeschkeHead of Institute Cluster IMA/ZLW & IfUphone: +49 [email protected]

Co-authored by:

Prof. Dr.-Ing. Tobias MeisenInstitute Cluster IMA/ZLW & [email protected]

Dr.-Ing. Christian BüscherResearch group leader „Production Technology“[email protected]

Thorsten Sommer, M. Eng.Team „Knowledge Engineering“[email protected]

Page 39: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

39

11.05.2016

S. Jeschke

1968 Born in Kungälv/Schweden

1991 – 1997 Studies of Physics, Mathematics, Computer Sciences, TU Berlin1994 NASA Ames Research Center, Moffett Field, CA/USA10/1994 Fellowship „Studienstiftung des Deutschen Volkes“ 1997 Diploma Physics

1997 – 2000 Research Fellow , TU Berlin, Institute for Mathematics2000 – 2001 Lecturer, Georgia Institute of Technology, GA/USA2001 – 2004 Project leadership, TU Berlin, Institute for Mathematics04/2004 Ph.D. (Dr. rer. nat.), TU Berlin, in the field of Computer Sciences2004 Set-up and leadership of the Multimedia-Center at the TU Berlin

2005 – 2007 Juniorprofessor „New Media in Mathematics & Sciences“ & Director of the Multimedia-center MuLF, TU Berlin

2007 – 2009 Univ.-Professor, Institute for IT Service Technologies (IITS) & Director of the Computer Center (RUS), Department of Electrical Engineering, University of Stuttgart

since 06/2009 Univ.-Professor, Head of the Institute Cluster IMA/ZLW & IfU, Department of Mechanical Engineering, RWTH Aachen University

since 10/2011 Vice Dean of the Department of Mechanical Engineering, RWTH Aachen University

since 03/2012 Chairwoman VDI Aachen

since 05/2015 Supervisory Board of Körber AG, Hamburg

Prof. Dr. rer. nat. Sabina Jeschke

Page 40: AI in 45 minutes - Cybernetics Lab · Deep learning The age of deep learning (deep neural networks)! ^Today, computers are beginning to be able to generate human-like insights into

40

11.05.2016

S. Jeschke

References

[Bloomberg, 2016] Why 2015 Was a Breakthrough Year in Artificial Intelligence, http://www.bloomberg.com/news/articles/2015-12-08/why-2015-was-a-breakthrough-year-in-artificial-intelligence, last visited 18.02.2016

[Cognitive Labs, 2016] http://cognitivlabs.com/the-age-of-neural-networks/, last visited 18.02.2016

[Hassabis, 2016] AlphaGo: using machine learning to master the ancient game of Go, https://googleblog.blogspot.de/2016/01/alphago-machine-learning-game-go.html, last visited 18.02.2016

[Intelligent Autonomous Systems, 2015] Collaborative assembly with phase estimation (ISRR 2015), https://www.youtube.com/watch?v=4qDFv02xlNo, last visited 18.02.2016

[Minh, 2015] Mnih, Volodymyr; et al. (2015). “Human-level control through deep reinforcement learning”, 518: 529–533.

[MiorSoft (reexre), 2014] Neuroevolution - Car learns to drive, https://www.youtube.com/watch?v=5lJuEW-5vr8, last visited 18.02.2016

[nature, 2015] http://www.nature.com/nature/journal/v518/n7540/fig_tab/nature14236_SV1.html, last visited 18.02.2016

[Ruiz, 2014] Ruiz, Paula Andrea Rotes; et al. (2014). “An Interactive Approach for the Post-processing in a KDD Process”. In: Advances in Production Management Systems. Innovative and Knowledge-Based Production Management in a Global-Local World”, pp 93-100

[SethBling, 2015] MarI/O - Machine Learning for Video Games, https://www.youtube.com/watch?v=qv6UVOQ0F44, last visited 18.02.2016

[Stanley, 2002] Stanley, Kenneth O. and Miikkulainen, Risto (2002). “Evolving Neural Networks through Augmenting Topologies”. In: EvolutionaryComputation: 10(2) 2002, pp 99–127.

[TU Delft, 2012] TU Delft robot Leo learns to walk, https://www.youtube.com/watch?v=SBf5-eF-EIw, last visited 18.02.2016

[UC Berkeley, 2015] BRETT the Robot learns to put things together on his own, https://www.youtube.com/watch?v=JeVppkoloXs, last visited 18.02.2016

[Lee et.al. 2011) Unsupervised Learning of Hierarchical Representations with Convolutional Deep Belief Networks, http://www.cs.princeton.edu/~rajeshr/papers/cacm2011-researchHighlights-convDBN.pdf ; last visited 23.02.2016, doi:10.1145/2001269.2001295

[economist 2016] Technology Quarterly: After Moore’s Law, http://www.economist.com/technology-quarterly/2016-03-12/after-moores-law; last visited 06.04.2016