VR in TSD

HTW Dresden - Artificial Intelligence Lab

Frank Bahrmann Andre Minz Sven HellbachHans-Joachim Böhme

26th November 2016

Agenda

F. Bahrmann VR in TSD 2 / 15

Motivation

State of VR Technology

Roadmap

User Tracking

Conclusion

Demo

Current Workflow

Motivation

⊲ Current Workflow

Current Workflow

State of VRTechnology

Roadmap

User Tracking

Conclusion

Demo

F. Bahrmann VR in TSD 3 / 15

1. Implement new features, hunt old bugs2. Drive to Museum3. Deploy new version4. Prepare Robot for experiments

(a) Attach new/exclusive Hardware(b) Charge Batteries(c) Drive to Starting Position

5. Prepare environment for experiments

(a) Wizard of Oz Workstation(b) Network Connections(c) (Prepare special exhibits)

6. Wait for visitors that are willing to cooperate with the robot / us7. Copy Log Files and Pack Up8. Drive to HTW9. Analyze the data and draw conclusions

10. GOTO 1

Current Workflow

Motivation

Current Workflow

⊲ Current Workflow

State of VRTechnology

Roadmap

User Tracking

Conclusion

Demo

F. Bahrmann VR in TSD 4 / 15

1. Implement new features, hunt old bugs2. Drive to Museum3. Deploy new version4. Prepare Robot for experiments

(a) Attach new/exclusive Hardware(b) Charge Batteries(c) Drive to Starting Position

5. Prepare environment for experiments

(a) Wizard of Oz Workstation(b) Network Connections(c) (Prepare special exhibits)

6. Wait for visitors that are willing to cooperate with the robot / us7. Copy Log Files and Pack Up8. Drive to HTW9. Analyze the data and draw conclusions

10. GOTO 1

Already available

Motivation

State of VRTechnology

⊲ Already available

Near Future

Impact

Roadmap

User Tracking

Conclusion

Demo

F. Bahrmann VR in TSD 5 / 15

Near Future

Motivation

State of VRTechnology

Already available

⊲ Near Future

Impact

Roadmap

User Tracking

Conclusion

Demo

F. Bahrmann VR in TSD 6 / 15

1. Wireless \o/

(a) GiFi - (60 Ghz-Wifi-Adapter by TPCast 1st Quarter2017) - 8gbs (real world conditions)

(b) LiFi - Visible Light Communications - 224 gbs (labconditions)

2. Higher Resolution

(a) OLED(b) 4k Mobile screens

3. Built In Eye Tracking

(a) FOVE Eyetracking VR Headset available forpreorder

(b) SMI Mobile Eye Tracking HMD (Sensor MotoricInstruments)

Impact

Motivation

State of VRTechnology

Already available

Near Future

⊲ Impact

Roadmap

User Tracking

Conclusion

Demo

F. Bahrmann VR in TSD 7 / 15

What is done...

Motivation

State of VRTechnology

Roadmap

⊲ What is done...

What is to do...

User Tracking

Conclusion

Demo

F. Bahrmann VR in TSD 8 / 15

• Greyboxing Environment with

◦ robot dummy◦ navigational scripts (motion + path planning)◦ Preparation of Points of Interest (later exhibits)

• Remodelled 3d-printer-ready CAD robot as usable 3D game object• Speech synthesis with same backend as real world robot• HTML5 based interactive robot display (based on awesomnium)• Locomotion via armswinger/vrtk script• First steps to technical photogrammetry

What is to do...

Motivation

State of VRTechnology

Roadmap

What is done...

⊲ What is to do...

User Tracking

Conclusion

Demo

F. Bahrmann VR in TSD 9 / 15

• More steps towards an easy accessible technical

photogrammetry• Implement interfaces to use our real world algorithms

within VR

◦ Path planning, obstacle avoidance◦ People tracking

• Projection onto exhibits• Refinements of existing solutions• Stepwise modelling of museum environment• User tests

User Tracking

Motivation

State of VRTechnology

Roadmap

⊲ User Tracking

Physical Properties

Intrinsic ParametersIIntrinsic ParametersII

Conclusion

Demo

F. Bahrmann VR in TSD 10 / 15

Physical Properties

Motivation

State of VRTechnology

Roadmap

User Tracking

⊲PhysicalProperties

Intrinsic ParametersIIntrinsic ParametersII

Conclusion

Demo

F. Bahrmann VR in TSD 11 / 15

• User position in space• Gaze Estimation• Position of Points of Interest and of robot• Speech Recognition with

◦ acoustic and semantic language processing◦ interest / emotion estimation

Intrinsic Parameters I

Motivation

State of VRTechnology

Roadmap

User Tracking

Physical Properties

⊲IntrinsicParameters I

Intrinsic ParametersII

Conclusion

Demo

F. Bahrmann VR in TSD 12 / 15

Wheel of Emotions, by RobertPlutchik

..., by Robin Nixon

Intrinsic Parameters II

Motivation

State of VRTechnology

Roadmap

User Tracking

Physical Properties

Intrinsic ParametersI

⊲IntrinsicParameters II

Conclusion

Demo

F. Bahrmann VR in TSD 13 / 15

TiltBrush Color Picker by Google

Conclusion

Motivation

State of VRTechnology

Roadmap

User Tracking

Conclusion

⊲ Conclusion

Demo

F. Bahrmann VR in TSD 14 / 15

• Amazingly easy to create software for/in/with VR with aid ofmodern game engines

• New VR hardware can immerse people in a virtual scene• Challenges:

◦ Cope with photogrammetry problems◦ Usability Testing the usability tests

• Find ways to apply VR to elderly care

Demo

F. Bahrmann VR in TSD 15 / 15

Thanks for listening and

enjoy the video.