Container Environments with Intel SGX · 2017-07-14 · reduce number of enclave exits (Challenge 2) 15 SCONE C library Asynchronous system calls M:N threading Libraries Application

SCONE: Secure Linux Container Environments

with Intel SGXS. Arnautov, B. Trach, F. Gregor, Thomas Knauth, and A. Martin, Technische Universität Dresden; C.

Priebe, J. Lind, D. Muthukumaran, D. O'Keeffe, and M. Stillwell, Imperial College London; D. Goltzsche, Technische Universität Braunschweig; D. Eyers, University of Otago; R. Kapitza, Technische Universität

Braunschweig; P. Pietzuch, Imperial College London; C. Fetzer, Technische Universität Dresden

[email protected]

mailto:[email protected]?subject=

Trust Issues: The Provider’s Perspective

• Cloud provider does not trust users

• Use virtual machines to isolate users from each other and the host

• VMs only provide one way protection

2

Redis

OS

VMM

Firmware

Cloud platform

Staff

…

trust

ed

Trust Issues: The User’s Perspective

• Users trust their application

• Users must implicitly trust the cloud provider

• Existing applications implicitly assume trusted operating system

3

Redis

OS

VMM

Firmware

Cloud platform

Staff

…

untru

sted

Containers are the new VMs

• Containers provide resource isolation and bundling

• Smaller resource overhead than virtual machines

• Convenient tooling to create and deploy applications in the cloud

4

Disaster!

5

OS

VMM

Firmware

Cloud platform

Staff

…

untru

sted

Disaster!

6

OS

VMM

Firmware

Cloud platform

Staff

…

untru

sted

Disaster!

7

OS

VMM

Firmware

Cloud platform

Staff

…

untru

sted

Disaster!

8

OS

VMM

Firmware

Cloud platform

Staff

…

untru

sted

We want to …

9

OS

VMM

Firmware

Cloud platform

Staff

…

untru

sted

We want to …

9

OS

VMM

Firmware

Cloud platform

Staff

…

untru

sted

• run unmodified Linux applications …

We want to …

9

OS

VMM

Firmware

Cloud platform

Staff

…

untru

sted


• in containers …

We want to …

9

OS

VMM

Firmware

Cloud platform

Staff

…

untru

sted



• in an untrusted cloud …

We want to …

9

OS

VMM

Firmware

Cloud platform

Staff

…

untru

sted




• securely and …

We want to …

9

OS

VMM

Firmware

Cloud platform

Staff

…

untru

sted




• securely and …

• with acceptable performance

1010

Secure Guard Extensions

• New enclave processor mode

• Users can create a HW-enforced trusted environment

• Only trust Intel and Secure Guard Extensions (SGX) implementation

OS

VMM

Firmware

Cloud platform

Staff

…

untru

sted

Enclave

SGX: HW-enforced Security

• 18 new instructions to manage enclave life cycle

• Enclave memory only accessible from enclave

• Certain instructions disallowed, e.g., syscall

11

… EENTER …

Execute …

Return

privileged access from

OS, VMM, SMM forbidden

untrusted trusted

Challenge 1: Interface

• Haven (OSDI’14): library operating system in enclave

• Large TCB → more vulnerable

• Small interface (22 system calls)

• Shields protect the interface

12

Application Code

Host OS

Library OSC Library Libraries

Shielding layer

Library OS inside TCB

Exte

rnal

cont

ainer

inte

rface

trus

ted

untru

sted

Challenge 1: Interface

• Small TCB

• C library interface is complex

• Harder to protect

13

Application Code

Shim C Library

C LibraryHost OS

Libraries

Minimal TCB

Challenge 2: Performance

14

syst

em c

all f

requ

ency

(100

0s/s

econ

d)

1

100

10000

Threads

1 2 4 8

native

synchronous enclave exits

• pwrite() with 32 byte buffer • 4 cores with hyper threading

Challenge 2: Performance

14

syst

em c

all f

requ

ency

(100

0s/s

econ

d)

1

100

10000

Threads

1 2 4 8

native

synchronous enclave exits


8×

Host operating system

SCONE Architecture

15

LibrariesApplication

SCONE module Intel SGX driverContainer (cgroups)


SCONE Architecture• Enhanced C library → small

TCB (Challenge 1)

15

SCONE C library





TCB (Challenge 1)

• Asynchronous system calls and user space threading reduce number of enclave exits (Challenge 2)

15

SCONE C libraryAsynchronous system calls

M:N threading





TCB (Challenge 1)

• Asynchronous system calls and user space threading reduce number of enclave exits (Challenge 2)

• Network and file system shields actively protect user data

15

SCONE C libraryAsynchronous system calls

M:N threadingNetwork shield File system shield



Anatomy of a System Call

16

enclavekernel

read, fd, buf, size


17

enclave

read(fd, buf, size)

[0]

[2]

kernel

T1

[1]

system call slots

read, fd, buf, size


18

enclavekernel

read(fd, buf, size)S1T1

[0]

[2][1]

system call slots


19

enclavekernel

read(fd, buf, size)

[0]

T1

read, fd, buf, size[0]

[2][1]

system call slots


19

enclavekernel

read(fd, buf, size)T1


[2][1]

system call slots

read, fd, buf, sizeread(fd, buf, size)T2


19

enclavekernel



[2][1]

system call slots



19

enclavekernel



[2][1]

system call slots

switch to ready user space thread


read, fd, buf, size


19

enclavekernel



[2][1]

system call slots



read, fd, buf, size


19

enclavekernel



[2][1]

system call slots


[2]

read, fd, buf, size


20

enclavekernel




[2][1]

system call slots

read, fd, buf, size


20

enclavekernel




[2][1]

system call slots

[0]#2&$?%

read, fd, buf, size


20

enclavekernel




[2][1]

system call slots


[0]#2&$?%

read, fd, buf, size


20

enclavekernel




[2][1]

system call slots

[0]#2&$?%

read, fd, buf, size


20

enclavekernel




[2][1]

system call slots

[0]#2&$?%

GET K1decrypt buffer into enclave

Container Integration

21

Repository Docker Engine

Secure Image Enclave

SCONE ClientDocker Client


21




1. push image


21




1. push image

2. run


21




1. push image

3. pull image

2. run


21




1. push image

3. pull image

4. execute

2. run


21




1. push image

3. pull image

5. secure channel

4. execute

2. run

Syst

em c

all f

requ

ency

(100

0s/s

econ

d)

1

100

10000

Threads

1 2 3 4 5 6 7 8

System Call Performance

22

native

asyncsync


Syst

em c

all f

requ

ency

(100

0s/s

econ

d)

1

100

10000

Threads

1 2 3 4 5 6 7 8


22

native

asyncsync

async with 1 thread achieves 80%


Syst

em c

all f

requ

ency

(100

0s/s

econ

d)

1

100

10000

Threads

1 2 3 4 5 6 7 8


22

native

asyncsync

async with 1 thread achieves 80%

optimized queue may help


Apache Throughput

23

Late

ncy

(sec

onds

)

0

1

2

3

4

Throughput (requests / second)0 15000 30000 45000 60000

glibcasync

sync

0.8×

0.7×

Performance Overview

24

Application Throughput w.r.t. nativeasync (%) sync (%)

Memcached 120 113Apache 80 70NGINX 80 36Redis 60 20


24



inline encryption has less overhead


24



inline encryption has less overhead

inline encryption hurts performance with single thread

Summary

• Small trusted computing base (0.6× – 2.0× of native binary size)

• Low runtime overhead (0.6× – 1.2× of native throughput)

• Transparent to the container engine (e.g. Docker)

25

Documents

Container Environments with Intel SGX · 2017-07-14 · reduce number of enclave exits (Challenge 2) 15 SCONE C library Asynchronous system calls M:N threading Libraries Application