Next-GenerationCMOS Image Sensor:

Quanta Image Sensors

Eric R. FossumPlenary Talk (Zoom)

Matter and Technology Annual MeetingHelmholtz Association

June 16, 2021

THAYER SCHOOL OF ENGINEERING AT DARTMOUTH © ER Fossum 1

© ER Fossum 2

• Dr. Jiaju Ma Gigajot• Dr. Saleh Masoodian Gigajot• Dr. Leo Anzagira ON-Semi Gigajot• Dr. Donald Hondongwa Varex Gigajot• Dr. Dakota Starkey Gigajot• Dr. Song Chen FB/Oculus• Mr. Wei Deng • Mr. Zhaoyang Yin• Yue Song• Rachel Zizza• Prof. Kofi Odame• Prof. Jifeng Liu• Prof. Alex Hartov

• Thank you Rambus, TSMC, DARPA, NASA/RIT, Goodix

Contributors to QIS work at Dartmouth

• Mike Guidash (Rambus)• Jay Endsley (Rambus)• Prof. Yue Lu (Harvard)• Prof. Stanley Chan (Purdue)• Dr. Igor Carron (the net)• Prof. Atsushi Hamasaki (Hiroshima)• Mr. Ryohei Funatsu (NHK)

© ER Fossum 3

Cubicle

Computational image reconstruction

X-Y-t Bit Density Gray Scale

Vision: A billion pixels readout at 1000 fps (1Tb/s) with single photon-counting capability and consuming less than a watt.

Quanta Image Sensor (QIS)“Count Every Photon”

US Patent 9,225,918 Priority May 27, 2005

Specialized pixel “jot”

© ER Fossum 4

Imaging Paradigm Shift

• Might seem like a lot of extra work compared to an integrating bucket of charge in conventional CMOS image sensors (CIS).

• Counting every (visible) photon is about as sensitive as one can get for photography, security, defense, space, etc.

• Helps with small pixel vs. full-well capacity trade off.• Allows new capabilities in computational imaging such as:

• Trade off in sensitivity and resolution that can be scene-dependent or attention-dependent.

• Permits time-delay and integration in multiple independent tracks and arbitrary directions.

• Allows motion blur compensation for multiple targets.• Allows high apparent SNR for very low photon flux.

© ER Fossum 5

QIS Brief Timeline

1Mpixel Stacked RO 1000fps 1b QIS0.21e- analog

‘04 ‘05 ‘06 ‘07 ‘08 ‘09 ‘10 ‘11 ‘12 ‘13 ‘14 ‘15 ‘16 ‘17 ‘18 ‘19

Concept proposed by EF in J.Nakamura DSC book chapter

Concept presented at 2005 CCD/AIS Workshop

EF @ Siimpel EF @ Samsung

EF @ Dartmouth

Small, briefeffort at SamsungLed by EF

Start of CIS-QIS effortat Dartmouth and image reconstruction

Work on low power fast row x row readout

Pump gate jot

Sub 0.3e-

Gigajot spinoff

GJ 1st tapeout

Lower tempmeasurements

Color

‘20

2x ½ MpixSPAD-QIS

QVGA SPAD-QIS

© ER Fossum 6

Photon and photoelectron arrival rate described by Poisson process

𝑃𝑃 𝑘𝑘 =𝑒𝑒−𝐻𝐻 𝐻𝐻𝑘𝑘

𝑘𝑘!

Probability of k arrivals

𝑃𝑃 0 = 𝑒𝑒−𝐻𝐻

𝑃𝑃 𝑘𝑘 > 0 = 1 − 𝑃𝑃 0 = 1 − 𝑒𝑒−𝐻𝐻

Define quanta exposure H = φ τ H = 1 means expect 1 arrival on average.

Monte Carlo

For 1b QIS, only two states of interest

For ensemble of M jots, the expected number of 1’s : 𝑀𝑀1 = 𝑀𝑀 � 𝑃𝑃[𝑘𝑘 > 0]

© ER Fossum 7

Photoresponse as bit density

𝐵𝐵𝐵𝐵𝐵𝐵 𝐷𝐷𝑒𝑒𝐷𝐷𝐷𝐷𝐵𝐵𝐵𝐵𝐷𝐷 𝐷𝐷 ≜𝑀𝑀1

𝑀𝑀= 1 − 𝑒𝑒−𝐻𝐻

Average of one arrival per jot per integration time© ER Fossum 8

Issues with SPADs for QIS application

SPADs use avalanche multiplication for gain• High internal electric fields• Higher operating voltages (15-20V)• Larger pixels (8-25um)• High dark count rates (100-1000Hz)• Dead time• Low fill factor (low PDE <50%)• Low manufacturing yield• Small array sizes (below 0.1M jots)

But, SPADS are excellent for time resolved photon detection

© ER Fossum 9

2021 IEEE JSTQE Gramuglia et al

25um

Our approach (CMOS QIS)

No avalanche multiplication, one electron per photonUse very low capacitance sense node

∆V = ∆Q / Ce.g. 1mV = 1.6x10-19C / 160aF

One pixellight

electronsin silicon

amplifier

correlateddoublesampling (CDS)

Conversion gain CGdefined as q/C

or volts/electron

© ER Fossum 10

Voltage Output with No Electronics Noise𝑃𝑃 𝑘𝑘 =

𝑒𝑒−𝐻𝐻 𝐻𝐻𝑘𝑘

𝑘𝑘!, 𝑘𝑘 = 0, 1, 2, 3 …

H=2Probability mass function =0.27

Probability mass function =0.18

Probability mass function =0.09

Poisson probability mass function

CG = conversion gain = q/C [V/e-]

© ER Fossum 11

Quantized Values Broadened by Readout Noise

“0” “1”Single-bit QIS

© ER Fossum 12

Multi-bit jot increases flux capacity

Single bit jot0, 1 electrons

Multi-bit (2b) jot0, 1, 2, 3 electrons

𝜙𝜙𝑤𝑤𝑤𝑤 = 𝑗𝑗𝑗𝑗𝑟𝑟 2𝑤𝑤 − 1 /𝛿𝛿�̅�𝛾

At the flux capacity, there is an average of 2𝑤𝑤 − 1 photoelectronsper n-bit jot

Can increase flux capacity at same jot density and field readout rateOr, relax field readout rate and/or jot density for same flux capacity

Little impact on detector and storage well. Little impact on FD CG or voltage swing (e.g. 1mV/e -> 31mV swing for 5b jot.

© ER Fossum 13

Multi-bit QIS for Photon Number Resolution(e.g. 2-bit)

“00” “01” “10” “11”

© ER Fossum 14

Pump-Gate Jot: Minimize TG-FD overlap capacitanceHighest possible CG

(Lowest possible cap.)

BSI

TGFD

SW

BSI

vertical lateral

US Patent No. 9,728,565 B2Fossum, Ma, Hondongwa © ER Fossum 15

Experimental DataPhoton Counting Histogram for “a Golden Pixel”

20k reads of same jot, 0.175e- rms read noise ~21DN/e- (61.2uV rms 350uV/e- or 0.45fF)Room temperature, no avalanche, 20 CMS cycles, jot:TPG PTR BC

© ER Fossum 16

Model vs. Data = New characterization toolsH=2.12e- Un=0.188e- rms

Conversion gain from peak spacingQuanta exposure from relative peak heights

Read noise from valley-peak modulation value

© ER Fossum 17

Experimental DataPhoton Counting Histograms

20k reads of same jot, 0.20e- rms read noise ~21DN/e-Room temperature, no avalanche, 20 CMS cycles, jot:TPG PTR BC

Ma, Masoodian, Wang, Fossum 2017

H=8.25

© ER Fossum 18

Quantum Efficiency

Courtesy Gigajot

Standard CIS BSI Process

© ER Fossum 19

Lag

© ER Fossum © ER Fossum 20

Very Low Dark Current (0.07 cps at RT)

Room Temp: ~0.07e-/s avg. (~1pA/cm2)Previously measured ~2x every 10C

Storage well isolated from surface

Courtesy Gigajot

© ER Fossum 21

Noise in first transistor is critical

Log P(f)

Log f

1/f ~ 1/ gate area

thermalS

DG

© ER Fossum 22

1/f Noise Modelling Review

Hooge's mobility fluctuation modelPhonon-scattering-induced mobility fluctuation.

McWhorter's number fluctuation model

Carrier number fluctuation, interface traps.

Berkeley unified modelNumber fluctuation and the correlated mobility fluctuation at surface

Probability of occurrence: ~2%

SFRTN

1/f + thermal noise

Empirical and bulk

1DN=17uV350uV/e-

13e-input referred

1e- trapped at Si-SiO2equiv. to 13e- on gate!

© ER Fossum 23

At 0.14x0.27um gate area, expect 1-5 traps per MOSFET if NIT ≅ 1x1010 traps/cm2

Cooling to -70C Further reduces Read Noise and Improves CG

Every pixel is a little different

Recently, PRNU <1.5%[Gigajot]

© ER Fossum 24

“Backside” silicon surface

Readout “FD”

Detector wafer

Logic wafer

© ER Fossum

20 Mpixel single-photon detector array

• Process technology: TSMC CMOS BSI 45nm/65nm 2-layer stacking• Cluster parallel readout architecture for low power and modularity• 20 different 1Mpixel arrays on test chip• Readout Variation:

Analog Single-bit Digital

• Pixel: 1.1µm 1024x1024• Pixel variation: TPG, PTR, JFET

20 Mpixel single photon detector array© ER Fossum 26

Summary of Measured Results 1Mpixel 1b QIS Digital Subarray at Room Temp

Equiv. PD Dead Time <0.1%Array 1024 (H) x 1024 (V)

Field rate 1040fpsADC sampling rate 4MSa/s

ADC resolution 1 bit

Output data rate32 (output pins) x

34Mb/s= 1090Mb/s

Package PGA with 224 pins

Power

Array 2.3mW256 ADCs 7.5mW

Addressing 4.1mWI/O pads 3.7mW

Total 17.6mWFOM ADC 6.9pJ/b

Process 45nm (jot layer), 65nm (ASIC layer)

VDD

1.8V & 2.5V (Analog, digital and

array), 3V & 2.2V (I/O pads)

Jot typeBSI Tapered Pump

Gate2-Way Shared RO

Jot pitch 1.1µmBSI Fill Factor ~100%

Quantum Efficiency 79% @ 550nmConversion gain on

column 345µV/e-

Input Referred Noise 0.22e- r.m.s.Corresponding BER ~1%

Avg. Dark current (RT) 0.16e-/sEquiv. Dark Count Rate

(RT) 0.07Hz/jot

𝐹𝐹𝐹𝐹𝑀𝑀 =𝑃𝑃𝑃𝑃𝑃𝑃𝑒𝑒𝑃𝑃 𝐶𝐶𝑃𝑃𝐷𝐷𝐷𝐷𝐶𝐶𝐶𝐶𝐶𝐶𝐵𝐵𝐵𝐵𝑃𝑃𝐷𝐷

# 𝑃𝑃𝑗𝑗 𝐶𝐶𝐵𝐵𝑝𝑝𝑒𝑒𝑝𝑝𝐷𝐷 × 𝑗𝑗𝑃𝑃𝑓𝑓𝐶𝐶𝑒𝑒 𝑃𝑃𝑓𝑓𝐵𝐵𝑒𝑒[𝐶𝐶𝑝𝑝𝑏𝑏

]

© ER Fossum 27

J. Ma, S. Masoodian, D. Starkey, E.R. Fossum, Photon-number-resolving megapixel image sensor at room temperature without avalanche gain, OSA Optica, vol. 4, no. 12, pp.1474-1481, December 2017. https://doi.org/10.1364/OPTICA.4.001474

1Mjot prototype QIS experimental results

1040fps Target scene

Output

© ER Fossum 28

Saleh Masoodian Jiaju Ma

Gigajot spinoff (2017)

QIS great for low light, high resolution imagingand photon-number resolving systems

• Security systems• Low light vision• Internet of things (IOT)• Biological imaging• Astronomy• Quantum Cryptography• Photography• Cinematography

© ER Fossum 29

1 2 3 4 5 6 70

1Mpixel 3b QIS ImageExposure of 0.87e-/pixel average

Raw image and Histogram

2x2x2 cubicle sum only

2x2x2 cubicle denoise

© ER Fossum 30

0.01 lux, f/1.4 600msec

• Deep sub-electron read noise enables photon-counting and photon-number resolution in visible light range

• Fabricated in almost standard CMOS image sensor foundry process

• Operates at room temperature with ~1um or less pixel pitch

Near future: Probably QIS technology merges with CIS technology

Conclusions

© ER Fossum 33