Decoupling of real and digital content in projection-based augmented reality systems using time multiplexed image capture

TY - JOUR

T1 - Decoupling of real and digital content in projection-based augmented reality systems using time multiplexed image capture

AU - Soomro, Shoaib R.

AU - Ulusoy, Erdem

AU - Urey, Hakan

N1 - Publisher Copyright: © 2017 Society for Imaging Science and Technology.

PY - 2017/1

Y1 - 2017/1

N2 - Projection-based augmented reality systems overlay digital information directly on real objects, while at the same time use cameras to capture the scene information. A common problem with such systems is that cameras see the projected image besides the real objects to some degree. This crosstalk reduces the object detection and digital content registration abilities. The authors propose a novel time sharing-based technique that facilitates the real and digital content decoupling in real time without crosstalk. The proposed technique is based on time sequential operation between a MEMS scanner-based mobile projector and rolling shutter image sensor. A MEMS mirror-based projector scans light beam in raster pattern pixel by pixel and completes full frame projection over a refresh period, while a rolling shutter image sensor sequentially collects scene light row by row. In the proposed technique, the image sensor is synchronized with scanning MEMS mirror and precisely follows the display scanner with a half-period lag to make the displayed content completely invisible for camera. An experimental setup consisting of laser pico projector, an image sensor, and a delay and amplifier circuit is developed. The performance of proposed technique is evaluated by measuring the crosstalk in captured content and sensor exposure limit. The results show 0% crosstalk in captured content up to 8 ms sensor exposure. High capture frame rate (up to 45 fps) is achieved by cyclically triggering a 3.2 MP, 60 fps CMOS sensor and using a 60 Hz pico projector.

AB - Projection-based augmented reality systems overlay digital information directly on real objects, while at the same time use cameras to capture the scene information. A common problem with such systems is that cameras see the projected image besides the real objects to some degree. This crosstalk reduces the object detection and digital content registration abilities. The authors propose a novel time sharing-based technique that facilitates the real and digital content decoupling in real time without crosstalk. The proposed technique is based on time sequential operation between a MEMS scanner-based mobile projector and rolling shutter image sensor. A MEMS mirror-based projector scans light beam in raster pattern pixel by pixel and completes full frame projection over a refresh period, while a rolling shutter image sensor sequentially collects scene light row by row. In the proposed technique, the image sensor is synchronized with scanning MEMS mirror and precisely follows the display scanner with a half-period lag to make the displayed content completely invisible for camera. An experimental setup consisting of laser pico projector, an image sensor, and a delay and amplifier circuit is developed. The performance of proposed technique is evaluated by measuring the crosstalk in captured content and sensor exposure limit. The results show 0% crosstalk in captured content up to 8 ms sensor exposure. High capture frame rate (up to 45 fps) is achieved by cyclically triggering a 3.2 MP, 60 fps CMOS sensor and using a 60 Hz pico projector.

UR - https://www.scopus.com/pages/publications/85041509429

U2 - 10.2352/J.ImagingSci.Technol.2017.61.1.010406

DO - 10.2352/J.ImagingSci.Technol.2017.61.1.010406

M3 - Conference article

AN - SCOPUS:85041509429

VL - 61

SP - 54

EP - 59

JO - IS and T International Symposium on Electronic Imaging Science and Technology

JF - IS and T International Symposium on Electronic Imaging Science and Technology

M1 - jist0277

T2 - Engineering Reality of Virtual Reality 2017

Y2 - 29 January 2017 through 2 February 2017

ER -