optical-touch technology
`
`An Overview of Optical-Touch Technologies
`
`Recent breakthroughs haveled to a resurgencein optical touch systems, which werefirst
`introducedin the 1970s. Developers have been able to address issues of cost, performance
`in high-ambientlight, andform factor, to name just a few. This article details how these
`problems have been overcome and whatthe future holds for this technology, including a
`look at several completely new approaches to optical-touch systems.
`
`by Ian Maxwell
`
`E.DEVELOPEDatCarrollTouch
`
`(now part of Elo TouchSystems)in the 1970s
`and nowsold by numeroussuppliers, optical-
`touch systems offer many advantages when
`compared to other touch technologies. Many
`in the industry believe that, if not for two con-
`siderable drawbacks discussed below,optical-
`touch technology would today be the domi-
`nant touch technology. Recent technology
`developments in optical-touch screens could
`pave the wayto the renaissanceofoptical-
`touch technology as the dominant touch-
`screen technology.
`
`Introduction
`
`The conventionaloptical-touch system uses
`an array of infrared (IR) light-emitting diodes
`(LEDs) on twoadjacent bezel edges ofa dis-
`play, with photosensors placed on the two
`opposite bezel edges to analyze the system
`and determine a touch event. The LED and
`
`photosensorpairs create a grid of light beams
`across the display. An object (such as a finger
`or pen) that touches the screen interrupts the
`light beams, causing a measured decrease in
`Ian Maxwell is the Founder and Executive
`
`Director ofRPO, Inc. He has foundeda total
`of seven start-ups and is a Venture Partner
`with Allen & Buckeridge Venture Capital.
`He can be reachedat 18 Bulletin Place,
`Sydney NSW 2000, Australia; telephone
`+61 (2) 6125-4968, e-mail: i.maxwell@ rpo.
`biz.
`
`26
`
`Information Display 12/07
`
`light at the corresponding photosensors. The
`measured photosensor outputs can be used to
`locate a touch-point coordinate. Usually, the
`controller scans through the array of photo-
`sensors rather than measuringall of them
`simultaneously; thus, this touch technology is
`sometimescalled “scanning IR.” In more
`advanced versions of the technology, each
`
`photosensor measureslight from more than
`one LED,whichallows the controller to
`compensate for light blockage caused by non-
`moving debris on the screen (Fig. 1).
`This traditional type of optical touch has
`been used primarily in niche applications of
`the touch market. Historically, its broader use
`has been hampered by twofactors: the rela-
`
`Edgeof active
`display area
`
`_ Opto-matrix frame
`inside bezel
`
`Photoreceptors
`
`of infraredlight
`
`LEDscreate grid
`
`Inside and outside
`edges ofinfrared
`transparent bezel
`
`Fig. 1: A schematic representation of conventional optical-touch technology. Illustration
`courtesy of Elo TouchSystems.
`
`0362-0972/1/2007-0268|.0 @UERPWVARE EXHIBIT 2018, Page 1 of 5
`
`

`

`tively high cost of the technology compared to
`competing touch technologies andthe issue of
`performancein bright ambientlight. This
`latter problem is a result of background light
`increasing the noise floor at the optical sensor,
`sometimes to such a degree that the touch
`screen’s LED light cannot be detectedatall,
`causing a temporaryfailure of the touch
`screen. This is most pronouncedin direct-
`sunlight conditions where the sun has a very
`high energy distribution in the IR region.
`In addition, conventional optical touch has
`not been adopted for small handheld touch
`screens (such as in cell phones and PDAs) due
`to anumberof other technical reasons, includ-
`
`ing power consumption, mechanical packag-
`ing constraints, and resolution limitations
`whichlimit the system’s ability to detect
`small objects such as PDA-style pens.
`Because of their much lowercost, other tech-
`
`nologies such as analog-resistive technology
`have dominated the mobile-device touch-
`screen market.
`
`However,certain features of optical touch
`remain desirable and representattributes of
`the ideal touch screen, including the option to
`eliminate the glass or plastic overlay that most
`other touch technologies require in front of
`the display. In many cases, this overlay is
`coated with an electrically conducting trans-
`parent material such as indium tin oxide
`(ITO), which reducesthe optical quality of the
`display. This advantage of optical touch
`screens is extremely important for many
`device and display vendors because devices
`are often sold on the perceived quality of the
`user display experience.
`Anotherfeature of optical touch which has
`been long desired is the digital nature of the
`sensor output when compared to many other
`touch systemsthat rely on analog-signal
`processing to determine a touch position.
`These competing analog systems normally
`require continual re-calibration, have complex
`signal-processing demands (which adds
`cost and power consumption), demonstrate
`reduced accuracy and precision compared to
`a digital system, and have longer-term
`system-failure modes due to the operating
`environment.
`
`Yet another key advantage of optical touch
`is that there is normally no direct impact of a
`finger, pen, or other object with the touch-
`recognition hardware. This reducesthe possi-
`bility of failure modes typically caused by
`impactfailure, wear, or fatigue of the touch
`
`screen. Thisis also related to the requirement
`for low-pressure touch. In an optical-touch
`system, only interaction with the light beams
`is required — no force needs to be applied to
`the system for detection or activation.
`Finally, optical touch is capable of imple-
`menting multi-touch, something most other
`touch technologies cannoteasily achieve.
`Although multi-touch has not been widely
`deployedin the past, there has recently been
`renewedinterest in it, driven by new devices
`such as the Apple iPhone that make multi-
`touch an integral part of the user interface.
`
`Recent Technology Enhancements
`New Components and Improved Signal
`Processing: Since conventional optical-touch
`systems were first developed, key components
`such as LEDs, photodiodes, and CMOSchips
`have improved considerably in performance
`and reduceddrastically in cost. Technologies
`
`used to produce molded optics and algorithms
`for signal processing have also been devel-
`oped and improved. Asa result, conventional
`optical-touch technology has improved and
`hasat least maintained its competitive posi-
`tion against other touch technologies, which
`are also undergoing continuous improvement.
`Improved Optical System Design. More
`recently, companies such as Elo TouchSystems
`and IRTouchhaveattemptedto solve the
`backgroundor ambient-light issue for optical
`touch, primarily by using a combination of
`improvedbezel design (aperturing), optical
`filtering, and more sophisticated signal pro-
`cessing to enhancethe signal-to-noiseratio.
`Asan example of the latter, the infrared LEDs
`can be modulated with a specific frequency,
`while output of the photosensors can be
`demodulated at only that frequency, thus
`reducing the impact of the unmodulated
`
`infrared light in sunlight. The latest product
`
`Fig. 2: The Apple iPhone with projected-capacitive touch(left) and the Neonode N2cell phone
`with conventional optical touch(right). Photo courtesy of PenComputing.com.
`
`Information Display 12/07 27
`CARDWAREEXHIBIT 2018, Page 2 of 5
`
`

`

`
`optical-touch technology
`
`Optical
`Sensors
`
`i
`
`Glass Substrate
`
`Sensor
`Views
`
`lIluminated Borders
`
`ence for multimedia functionality). For exam-
`ple, consider the Neonode N2 and Apple
`iPhone shownin Fig. 2. It is immediately
`obviousthat the iPhone’s screen surface is
`flush while the N2’s screen surface is
`recessed. Based on the measurementof a
`
`sample product, the bezel height on the N2 is
`about 1.6 mm (including the thickness of the
`housing material) while the iPhonehas a bezel
`height of zero (flush). Other issues that may
`hinder Neonode’s touch-screen technology in
`the cell-phone market are cost and power con-
`sumption, both the result of the high number
`of optoelectronic components (LEDs and
`photodiodes) in the device.
`Anotherpotential challenge for this tech-
`nology and also the Apple iPhoneis the limi-
`tation for use as finger-touch only. Asian
`manufacturers of smart phonesprefer to
`include stylus-based touch inputs to support
`character recognition. The light-beam spacing
`on the Neonode N2is relatively wide at about
`2.5 beamsper centimeter, so that a finger
`touch covers aboutnine light-beam intersec-
`tions. This conserves power but makesthe
`touch screen unusable with a stylus. Even if a
`large stylus were used, handwriting recogni-
`tion wouldstill be impossible due to insuffi-
`cient resolution. For comparison, the conduc-
`tive-trace spacing on the iPhoneisrelatively
`narrow at about seven traces per centimeter,
`so that a finger touch covers about 25 trace
`intersections. However, even with its higher
`resolution, projected-capacitive technology
`inherently supports only finger touch, which
`eliminates use with a stylus or even with
`gloves, so the comparison is moot.
`
`NextWindow, SMART Technologies,
`and Others
`
`Camera-based optical touch has been imple-
`mented by NextWindow, SMARTTechnolo-
`gies, and atleast one newstartup.
`NextWindow’s optical-touch-screen tech-
`nology uses two line-scanning cameras
`(Fig. 3) located at adjacent corners of a dis-
`play. The cameras track the movementof any
`object close to the surface by detecting the
`interruption of an infrared light source. The
`light is emitted in a plane across the surface of
`the screen andis reflected back at the cameras
`
`by retro-reflecting strips located along three
`edges of the screen. (Retro-reflectors reflect
`light back alonga path thatis parallel to but
`opposite in direction from the angle of inci-
`dence.) Whena finger (or any object) touches
`
`Fig. 3: A schematic representation ofNextWindow’s camera-based optical-touch technology.
`Illustration courtesy ofNextWindow.
`
`specifications of manufacturers allow maxi-
`mum ambientlight in the range of 75-100
`kx, indicating that these techniques have
`been largely successful in eliminating optical
`touch’s sensitivity to sunlight.
`
`tion of defined light beams, and those that use
`complex signal processing to determine touch
`position from an image of the space abovethe
`display. The remainderof this article exam-
`ines these new optical-touch systems.
`
`New Typesof Optical-Touch Systems
`New componenttechnologies and reduced
`costs in key components have enabled the
`recent emergence of a numberofentirely
`new optical-touch systems. Combined with
`cheaper and more-sophisticated optical-
`system design tools, this creates the perfect
`conditions for the currenttotal re-examination
`
`of how optical-touch systems are designed
`and manufactured.
`
`There are two broad types of new optical-
`touch systems: those that rely on a light
`source to provide the light, whichis inter-
`rupted to detect touch, and those that use
`ambientlight and do not incorporatea light
`source. Additionally, these new systems can
`be divided into those that rely on an interrup-
`
`Neonode
`Neonodehastaken conventional IR touch
`
`technology, using LEDsand photodiodes, and
`essentially miniaturized it for use in handheld
`devices. In addition to using the technology
`in its own N2cell phone, Neonodeis also
`marketingit to other device makers. How-
`ever, it is currently unknown whetherthis
`technology is being adopted by any other
`cell-phone vendors. The key challenge for
`this technologylikely will be the high bezel
`height. Many cell-phone manufacturers are
`continually trying to create devices that are
`flush or near-flush on the top cover, and they
`also prefer displays that extend as close to the
`side edges of the device as possible (in order
`to maximize both the display size and experi-
`
`28
`
`Information Display 12/07
`
`CARDWAREEXHIBIT 2018, Page 3 of 5
`
`

`

`the screen, the controller analyzes the images
`from the cameras andtriangulates the position
`of the touching object. SMART Technolo-
`gies’ optical-touch-screen technology does
`essentially the same thing, except that it uses
`four area-scanning cameras.
`Whileit is technically possible to use an
`optical-touch-screen technology without a
`glass touch surface, neither supplier does so
`because of the need to protect the soft (2H)
`surface of the LCD. These technologies
`represent an advanceovertraditional optical-
`touch screens because they have feweractive
`components and, therefore, should be lower
`cost and have a longer mean time between
`failures (MTBF). NextWindow marketsits
`touch screensin sizes ranging from 12 to
`120 in.; most applications to date are found
`on monitor-sized displays (such as in the HP
`TouchSmart “family computer’) and large
`displays used in interactive digital-signage
`applications.’ Although the technology has a
`sufficiently high resolution and data rate to
`support handwriting recognition with a sty-
`lus, it is unlikely to be offered for displays
`below about 10 in. where palm rejection is
`not needed due to border width, cost, and
`power-consumption concerns. In general,
`camera-basedoptical touch is unlikely to be
`used in mobile devices in the near future.
`
`Perceptive Pixel
`Researchers at New York University have
`developed a new approachto large multi-
`touch screens that can detect 10, 20, or even
`more fingers. A new company,Perceptive
`Pixel, has been formed to commercialize the
`technology — althoughit has yet to take place
`— in applications ranging from interactive
`whiteboards to touch-screen tables and digital
`walls, any of which could be manipulated by
`more than just one person.
`Perceptive Pixel’s technology works by
`introducing IR LEDlight onto a glass or
`plastic rear-projection screen. The technology
`uses frustrated total internal reflection (FTIR),
`which means that touch is detected when a
`
`finger touches the glass surface andlightis
`scattered away from the finger and detected
`by a photosensor normal to the glass surface.”
`In Perceptive Pixel’s implementation, the
`photosensoris a camera located nextto the
`projector (see Fig. 4). Because the tech-
`nology is designed to be used only with rear-
`projection displays, it is not applicable to
`mobile devices.
`
`TotalInternal
`
`Acrylic Pane
`
`Scattered Light
`
`i Video Camera
`implemented in low-power mobile devices.
`
`Fig. 4: A schematic representation of Perceptive Pixel’s optical-touch technologybased on
`frustrated total internal reflection. Illustration courtesy of Perceptive Pixel.
`
`130 frames per second (fps) of touch recogni-
`tion in order to avoid perceived user lag. This
`type of processing speed may be a challenge
`for image-array-based touch technologies
`
`Sharp, Toshiba Matsushita Display
`(TMD), and Others
`Sharp, TMD,and LG.Philips LCD recently
`have all demonstrated optical-imaging touch
`systemsthat use the display itself as an
`optical-sensing element. These new LCDs
`integrate a light-sensing element (photodiode
`or phototransistor) into each LCDpixel,
`whichallowsthe display to act as a large-
`array photosensor; with appropriate image-
`analysis techniques, it can act as a touch
`sensor or even a card scanner. A recent
`
`demonstration by Sharp includeda 3.5-in.
`LCDwith a photosensorresolution of
`320 x 480 pixels and a scanningrate of about
`1 sec for the entire display. Because this is an
`inherently digital technology,it has the capac-
`ity to recognize multi-touch events (Fig. 5).
`Asa touch-screen technology for mobile
`devices, one challenge for this technology
`will be signal processing undera variety of
`ambient-light conditions. Unlike simple touch
`screens, this technology requires a complex
`imageto be analyzed to determine if a touch
`event has occurred. This requires more
`sophisticated, expensive, and power-hungry
`processors compared with simple touch
`screens. In addition, varying background-
`lighting conditions further complicate the
`image analysis. Another concern is speed.
`For example,it is often quoted that hand-
`writing detection requires a minimum of
`
`Fig. 5: Sharp’s 3.5-in. LCD with in-cell light-
`sensing optical touch. Photo courtesy
`of Nikkei Business Publications Tech-On!
`
`Information Display 12/07 29
`CARDWAREEXHIBIT 2018, Page 4 of 5
`
`

`

`
`optical-touch technology
`
`Light Source i: :
`
`Transmit Side
`Waveguides
`
`(ASIC)
`
`Receive Side
`Waveguides
`
`Light
`Detector
`
`design used by RPOis quite sophisticated, but
`has resulted in a physical system that is simple
`and cheap to assemble.
`This system was demonstrated by RPOat
`Display Week 2007, where multi-touch built
`into a PDA device was shown. DWT is now
`
`being placed into products in cooperation with
`a numberof lead customers. In principle, this
`system can be utilized for a display of any
`size, but RPOis initially targeting small—to—
`medium-sized displays in consumerelectron-
`ics and automotive applications.
`
`Conclusion
`
`It is likely that each of these new optical-
`touch technologies will address niches of
`the large and growing touch-screen market.
`Barring any technology shortcomings, we
`expect that these optical-touch technologies
`will provide key benefits over other compet-
`ing touch technologies. Together, these new
`technologies, if grouped together as “optical-
`touch systems,” could ultimately capture a
`large share of the total touch-screen market.
`Ofparticularinterest is the sudden growth
`of touch screens in handheld devices, driven
`by the Apple iPhone, other smart phones, GPS
`handheld devices, and personal media players.
`Ofthe above technologies, Neonode, Sharp,
`TMD, and RPOareexplicitly targeting this
`space and wish to compete with the incum-
`bent resistive and projected-capacitive touch
`technologies.
`
`References
`‘Introducing the NextWindow 1900 Optical
`Touch Screen: A NextWindow White Paper”
`(2007).
`2]. Han, “Low-Cost Multi-Touch Sensing
`through Frustrated Total Internal Reflection,”
`Proceedings of the 18th Annual ACM Sympo-
`sium on User Interface Software and Technol-
`ogy (UIST), (2005) (ACM 1-59593-023-
`X/05/0010).
`
`Fig. 6: A schematic representation of RPO’s Digital Waveguide Touch.
`
`The displays used in mobile devices vary
`greatly in size, aspect ratio, and resolution,
`with noreal standardization by manufacturers.
`Therefore, in making a sensorbuilt into each
`display, high costs associated with the design
`and NRE ofthe more complex touch-sensing
`LCDsare to be expected. In addition, these
`LCDsprobably have lowerpixel-aperture
`ratios, and hence maynotbe as bright as simi-
`lar displays without the touch sensorbuilt in.
`
`RPO’s Digital Waveguide Touch™
`RPO’s Digital Waveguide Touch (DWT)™ is
`an optical-touch system that expands on the
`basic concept ofthe traditional IR system.
`This system uses one or two low-cost LEDs
`to provide a managedlight source (effectively
`a planar sheet of IR light) projected from two
`adjacent bezel edges, and then utilizes poly-
`meroptical waveguidesat the other two adja-
`cent bezel edges to channel the light into
`separate 10-um channels leading to a small
`photosensorarray (Fig. 6).
`This improvement on conventional IR
`touch effectively addresses all knownshort-
`comingsof thelatter as follows.
`Becausethe positioning of the optoelec-
`tronic components (LED andsensor) has been
`decoupled from the bezel of the display, the
`impact of the touch system on the bezel height
`and width is greatly reduced compared to
`conventionaloptical-touch systems. RPO has
`
`30
`
`Information Display 12/07
`
`working demonstrators with only 2 mm of
`touch system outside ofthe active area of the
`display and a profile height of only 0.5 mm
`from the top of the protective display cover
`(lens) to the inside surface of the device’s
`housing.
`DWT has a much lower componentcost
`due to having only one or two LEDsand one
`photosensor chip and a much higherresolu-
`tion due to the “digitization” of the receive-
`side optical signal into separate optical
`waveguide channels, which are independently
`detected by individualpixels at the photo-
`sensor array. Hence, pen detection and
`handwriting recognition are possible.
`Ambient-light conditions are not an issue
`due to the use of new approachesto filtering
`and aperturing and becauseofthe small size
`of the optical waveguide receiving channels.
`The key enabler for this technologyis the
`availability of low-cost lithographically
`printed polymeroptical waveguides devel-
`oped by RPO. The company uses LCD-like
`processtools to deposit wet films and
`processesthesefilms by using direct photo-
`patterning, followed by solvent development.
`While it sounds very simple,it took many
`years of developing the polymer materials and
`the manufacturing processin orderto attain
`waveguidesof sufficiently high resolution that
`also meetall the requirements of yield and
`durability. In addition, the optical system
`
`CARDWAREEXHIBIT 2018, Page 5 of 5
`
`