throbber
Embedded Touch Terminology...1
`
`«to Key defining characteristic
`
`+ Touch capability is provided by a display manufacturer
`instead of a touch-module manufacturer
`
`0 Touch-module manufacturers can't do in-cell or on-cell
`
`«:0 Marketing Terminology Alert!
`
`+ Some display manufacturers call all their embedded touch “in-cel|”,
`even though they may be supplying hybrid or on-cell
`
`+ Some display manufacturers use a brand name to encompass all
`their embedded touch products
`
`0 For example, “Touch On Display” from lnnolux
`
`+ Some display manufacturers direct-bond or air-bond an external
`touchscreen to their display and call it “out-ce|l”
`
`SID DISPLAY WEEK ‘14
`
`intel
`
`SAMSUNG EXHIBIT 1011 (Part 2 of 3)
`
`

`
`Embedded Touch Terminology...2
`
`Touch sensor is physically inside the LCD cell
`Touch sensor can be:
`
`0 Capacitive electrodes (same as p-cap)
`
`o Light-sensing elements (rare)
`
`On-Cell Touch sensor is on top of the color-filter glass
`(LCD) or the encapsulation glass (OLED)
`
`o Capacitive electrodes (same as p-cap)
`
`Touch sensor has sense electrodes on top of the
`Hybrid
`(In-Celll color-filter glass 1 drive electrodes inside
`On-Cell) the cell
`
`0 IPS LCD: Segmented Vcom electrodes on
`the TFT glass
`
`0 Non-IPS LCD: Segmented Vcom electrodes
`on the underside of the color filter glass
`
`SID DISPLAY WEEK ‘14
`
`
`
`

`
`Early Embedded Methods All Failed
`
`oz» Attempts to develop embedded touch in 2003-2011
`
`were all trying to invent something new while
`leveraging the LCD design
`
`+ “Pressed” capacitive, first mass-produced by Samsung in 2009
`
`4 Light-sensing, first mass-produced by Sharp in 2009
`
`4 Voltage-sensing (“digital switching”), first mass-produced by
`Samsung
`
`«to But none of them was really successful
`
`+ Insufficient signal-to-noise ratio for robust operation
`
`+ The need to press the display surface, which prevented the
`use of a protective cover-glass
`
`+ The unreliability of pressing the display very close to the frame,
`where the color-filter glass has little ability to move
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`First Successful Embedded Touch:
`
`OLED On-Cell P-Cap
`
`«:0 Samsung S8500 Wave mobile
`phone with Super AMOLED on-cell
`p-cap touch (Feb. 2010)
`4 3.3-inch 800x480 (283 ppi) AM-OLED
`4 “Super AMOLED” is Samsung’s
`
`(odd) branding for on-cell touch
`4 Sunlight readable
`0 AR coating & no touchscreen overlay
`
`On Ce
`(5“Pe'.' AM
`- r ~
`:
`
`Window=
`direct-bond
`
`cover—glass
`
`_
`.....s.,...;
`2
`3
`'i”°'§@'
`
`.
`
`Mobile World Congress 2010
`
`Source: Samsung
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`On-Cell P-Cap
`
`Cover Glass ("Lens")
`Decoration
`
`Top Polarizer
`
`Metal Bridges
`1 Insulator
`
`Sense 8. Drive Electrodes (ITO)
`Color Filter Glass
`
`Color Filter
`
`Liquid Crystal
`TFTs
`
`TFT Array Glass
`
`«:0 Principle
`
`Source: The author
`
`§ ITO P-cap electrode array is deposited on top of the color filter
`glass (under the top polarizer)
`
`o Exactly the same function as discrete (standalone) p-cap
`
`o Shown above is one ITO layer with bridges; it could also be
`two layers with a dielectric instead
`
`SID DISPLAY WEEK ‘l4
`
`114
`
`
`
`

`
`The Display-Makers
`Quickly Got the Idea
`
`°:~ Don’t try to invent something new; figure out
`how to apply what already works (p-cap)!
`
`otv The result: Sony’s (JDI) “Pixel Eyes” hybrid
`in-celllon-cell mutual capacitive
`0 First successful high-volume embedded touch in LCD
`
`Segmented anti-static shield
`on color-filter glass
`(sense electrodes)
`
`VCOM electrodes ’
`on TFTg|ass
`(drive electrodes)
`
`'
`
`"
`
`M
`
`‘
`
`SID DISPLAY WEEK ‘14
`
`115
`
`Source: Japan Display; annotation by the author
`
`
`
`

`
`First Phones with Hybrid ln-Ce||/
`On-Cell Mutual-Capacitive (May 2012)
`
`«to Sony Xperia P and HTC EVO Design 4G (not the iPhone 5)
`
`oz» Similar LCDs
`
`# 4-inch 960x540
`
`LTPS (275 ppi) with
`different pixel arrays
`
`~:~ Same touch solution
`
`4 Synaptics
`C|earPad 3250
`
`(four touches)
`
`«to <100 um thinner than
`
`one-glass solution!
`
`Source: Sony
`
`Source: HTC
`
`SID DISPLAY WEEK ‘14
`
`intel‘
`
`
`
`

`
`Apple iPhone 5: First Fully In-Cell
`Mutual Capacitive (Sept. 2012)
`
`03° Structure
`
`+ Both sense and drive electrodes are in the TFT array, created by
`switching existing traces so they become multi-functional
`
`4 Apple has said they may change
`to Innolux “Touch On Display”
`(TOD, |nnolux’s brand name for ALL
`of their embedded touch structures)
`in iPhone 6
`
`0 That doesn't actually tell us
`anything, since TOD includes
`
`all three embedded structures...
`
`SID DISPLAY WEEK ‘l4
`
`Source: CNET
`
`
`
`

`
`Apple’s iPhone-5 Electrode Structure
`
`a
`
`éé
`
`Suhpnalc =
`
`1
`
`tr;tHw».& -~
`
`ULU __UUU
`
`‘>
`
`I UUU II ULM
`
`Source: BOE Technology Group's Central Research Institute
`
`SID DISPLAY WEEK ‘14
`
`
`
`

`
`Other In-Cell Electrode Structures
`
`(Based On Patents)
`
`«:0 Apple & Samsung
`
`+ Drive electrodes are segmented VCOM
`
`+ Sense electrodes are metal overlaid on the CF black matrix
`
`«to Apple & Samsung
`
`+ Drive electrodes are ITO stripes deposited on top of a dielectric
`layer over the color filter material
`
`+ Sense electrodes as above
`
`~:~ Sharp
`
`+ Both drive & sense electrodes are deposited on the bare CF-glass,
`before the black matrix and color-filter material are applied
`
`«to LG Displays
`
`4 Self-capacitive method using just segmented VCOM
`
`SID DISPLAY WEEK ‘14
`
`
`
`

`
`Summary of Sensor Locations
`
`Discrete sensor
`(separate glass)
`
`To of cover— lass
`
`Bottom of cover—glass
`(OGS = G2)
`
`lndustry standard
`Glass or PET
`Easy to add shield layer
`Display unconstrained
`
`Good for sensing
`Widest sensing area
`Disla unconstrained
`
`Thickness & weight
`
`lmractical
`
`Complex lens (yield)
`Limited durability
`
`To of olarizer
`
`lmractical
`
`(1 or 2 layers)
`
`Lower cost (1 layer)
`
`Limited to display size
`
`Both sides of CF glass
`(hybrid for non-IPS)
`
`Slightly thinner
`Slightly lower cost
`
`Top of CF glass and
`in TFT array
`h brid for IPS
`
`Highest performance
`Slightly thinner
`Sli htl
`lower cost
`
`2-sided CF process
`Limited to display size
`Re uires disla interation
`
`2-sided CF process
`Limited to display size
`Re uires disla interation
`
`In cell (on TFT array
`for IPS; split between
`TFT and CF for non—lPS
`
`High performance
`Thinnest
`Potentiall
`
`lowest cost
`
`Limited to display size
`Requires display integration
`Com lex desi n
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Integrating the Touch Controller
`and the Display Driver lC...1
`
`Host Processor
`
`and Display
`Accelerator
`
`SID DISPLAY WEEK ‘14
`
`
`
`

`
`Integrating the Touch Controller
`and the Display Driver lC...2
`
`«:0 Advantages
`
`+ Full synchronization of touch and DDI
`
`+ Can work with any sensor (discrete, OGS, on-cell, in-cell, hybrid)
`
`+ Reduced latency
`
`0 70 ms to 20 ms
`
`4 Capable of user-input and feedback without CPU involvement
`
`0 Done by programming the display configuration blocks of flash memory
`
`a Overlay capability plus image fade-in/out, animation, translation, etc.
`
`+ Can support wake-on-touch
`
`o Can display sprites or graphics for log-in screen
`
`»:« Disadvantages
`
`# Design is LCD-specific (resolution & pixel layout)
`
`+ Substantial NRE; appropriate only for high-volume
`
`SID DISPLAY WEEK ‘14
`
`
`
`

`
`Comparison of Discrete (e.g., OGS)
`Touch with Embedded Touch...1
`
`«to Cost: is embedded touch really “free”? N0!
`
`+ Barrier to entry
`
`c There is much more intellectual property (IP) on embedded touch
`layer-structure & driving; making sure you don’t infringe costs money
`
`+ Development cost
`
`a Embedded touch is much more complex to develop than OGS
`
`a High volume is required (5M) to make it practical
`
`§ Cover glass, decoration & bonding
`
`0 Similar to discrete (OGS), but embedded cover-glass is just
`glass & decoration (no ITO), so it's easier to manufacture
`
`a Sheet-type OGS may not be as strong as plain cover-glass
`
`+ Touch controller
`
`0 No integration = same cost (but performance is poor)
`
`a Linked to TCON for timing control = same cost (slightly different chip)
`
`o Integrated with TCON = saves $1-$2 in material cost
`
`SID DISPLAY WEEK '14
`
`inte|'
`
`
`
`

`
`Comparison of Discrete (e.g., OGS)
`Touch with Embedded Touch...2
`
`°:~ Cost (continued)
`
`+ FPC to connect electrodes
`
`0 On-cell and hybrid = same
`
`0 In-cell = none if touch controller is COG; saves another $1-$2
`
`4 Electrode material
`
`0 Discrete OGS currently uses ITO; could move to printed metal-mesh,
`which could save $10+ in tablet size (once sensor competition gets real)
`
`0 On-cell = same as discrete ITO
`
`0 Hybrid = only half as much added ITO (little material cost-difference)
`
`o In-cell = no added ITO
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Comparison of Discrete (e.g., OGS)
`Touch with Embedded Touch...3
`
`»:~ Performance
`
`+ On-cell = same as discrete or worse
`
`0 If you build the color-filter first (focus on LCD yield) then
`you can’t use high-temperature ITO so touch performance is worse
`
`0 If you build the touch electrodes first for good performance, then
`
`you can’t thin the color-filter glass
`
`+ Hybrid = same
`
`+ In-cell = worse, but should improve to be same as SNR goes up
`
`oz» Thickness
`
`0 Embedded is typically 100 um thinner than discrete OGS
`
`+ But the thickness variation between smartphone models with
`embedded touch is ~1.0 mm due to other features, so 0.1 mm
`
`doesn't mean that much to the consumer (it’s mostly marketing!)
`
`SID DISPLAY WEEK ‘14
`
`intel"
`
`
`
`

`
`Comparison of Discrete (e.g., OGS)
`Touch with Embedded Touch...4
`
`«:0 Weight
`
`+ Embedded = discrete (same number of sheets of glass)
`
`~:~ Power consumption
`
`+ On-cell & hybrid = same as discrete
`
`+ In-cell with integrated touch & TCON = probably lower, but touch
`power consumption is much lower than LCD power—consumption,
`so the decrease isn’t very significant
`
`«:0 Off-screen icons
`
`+ Discrete = no problem
`
`+ Embedded = eguires additional circuitry
`
`SID DISPLAY WEEK ‘14
`
`
`
`

`
`Embedded Touch Conc|usions...1
`
`oz» Embedded touch isn’t a clear win in either cost or
`
`technology; it’s all about who gets the touch revenue!
`
`oz» The driving force in embedded touch is the display-
`makers’ need to add value in order to increase their
`
`profitability
`
`°:~ Embedded touch provides little advantage to the
`end-user (consumer)
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Embedded Touch Conc|usions...2
`
`0:0 It’s not clear that embedded touch will offer significant
`
`cost-savings to the device OEM, since OGS can be
`further cost-reduced with ITO-replacement materials
`
`0:0 The display-makers will take some market share
`
`with embedded touch in high-volume products
`
`(Displaysearch says 25% in 2018) but embedded
`touch is unlikely to become dominant because the
`touch-panel makers won’t let their business be
`destroyed
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Large-Format P-Cap
`
`03° Introduction
`
`03° ITO Electrodes
`
`03° Wire Electrodes
`
`~'° Metal Mesh Electrodes
`
`6. Applications
`
`SID DISPLAY WEEK ‘14
`
`
`
`

`
`Introduction
`
`oz» Large-format touch is a much more wide-open space
`than consumer-electronics touch
`
`§ Multi-touch infrared (IR) has replaced traditional (single-touch) IR
`
`+ Camera-based optical has dropped substantially with the
`exit of NextWindow (SMART Technologies) from the market
`
`+ Startup: Sentons is taking a new approach to bending-wave
`
`+ Startup: RAPT is taking a new approach to in-glass optical
`
`4 P-cap with metal mesh is a threat to all other large-format
`touch technologies
`
`o Commonality of user experience (UX) with the 3 billion p-cap units
`shipped since 2007 may be the driving force
`
`0 Cost and complexity (as always) are the impediment
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`ITO Electrodes
`
`~30 3M has managed to get ITO electrodes to work
`
`in a 46-inch display (larger than any other with ITO)
`
`+ They won't disclose their secret sauce
`T V
`
`SID DISPLAY WEEK ‘l4
`
`Source: Photo by Author
`
`
`
`

`
`Wire Electrodes...1
`
`03 One more sensor variation: 10-micron wires
`
`between two sheets of PET or glass
`
`+ Commonly used for large-format touchscreens
`
`+ Two main suppliers: Visual Planet & Zytronic, both in the UK
`
`9 floor-to-ceiling
`
`Visual Planet
`
`touchscreens in
`
`the University of
`Oregon Alumni
`
`SID D|SPLAY WEEK ‘l4
`
`132
`
`ifltel
`
`Source: The University of Oregon
`
`
`
`

`
`Wire Electrodes...2
`
`03° Zytronic’s new multi-touch large-format p-cap
`
`+ Previous Zytronic products were self-capacitive (2-touch max)
`
`o Binstead’s frequency-variation patent was the basis of sensing
`
`+ New product is mutual-capacitive with very dense electrode pattern
`
`a Traditional measurement of capacitance reduction caused by finger
`
`o ~1.5 mm electrode spacing in 6 mm x 6 mm cell
`
`> Density reduces visibility because the human visual system sees
`a more uniform contrast
`
`o 10-micron insulated copper wires allow crossover (“single layer”)
`
`>100’s Q/m at 10 pm
`
`0 Can be applied to glass or film (including curved surfaces)
`
`0 Initial controller handles all sizes up to 72”; 100”+ possible
`
`a Minimum 10 touches with palm rejection
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Wire E|ectrodes...3
`
`«:« Jeff Han from Perceptive Pixel (acquired by Microsoft
`
`in mid-2012) showed an 82” at CES 2012 (with active
`stylus) and a 72” at Digital Signage Expo (DSE) 2012
`
`§ Metal electrodes (not ITO) — although Jeff wou|dn’t talk about the
`electrode material or who is manufacturing the touchscreens
`
`Source: Photos by Author
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Wire E|ectrodes...4
`
`0:0 Both the 72” & 82” look much better than the
`
`traditional Zytronic zig-zag 10-micron wire pattern
`
`72" at DSE 2012
`
`Source: Photos by Author
`
`— oN U
`N‘
`
`J
`U)
`
`D '
`
`56
`
`in
`N
`
`SID D|SPLAY WEEK ‘14
`
`
`
`

`
`Metal-Mesh Electrodes
`
`03° “lnvisible” metal-mesh electrodes are the biggest
`threat & opportunity in large-format p-cap
`
`+ Many suppliers are working on this
`
`+ Few (if any) have made formal product announcements
`
`4 Display sizes of 42” to 55” are frequently mentioned
`
`0 There are significant challenges
`
`0 Total number of connections is large (~250 + ~150 = 400 for 55”)
`
`0 Multiple ganged controllers are required
`
`0 Longer electrodes means slower sensing (larger RC time-constant)
`
`o Much larger number of electrodes takes longer to sense
`
`0 Number of suppliers able to print on 1,200 mm web is limited
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Applications...1
`
`oz» Large-format multi-touch applications
`
`SID DISPLAY WEEK ‘l4
`
`Source:
`
`Zytronic
`
`
`
`

`
`App|ications...2
`
`~30 Applications for curved large-format touchscreens
`
`Kiosks
`Public Information
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Applications...3
`
`«to BUT, stepping back from a technology focus, is the
`large-format touch market likely to start shrinking?
`
`+ Interactive media walls — touch is very necessary
`
`a Mu|tiTaction makes the best vision-based touch today
`(author’s opinion)
`
`0 Point-of-information — touch still seems necessary
`
`+ Digital signage - interaction via smartphone
`
`+ Education — interaction via tablets (including multi-user!)
`
`4 TV — interaction via mobile & motion-based devices
`
`+ Horizontal home-gaming tables — will they ever exist?
`
`+ Other large-format app|ications??
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Stylus Technologies
`
`0:» History
`
`oto Use Cases
`
`»:~ Passive Stylus
`
`oto Electromagnetic Resonance (EMR) Stylus
`
`0:» Active P-Cap Stylus
`
`03° Prediction
`
`«to Other Active Stylus Technologies
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Stylus History...1
`
`«to Microsoft Tablet PCs, PDAs, and early smartphones
`(e.g., Trio) always had styli (1989 to 2007), so why are
`we so finger-focused now?
`
`0 Steve Jobs and the iPhone in 2007 — “Who needs a stylus?”
`
`9 Microsoft’s failure to make the stylus-based Tablet PC a success
`with consumers caused them to de-emphasize the stylus and
`focus on finger-touch in Windows 7; that has continued and
`become even stronger in Windows 8
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Stylus History...2
`
`03° Is the stylus coming back into the consumer space?
`
`YES!
`
`+ All the major p-cap controller suppliers support active & passive
`
`4 PC OEMs want to differentiate their products from Apple’s
`
`+ Legacy Windows software on a Win8 tablet needs a stylus
`
`+ Android (in Ice Cream Sandwich) supports stylus messages
`
`+ Samsung has shipped >15M Galaxy Notes in two sizes
`
`+ Consumption isn't enough; a stylus is great for creation
`
`Source: Almel
`
`SID DISPLAY WEEK ‘14
`
`
`
`

`
`Stylus Use-Cases...1
`
`oz» Taking notes (in both Windows and Android)
`# Notes are automatically converted into text in background; being
`able to search your “ink” notes is very powerful
`
`oz» Annotating documents
`4 Typically Office or PDF
`
`«:0 Quick sketches
`
`0 Typical whiteboard-type sketches
`
`oto Precision pointing device, e.g. with Windows 8 Desktop
`+ When you’re trying to select tiny Ul elements
`
`«to Artistic drawings
`+ |t’s unbelievable what a real artist can do...
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Stylus Use Cases...2
`
`t
`.am-m-«
`23232211?
`lfluouszotnau mum
`3”“-M
`3 oaolnn mun Lynn
`_3oms2naa-ugmn
`
`_
`
`‘
`
`'
`
`%
`
`,
`V;
`I
`t
`“
`._ _
`‘I ,
`
`..
`
`;;
`'
`
`‘
`
`3
`
`--
`
`Created with
`
`an N-Trig active
`stylus on a
`Fujitsu Lifebook
`using ArtRage
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Passive Stylus...1
`
`03° A passive stylus can be any conductive object
`
`+ Metal rod
`
`+ Conductive plastic
`
`+ Ballpoint pen
`
`+ #2 pencil (shown at CES 2014)
`
`0 Long fingernail
`
`4 And those horrible 7 mm conductive-rubber-tipped styli
`
`o Needed for backwards compatibility with early tablets with low SNR
`
`~:« Tip diameter
`
`+ State of the art is 1.5 to 2.0 mm
`
`0 Next generation is 1.0 mm
`
`+ Essentially every controller supplier supports this now
`but not many have made it out into shipping products yet
`
`SID DISPLAY WEEK ‘14
`
`
`
`

`
`Passive Stylus...2
`
`«:0 Advantages
`
`+ Extremely low cost
`
`+ Easily replaceable
`
`+ Can be made any size and comfort level by low-tech methods
`
`+ Improves as SNR increases
`
`«to Disadvantages
`
`# No hover that meets Microsoft’s specification
`
`+ There's no OS support (yet) for differentiating between
`finger & stylus
`
`+ No pressure-sensing, so art and handwriting aren't as good
`
`+ Resolution can't be better than a finger
`
`SID DISPLAY WEEK ‘14
`
`
`
`

`
`Electromagnetic Resonance
`(EMR) Stylus...1
`
`«:0 Key characteristics
`
`+ Second sensor under the LCD
`
`4 Batteryless electronic stylus
`
`Cover glass
`
`Sensor
`
`Acer TM100
`
`/(The first
`
`Microsoft
`
`Tablet PC
`
`convertible)
`
`SID DISPLAY WEEK ‘l4
`
`147
`
`ifltel
`
`Source: Wacom
`
`
`
`

`
`EMR Stylus...2
`
`Penabled
`
`UJBCOITT
`
`Pressure-sensitive
`
`capacitor (Cnp)
`
`Cordless pen
`
`without battery
`
`Pen equivalent circuit
`
`Sensor grid Schematic
`
`Transmitted RF_.-" "-._Received RF
`
`Many wires
`
`Serial/USB
`
`interface
`
`to host
`
`SID DISPLAY WEEK ‘l4
`
`III—II_
`III—III
`I
`III—III
`II
`
`
`
`

`
`EMR Stylus...3
`
`«:« Variations
`
`+ Sensor substrate (rigid FR4 vs. flexible 0.3 - 0.6 mm PET)
`+ Pen diameter (3.5 mm “PDA pen” to 14 mm “executive” pen)
`
`0:» Size range
`+ 2” to 14"
`
`«to Controllers
`
`4 Proprietary
`
`Controller for 10.4”
`'3’ Advantages
`S°”'°"3"""°°'"
`A
`+ Very high resolution (1,000 dpi)
`4 Pen “hover” (mouseover = move cursor without clicking)
`+ Sensor is behind LCD = high durability & no optical degradation
`
`+ Batteryless, pressure-sensitive pen
`
`__
`r§3§’.i‘°$§"n‘S.
`
`T
`
`T
`
`H
`
`T
`
`T
`
`T
`
`Single controller can
`run both pen digitizer
`& p-cap finger touch
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`EMR Stylus...4
`
`oz» Disadvantages
`# Electronic pen = disables product if lost; relatively expensive
`+ Difficult integration requires lots of shielding in mobile computer
`4 Sensor can’t be integrated with some LCDs
`+ Single-source for mobile CE devices (Wacom) = relatively high cost
`
`«to Applications
`+ Phablets and tablets
`
`# E-book readers
`
`0 Opaque desktop graphics tablets
`+ Integrated tablet (pen) monitors
`
`«to Suppliers
`
`+ Wacom, Hanvon, Waltop,
`UC-Logic/Sunrex, KYE
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`EMR Stylus...5
`
`do Samsung Galaxy Note sketching demo at CES 2012
`
`The Galaxy
`Notes use
`
`both a
`
`P‘°aP
`
`touchscreen
`
`AND a
`
`Wacom
`
`EMR
`
`stylus
`(2 sensors!)
`
`SID DISPLAY WEEK ‘l4
`
`Source: Photos by Author
`
`
`
`

`
`Active P-Cap StyIus...1
`
`3) Stylus LF transmission
`containing:
`- TIP Pressure level
`- Buttons status
`
`2) Every cycle:
`- Atrigs drivelreceive analog signal
`to/from the ITO grid lines
`Finger touching the sensor affect
`signal level, allowing for touch
`detection
`
`-
`
`Atrigs also pick-up Slylus signal
`and determine its location based
`
`on signal level distribution across
`the ITO lines
`
`Source: N—Tn'g
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Active P-Cap Sty|us...2
`
`03- Variations
`
`+ One-way digital RF transmission from stylus to p-cap sensor,
`with both sense & drive electrodes acting as antennas
`o N-Trig has by far the most-developed user experience
`
`4 Two-way transmission between stylus and p-cap sensor
`o Stylus receives p-cap sensor drive-signal, amplifies it, adds digitally
`
`encoded stylus information, and transmits it back to sensor
`0 Atmel was the first to put this into production, but their user
`experience is still very immature
`
`+ Stylus generates intense e-field at tip
`a E-field Q capacitance to p-cap sensor operating as usual
`(finger subtracts capacitance)
`
`o Unclear if anyone is actually doing this...
`
`SID DISPLAY WEEK ‘14
`
`
`
`

`
`Active P-Cap Sty|us...3
`
`»:~ Advantages
`+ Uses existing (single) p-cap sensor
`4 Pen “hover” (mouseover = move cursor without clicking)
`+ Stylus tip can be very small (< 1 mm)
`+ High resolution and accuracy
`
`«to Disadvantages
`+ Stylus requires power source (battery or super-capacitor),
`which requires charging contacts in stylus-garage and
`charging circuit in host computer
`+ Stylus technology is unique to each p-cap controller supplier
`c Total lack of interoperability will probably prevent active stylus
`from ever becoming mainstream
`
`o OEMs’ desire to obtain high margin on accessories makes the
`problem even worse
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Active vs. Passive Stylus Summary
`
`«z» This battle’s been going on since the 1990s...
`
`versus
`
`+ Very |ow—cost
`
`+ More expensive
`
`+ “Good enough”
`+ Improves as SNR increases
`
`+ #2 pencil is the gold standard
`+ “Artificial finger” in Windows
`
`+ More flexibility in Android
`
`+ Pressure-sensing
`4 Hover (required for Windows)
`
`4 Higher resolution
`+ Customizable features
`
`P—cap (powered)
`
`EMR (batteryless)
`
`+ N-Trig leads
`+ Wacom leads
`+ Others following
`+ Others insignificant
`+ N0 interoperability
`+ 2"“ sensor
`as
`Cost in high-volume is surprisingly close
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Prediction
`
`03° Passive stylus is going to win (become mainstream)
`
`+ Being “good enough” is very important in the touch industry!
`
`+ |t’s the lowest-cost solution
`
`+ However...
`
`0 There is still some chicken-and-egg regarding good support
`for stylus in application software
`
`a Some OEMs haven't bought into the need for a stylus yet
`(more chicken-and-egg)
`
`«to Active stylus will remain a niche
`
`+ Active stylus’ total lack of interoperability and very high
`price as a replacement accessory will prevent it from ever
`becoming mainstream
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Other Active-Stylus Technologies
`
`«to Combination ultrasonic & infrared
`
`# Used in many clip-on and clipboard-style digital note-taking
`accessories; also available for iPad
`
`0:. Embedded CMOS-camera stylus by Anoto
`+ Widely licensed for digital-pen note-taking accessories and
`form—fil|ing applications
`+ Used by LG Displays in large-format touch
`4 Used in Panasonic 4K 20" professional tablet shown at CES 2013
`
`«:0 Infrared LED light-pen
`+ Used by iDT| in their light-sensing in-cell touch monitor
`
`«:0 Visible laser-pointer
`4 Used by isiQiri in |arge—format touch
`+ Also works with iDT| light-sensing in-cell touch
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`92° Multi-Touch
`
`«:0 OS Application-Development Support
`
`«to Middleware
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Multi-Touch
`
`»:~ Multi-touch is defined as the ability to recognize
`two or more simultaneous touch points
`
`«to Multi-touch was invented in 1982 at the
`
`University of Toronto (not by Apple in 2007!)
`
`oz» “Pinching” gestures were first defined in 1983
`(not by Apple in 2007!)
`
`oz» Windows 7 (2009) & Windows 8 (2012) both support
`multi-touch throughout the OS and are architected to
`support an “unlimited” number (~100) of simultaneous
`touch points
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Multi-Touch Architecture
`
`’ Capable of decoding multiple
`
`streams of moving points and
`taking actions in response
`
`a
`
`Capable of delivering multiple
`
`streams of moving points (and
`acting on a defined subset of them)
`
`Capable of delivering sets of
`simultaneous points to the OS
`
`Capable of sensing multiple
`simultaneous points
`
`Source: The author
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Why Multi-Touch Has
`Become So lmportant...1
`
`«:0 Apple
`+ Apple established multi-touch as a “must-have” for coolness.
`The result is that people of all ages expect every display they
`see to be touchable with multiple fingers
`
`«:0 Gaming
`0 Gaming is a natural for multi-touch. Try playing air hockey
`without multi-touch. ..
`
`«to Unintended touches
`
`0 One of the major values of multi-touch is to allow the system
`to ignore unintended touches (“palm rejection”, “grip suppression”,
`etc.). As desktop screens become more horizontal (recline)
`this will become even more important.
`
`SID DISPLAY WEEK ‘14
`
`
`
`

`
`Why Multi-Touch Has
`Become So lmportant...2
`
`°:~ Multi-user collaboration
`
`+ When two people want to collaborate on a large screen (e.g.,
`a student and teacher on an interactive “whiteboard” LCD),
`multi-touch is essential
`
`0 Identifying which touch belongs to which user is still unsolved
`
`o It IS currently possible to uniquely identify multiple simultaneous styli
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`How Many Touches Are Enough?...1
`
`oz» The industry has multiple answers
`0 Microsoft settled for 5 touches in Win8 (they wanted 10)
`o But now under pressure from OEMs they have buckled and
`reduced it to TWO touches for All-in-One desktops (BIG mistake!)
`+ The p-cap touchscreen suppliers under 30” either say “10” or
`
`“as many as possible” (e.g., 3M’s p-cap supports 60+ touches)
`+ The large-format touchscreen suppliers say that 40 is enough
`
`~:~ In practice it depends on the hardware and
`
`controller firmware implementation
`
`0 Ideally the touchscreen should ignore all other touches beyond
`however many the product is guaranteeing
`
`+ This is usually called “palm rejection” and its implementation is
`
`absolutely critical to the user experience
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`How Many Touches Are Enough?...2
`
`»:~ The answer actually depends on the application
`
`+ For a small mobile device, 2-5 (one hand) are enough
`
`+ For a single-user app on El device (even an 82” screen),
`it’s hard to see why more than 10 (two hands) are needed
`
`+ For a multi-user app, it depends...
`o For a 55-inch gaming table, 40 (8 hands) is not unreasonable
`> The key touchscreen specification is probably response time (latency)
`
`o For a 65-inch interactive “whiteboard” LCD, 20 (4 hands) is
`probably enough, although an argument can be made for 40
`> BUT, the key touchscreen specifications are entirely different:
`minimum stylus tip size, pre-touch, jitter, ink-lag, etc., can all be critical
`
`From a video of a very
`cool muIti—pIayer game
`on the FIatFrog website
`
`Source: FIatFrog
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`#1 Reference On Multi-Touch
`
`»:~ “Mu|ti-Touch Systems that
`I Have Known and Loved”
`
`+ wvvw.bilIbuxton.com/mu|titouchOverview.html
`
`“If you can only manipulate one
`point
`you are restricted to the
`gestural vocabulary of a fruit fly.
`We were given multiple limbs
`for a reason. It is nice to be
`
`able to take advantage of them.”
`
`SID DISPLAY WEEK ‘l4
`
`Bill Buxton, 2008
`
`Principal Researcher,
`Microsoft Research
`
`
`
`

`
`For Windows, the “Logo”
`Is the Starting Point
`
`«:0 A set of touch performance standards designed
`to ensure a high-quality user experience
`
`+ 5 touch-point minimum
`+ Touchscreen jitter
`
`+ Extra input behavior
`+ High-resolution timestamp
`+ Input separation
`+ Noise suppression
`+ Physical input position
`+ Reporting rate
`+ Response latency
`4 Cold boot latency
`+ Touch resolution
`
`+ User experience
`4 Pre-touch
`
`+ Pen tests
`
`SID DISPLAY WEEK ‘l4
`
`Windows 8
`
`Compatible ]
`
`
`
`

`
`Windows 8 Touch
`
`I: Windows 8
`
`«to The Win8 Touch Logo specification is based on p-cap
`4 Win7 spec was based on optical, which had little relevance
`4 Win8 spec creates a common touch capability for mobile phones,
`tablets, notebooks, and desktops
`o This may be very significant for multi-platform applications!
`
`0:» Basic spec requirements
`4 Minimum of 5 simultaneous touches; must ignore an additional 5
`4 Tablets must be zero-bezel; otherwise 20 mm border minimum
`
`4 Respond to first touch in < 25 ms
`4 Subsequent touches must be < 15 ms at 100 Hz for all touches
`4 Better than 0.5 mm accuracy with < 2 mm offset from actual location
`4 No jitter when stationary; < 1 mm when moving 10 mm
`4 Pre-touch < 0.5 mm
`
`4 Finger separation >= 12 mm horizontal/vertical, 15 mm diagonal
`0 But on-screen keyboards and normal human behavior violates this!
`
`SID DISPLAY WEEK ‘l4
`
`intel
`
`
`
`

`
`Windows 8 Touch
`Application Development I=gWindows8
`
`«to There are multiple development environments
`commonly used in Windows 8, each of
`which handles touch differently
`
`+ Native C++ (Win32/COM)
`
`§ Managed environment (.NET Framework)
`
`§ Silverlight & WPF (Windows Presentation Foundation)
`
`+ Adobe Flash
`
`+ Modem (Win-8) using C# and XAML or HTML5 and JavaScript
`
`o Modern apps today only represent one aspect of business computing:
`
`reporting/dashboards, with moderate-to-light data updating
`
`0:0 From my perspective...
`
`+ As a hardware person, I find the level of detail required
`to do anything significant in touch software to be excruciating
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Android Touch
`Application Development
`
`"'
`
`«to Android has an extensive and growing API
`for touch & stylus
`
`+ I hear complaints about the degree of bugginess
`
`+ From what I can tell, the level of tediousness is a
`
`little better than Windows
`
`+ The Android API supports up to 256 touches, but the actual number
`depends on the hardware & firmware implementation
`in the device — 2 to 5 isn’t unusual
`
`+ Fragmentation of Android (different versions from each OEM)
`appears to make developing a robust run-on—anything Android
`touch application very difficult
`
`~:~ The language decision is easy — it’s Java or nothing
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`iOS Touch
`Application Development
`
`.
`IDS 7
`
`«:0 iOS seems to have the most constrained touch
`
`application development environment
`
`+ But it’s not any easier than Android -- in the chapter on touch in
`“Programming iOS 5" (an O’Rei||y book), the words “messy” and
`“tricky” seem to occur a lot
`
`~:~ The language decision is easy — it’s Objective-C
`or nothing
`
`SID DISPLAY WEEK ‘14
`
`
`
`

`
`Middleware...1
`
`(Consumer Electronics)
`
`oz» The best example of middleware in CE devices is
`from Myscript (formerly “Vision Objects”)
`
`+ This is what makes the Samsung Galaxy Notes possible
`
`+ Extremely powerful, configurable capabilities
`
`o Note-taking, handwriting recognition, mathematics (including
`equations), music notation, even “ink as a data—type” (same
`
`concept as in Windows, stores both ink and ASCII text)
`
`Ul: A thin layer of Samsung‘ look 8: feel
`
`Myscripti Mziddlew-are
`
`(CO_ntai'n-s most of the Notes’ functionality)
`
`Android
`
`Source: The author
`
`Samsung Galaxy Notes’ software stack
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Middleware...2
`(Large-Format I Commercial)
`
`«to The best middleware for large-format applications
`(in the author’s opinion) is Snowflake
`
`+ Good starting point for commercial applications
`
`+ Includes 30+ multi-touch apps (entertainment, presentation,
`creativity, media-browsing, etc.)
`
`+ Includes an SDK
`
`+ Runs on Win 8/7Nista/XP, Mac OS X Lion & Snow Leopard,
`and Linux Ubuntu
`
`«to Snowflake simplifies handling...
`
`4 Touch & gesture events, audio, video, images
`
`+ PDFs, 3D, on-screen keyboards, web browsing
`
`+ Multiple languages, QuickTIme integration, etc.
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Midd|eware...3
`
`0:0 Snowflake home screen
`1
`
`g-I
`
`@Eflw&%®%
`
`‘I
`
`3D ‘wzmir
`
`-"Hr Hr-nr-V f«lx».?r*: Ra ccrr.
`
`Bra-.-sf-0:-r
`
`E3-.143”, mars Colic-rt Conntedge
`
`Curling
`
`weuwmeea
`
`Frngqy
`
`(3-ooalc Maps
`
`‘£5;-(J. -,fl=..r: 1'
`
`$3.’.-‘:v.'*I3raw.I.J.'1 ‘.~’:de=o
`
`Urumfi
`
`SID DISPLAY WEEK ‘l4
`
`Source: NuiTeq
`
`
`
`

`
`Middleware...4
`
`«:0 Other alternative “middleware” for large-format
`
`+ Omnitapps
`
`o Less complete, Windows only, no SDK, more for product marketing
`
`+ Intuilab
`
`a Commercial multi-touch application platform with Kinect, RFID, etc.
`
`+ GestureWorks (ldeum)
`
`o Robust Flash multi-touch development environment
`
`4 22 Miles
`
`a Sales productivity application for iOS, Android, Windows & Mac
`
`+ Sotouch
`
`0 Application platform for wayfinding and presentations
`
`+ Fingertapps (Unlimited Realities)
`
`o Multi-touch demo software
`
`SID DISPLAY WEEK ‘14
`
`
`
`

`
`Conclusions
`
`«:0 Future Trends & Directions
`
`«to Suggested Reading on Touch
`
`0:0 Recommended Conferences & Trade Shows on Touch
`
`SID DISPLAY WEEK ‘l4
`
`
`
`

`
`Future Trends & Directions...1
`
`03° P-cap is here to stay
`
`+ It is totally dominating consumer electronics
`
`+ Consumer p-cap is getting much closer to meeting commercial
`application requirements
`
`0 For example, glove-touch and water-resistance
`
`+ P-cap’s capabilities are becoming increasingly attractive in
`commercial applications
`
`0 Curved touch-panels, particularly in automotive
`
`0 Light touch expected by ALL touch-panel users
`
`0 Flat-bezel in customer-facing applications
`
`0 Mu|ti—touch wher

This document is available on Docket Alarm but you must sign up to view it.


Or .

Accessing this document will incur an additional charge of $.

After purchase, you can access this document again without charge.

Accept $ Charge
throbber

Still Working On It

This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.

Give it another minute or two to complete, and then try the refresh button.

throbber

A few More Minutes ... Still Working

It can take up to 5 minutes for us to download a document if the court servers are running slowly.

Thank you for your continued patience.

This document could not be displayed.

We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.

Your account does not support viewing this document.

You need a Paid Account to view this document. Click here to change your account type.

Your account does not support viewing this document.

Set your membership status to view this document.

With a Docket Alarm membership, you'll get a whole lot more, including:

  • Up-to-date information for this case.
  • Email alerts whenever there is an update.
  • Full text search for other cases.
  • Get email alerts whenever a new case matches your search.

Become a Member

One Moment Please

The filing “” is large (MB) and is being downloaded.

Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.

Your document is on its way!

If you do not receive the document in five minutes, contact support at support@docketalarm.com.

Sealed Document

We are unable to display this document, it may be under a court ordered seal.

If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.


Access Government Site

We are redirecting you
to a mobile optimized page.





Document Unreadable or Corrupt

Refresh this Document
Go to the Docket

We are unable to display this document.

Refresh this Document
Go to the Docket