The Application of
`Programmable
`
`.
`
`ea
`
`Ws
`in Mobile
`
`VE LUGae CC EUea IPR2022-00602
`
`Once hy
`
`Apple EX1053 Page 1
`
`IPR2022-00602
`Apple EX1053 Page 1
`
`

`

`Copyright © 2002 by John Wiley & Sons, Ltd
`Baffins Lane, Chichester,
`West Sussex, PO19 1UD, England
`National 01243 779777
`International (+44) 1243 779777
`e-mail (for orders and customerservice enquiries): cs-books @wiley.co.uk
`Visit our Home Page on http://www.wiley.co.uk or http://www.wiley.com
`
`All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system, or
`transmitted, in any form or by any means,electronic, mechanical, photocopying, recording, scanning or
`otherwise, except underthe terms of the Copyright Designs and Patents Act 1988 or underthe terms of a
`licence issued by the Copyright Licensing Agency, 90 Tottenham Court Road, London, W1P SHE, UK,
`without the permission in writing of the Publisher, with the exception of any material supplied speci-
`fically for the purpose of being entered and executed on a computer system, for exclusive use by the
`purchaserof the publication.
`
`Neither the author(s) nor John Wiley & Sons Ltd accept any responsibility or liability for loss or damage
`occasioned to any person or property through using the material,
`instructions, methods or ideas
`containedherein,oracting or refraining from acting as a result of such use. The author(s) and Publisher
`expressly disclaim all
`implied warranties,
`including merchantability of fitness for any particular
`purpose.
`
`Designations used by companiesto distinguish their products are often claimed as trademarks.In all
`instances where John Wiley & Sons is aware ofa claim, the product namesappearin initial capital or
`capital letters. Readers, however,should contact the appropriate companies for more complete informa-
`tion regarding trademarksandregistration.
`
`Other Wiley Editorial Offices
`
`John Wiley & Sons, Inc., 605 Third Avenue,
`New York, NY 10158-0012, USA
`
`WILEY-VCH Verlag GmbH
`Pappelallee 3, D-69469 Weinheim, Germany
`
`John Wiley & Sons Australia Ltd, 33 Park Road, Milton,
`Queensland 4064, Australia
`
`John Wiley & Sons (Canada) Ltd, 22 Worcester Road
`Rexdale, Ontario, M9W 1L1, Canada
`
`John Wiley & Sons (Asia) Pte Ltd, 2 Clementi Loop #02-01,
`Jin Xing Distripark, Singapore 129809
`
`British Library Cataloguing in Publication Data
`
`A catalogue record for this book is available from the British Library
`ISBN 0471 48643 4
`
`Typeset in Times by Deerpark Publishing Services Ltd, Shannon,Ireland.
`Printed and boundin Great Britain by T. J. International Ltd, Padstow, Cornwall.
`
`This bookis printed on acid-free paper responsibly manufactured from sustainable forestry, in which at
`least twotrees are planted for each one used for paper production.
`
`IPR2022-00602
`Apple EX1053 Page 2
`
`IPR2022-00602
`Apple EX1053 Page 2
`
`

`

`
`
`
`
`XV
`
`OmAWNH
`
`Contents
`
`Biographies
`
`List of Contributors
`
`1 Introduction
`Edgar Auslander and Alan Gatherer
`1.1 It’s a Personal Matter
`1.2 The Super Phone?
`1.3 New Services
`1.4 The Curse and Opportunity of Moore’s Law
`1.5 The Book
`
`2 The History of DSP Based Architectures in Second Generation Cellular Handsets
`Alan Gatherer, Trudy Stetzler and Edgar Auslander
`2.1 Introduction
`2.2 A History of Cellular Standards and Wireless Handset Architectures
`2.2.1 1G and 2G Standards
`2.2.2 2.5G and 3G Standards
`2.2.3 Architecture Evolution
`2.3 Trends in Low Power DSPs
`2.3.1 Process Improvement
`2.3.2 Instruction Set Enhancement
`2.3.3 Power Management
`References
`
`3 The Role of Programmable DSPs in Dual Mode (2G and 3G) Handsets
`Chaitali Sengupta, Nicolas Veau, Sundararajan Sriram, Zhenguo Gu and Paul Folacci
`3.1 Introduction
`3.2 The Wireless Standards
`3.3 A Generic FDD DSDigital Baseband (DBB) Functional View
`3.4 Functional Description of a Dual-Mode System
`3.5 Complexity Analysis and HW/SW Partitioning
`3.5.1 2G/3G Digital Baseband Processing Optimized Partitioning
`3.6 Hardware Design Approaches
`3.6.1 Design Considerations: Centralized vs. Distributed Architectures
`3.6.2 The Coprocessor Approach
`3.6.3 Role of DSP in 2G and Dual-Mode
`3.7 Software Processing and Interface with Higher Layers
`3.8 Summary
`3.9 Abbreviations
`References
`
`oe sawentnnhd panne, eal te PERO NYTMage, OTA MTEL WB
`
`
`IPR2022-00602
`Apple EX1053 Page 3
`
`IPR2022-00602
`Apple EX1053 Page 3
`
`

`

`vi
`
`Cc
`
`ontents
`
`4 Programmable DSPs for 3G Base Station Modems
`Dale Hocevar, Pierre Bertrand, Eric Biscondi, Alan Gatherer, Frank Honore, Armelle Laine,
`Simon Morris, Sriram Sundararajan and Tod Wolf
`4.1 Introduction
`4.2 Overview of 3G Base Stations: Requirements
`4.2.1] Introduction
`4.2.2 General Requirements
`4.2.3 Fundamental CDMA BaseStation Base Band Processing
`4.2.4 Symbol-Rate (SR) Processing
`4.2.5 Chip-Rate (CR) Processing
`4.3 System Analysis
`4.3.1 SR Processing Analysis
`4.3.2 CR Processing Analysis
`4.4 Flexible Coprocessor Solutions
`4.4.1 Viterbi Convolutional Decoder Coprocessor
`4.4.2 Turbo Decoder Coprocessor
`4.4.3 Correlator Coprocessor
`4.5 Summary and Conclusions
`
`5 The Use of Programmable DSPsin Antenna Array Processing
`Matthew Bromberg and Donald R. Brown
`5.1 Introduction
`5.2 Antenna Array Signal Model
`5.3 Linear Beamforming Techniques
`5.3.1 MaximumLikelihood Derivation
`5.3.2 Least Mean Square Adaptation
`5.3.3 Least Squares Processing
`5.3.4 Blind Signal Adaptation
`5.3.5 Subspace Constraints
`5.3.6 Exploiting Cyclostationarity
`5.3.7 Transmit Beamformer Techniques
`5.4 Multiple Input Multiple Output (MIMO)Signal Extraction
`5.4.1 MIMOLinear System Model
`5.4.2 Capacity of MIMO Communication Channels
`5.4.3 Linear Estimation of Desired Signals in MIMO Communication Systems
`5.4.4 Non-linear Estimation of Desired Signals in MIMO Communication Systems
`5.4.5 Conclusions
`References
`
`6 The Challenges of Software-Defined Radio
`Carl Panasik and Chaitali Sengupta
`6.1 Cellular Communications Standards
`6.2 What is SDR?
`6.3 Digitizing Today’s Analog Operations
`6.4 Implementation Challenges
`6.5 Analog and ADC Issues
`6.6 Channel Filter
`6.7 Delta-Sigma ADC
`References
`
`6.8 Conclusion
`
`41
`
`4]
`42
`4)
`42
`43
`44
`44
`46
`46
`46
`48
`48
`50
`52
`54
`
`57
`
`57
`58
`62
`62
`66
`67
`71
`73
`75
`77
`83
`83
`86
`87
`90
`93
`93
`
`97
`98
`98
`101
`103
`103
`7
`"9
`
`l
`
`IPR2022-00602
`Apple EX1053 Page 4
`
`IPR2022-00602
`Apple EX1053 Page 4
`
`

`

`
`
`
`
`Contents vii
`
`7 Enabling Multimedia Applications in 2.5G and 3G Wireless Terminals: Challenges and
`Solutions
`Edgar Auslander, Madhukar Budagavi, Jamil Chaoui, Ken Cyr, Jean-Pierre Giacalone,
`Sebastien de Gregorio, Yves Masse, Yeshwant Muthusamy, TiemenSpits and Jennifer Webb
`7.1 Introduction
`7.1.1 “DSPs take the RISC”
`7.2 OMAP H/W Architecture
`7.2.1 Architecture Description
`7.2.2 Advantages of a Combined RISC/DSP Architecture
`7.2.3 TMS320C55x and Multimedia Extensions
`7.3 OMAPS/W Architecture
`7.4 OMAP Multimedia Applications
`7.4.1 Video
`7.4.2 Speech Applications
`7.5 Conclusion
`Further Reading
`
`8 A Flexible Distributed Java Environment for Wireless PDA Architectures Based on DSP
`Technology
`Gilbert Cabillic, Jean-Philippe Lesot, Frédéric Parain, Michel Bandtre, Valérie Issarny, Teresa
`Higuera, Gérard Chauvel, Serge Lasserre and Dominique D’Inverno
`8.1 Introduction
`8.2 Java and Energy: Analyzing the Challenge
`8.2.1 Analysis of Java Opcodes
`8.2.2 Analyzing Application Behavior
`8.2.3 Analysis
`8.3 A Modular Java Virtual Machine
`8.3.1 Java Implantation Possibilities
`8.3.2 Approach: a Modular Java Environment
`8.3.3 Comparison with Existing Java Environments
`8.4 Ongoing Work on Scratchy
`8.4.1 Multi-Application Management
`8.4.2 Managing the Processor’s Heterogeneity and Architecture
`8.4.3 Distribution of Tasks and Managementof Soft Real-Time Constraints
`8.4.4 Energy Management
`8.5 Conclusion
`References
`
`9 Speech Coding Standards in Mobile Communications
`Erdal Paksoy, Vishu Viswanathan and Alan McCree
`9.1 Introduction
`9.2 Speech Coder Attributes
`9.3 Speech Coding Basics
`9.3.1 Waveform Coders
`9.3.2 Parametric Coders
`9.3.3 Linear Predictive Analysis-by-Synthesis
`9.3.4 Postfiltering
`9.3.5 VAD/DTX
`9.3.6 Channel Coding
`9.4 Speech Coding Standards
`9.4.1] ITU-T Standards
`9.4.2 Digital Cellular Standards
`9.4.3 Wideband Standards
`
`107
`
`107
`107
`11
`111
`113
`113
`114
`116
`116
`116
`117
`117
`
`119
`
`119
`120
`120
`121
`125
`127
`127
`129
`131
`132
`133
`133
`133
`133
`133
`134
`
`137
`
`137
`138
`139
`141
`141
`143
`146
`146
`146
`147
`147
`148
`152
`
`IPR2022-00602
`Apple EX1053 Page 5
`
`IPR2022-00602
`Apple EX1053 Page 5
`
`

`

`Vili
`
`ContentsLe
`
`9.5 Speech Coder Implementation
`9.5.1 Specification and Conformance Testing
`9.5.2 ETSIITU Fixed-Point C
`9.5.3 DSP Implementation
`9.6 Conclusion
`Acknowledgements
`References
`
`10 Speech Recognition Solutions for Wireless Devices
`Yeshwant Muthusamy, Yu-Hung Kao and Yifan Gong
`10.1 Introduction
`10.2 DSP Based Speech Recognition Technology
`10.2.1 Problem: Handling Dynamic Vocabulary
`10.2.2 Solution: DSP-GPP Split
`10.3 Overview of Texas Instruments DSP Based Speech Recognizers
`10.3.1 Speech Recognition Algorithms Supported
`10.3.2 Speech Databases Used
`10.3.3 Speech Recognition Portfolio
`10.4 TIESR Details
`10.4.1 Distinctive Features
`10.4.2 GrammarParsing and Model Creation
`10.4.3 Fixed-Point Implementation Issues
`10.4.4 Software Design Issues
`10.5 Speech-Enabled Wireless Application Prototypes
`10.5.1 Hierarchical Organization of APIs
`10.5.2 InfoPhone
`10.5.3 Voice E-mail
`10.5.4 Voice Navigation
`10.5.5 Voice-Enabled Web Browsing
`10.6 Summary and Conclusions
`References
`
`11 Video and Audio Coding for Mobile Applications
`Jennifer Webb and Chuck Lueck
`11.1 Introduction
`11.2 Video
`11.2.1 Video Coding Overview
`11.2.2 Video Compression Standards
`11.2.3 Video Coding on DSPs
`11.2.4 Considerations for Mobile Applications
`11.3 Audio
`11.3.1 Audio Coding Overview
`11.3.2 Audio Compression Standards
`11.3.3 Audio Coding on DSPs
`11.3.4 Considerations for Mobile Applications
`11.4 Audio and Video Decode on a DSP
`References
`
`12 Security Paradigm for Mobile Terminals
`EdgarAuslander, Jerome Azema, Alain Chateau and Loic Hamon
`12.1 Mobile Commerce General Environment
`12.2 Secure Platform Definition
`12.2.] Security Paradigm Alternatives
`12.2.2 Secure Platform Software Component
`12.2.3 Secure Platform Hardware Component
`
`
`
`153
`153
`154
`155
`155
`156
`156
`
`160
`
`160
`160
`16]
`16]
`16]
`161
`161
`162
`165
`165
`166
`167
`168
`168
`169
`171
`172
`173
`174
`175
`176
`
`179
`
`179
`18]
`182
`186
`187
`188
`190
`19]
`193
`195
`196
`198
`200
`
`201
`
`202
`203
`204
`204
`205
`
`IPR2022-00602
`Apple EX1053 Page 6
`
`IPR2022-00602
`Apple EX1053 Page 6
`
`

`

`
`
`Contents
`ix_rr’
`
`12.3 Software Based Security Component
`12.3.1 Java and Security
`12.3.2 Definition
`12.3.3 Features for Security
`12.3.4 Dependency on OS
`12.4 Hardware Based Security Component: Distributed Security
`12.4.1 Secure Mode Description
`12.4.2 Key Management
`12.4.3 Data Encryption and Hashing
`12.4.4 Distributed Security Architecture
`12.4.5 Tampering Protection
`12.5 Secure Platform in Digital Base Band Controller/MODEM
`12.6 Secure Platform in Application Platform
`12.7 Conclusion
`
`13 Biometric Systems Applied To Mobile Communications
`Dale R. Setlak and Lorin Netsch
`13.1 Introduction
`13.2 The Speaker Verification Task
`13.2.1 Speaker Verification Processing Overview
`13.2.2 DSP-Based Embedded Speaker Verification
`13.3 Live Fingerprint Recognition Systems
`13.3.1 Overview
`13.3.2 Mobile Application Characterization
`13.3.3 Concept of Operations
`13.3.4 Critical Performance Metrics
`13.3.5 Basic Elements of the Fingerprint System
`13.3.6 Prototype Implementation
`13.3.7 Prototype System Processing
`13.4 Conclusions
`References
`
`14 The Role of Programmable DSPsin Digital Radio
`Trudy Stetzler and Gavin Ferris
`14.1 Introduction
`14.2 Digital Transmission Methods
`14,3 Eureka-147 System
`14.3.1 System Description
`14.3.2 Transmission Signal Generation
`14.3.3 Receiver Description
`14.4 IBOC
`14.5 Satellite Systems
`14.6 Conclusion
`References
`
`15 Benchmarking DSP Architectures for Low Power Applications
`David Hwang, CimarronMittelsteadt and Ingrid Verbauwhede
`15.1 Introduction
`15.2 LPC Speech Codec Algorithm
`15.2.1] Segmentation
`15.2.2 Silence Detection
`15.2.3 Pitch Detection Algorithm
`15.2.4 LPC Analysis — Vocal Tract Modeling
`15.2.5 Bookkeeping
`
`205
`205
`205
`206
`207
`207
`208
`210
`211
`z12
`213
`214
`215
`215
`
`217
`
`217
`219
`
`224
`225
`225
`226
`226
`228
`233
`247
`248
`251
`251
`
`253
`
`253
`254
`255
`255
`262
`265
`279
`284
`285
`286
`
`287
`
`287
`
`288
`288
`289
`289
`290
`
`IPR2022-00602
`Apple EX1053 Page 7
`
`IPR2022-00602
`Apple EX1053 Page 7
`
`

`

`Contents
`x
`——————————————————————
`
`15.3 Design Methodology
`15.3.1 Floating-Point to Fixed-Point Conversion
`15.3.2 Division Algorithm
`15.3.3 Hardware Allocation
`15.4 Platforms
`15.4.1 Texas Instruments TI C54x
`15.4.2 Texas Instruments TI C55x
`15.4.3 Texas Instruments TI C6x
`15.4.4 Ocapi
`15.4.5 AIRT Designer
`15.5 Final Results
`15.5.1 Area Estimate
`15.5.2 Power Estimate
`15.6 Conclusions
`Acknowledgements
`References
`
`16 Low Power Sensor Networks
`Alice Wang, Rex Min, Masayuki Miyazaki, Amit Sinha and Anantha Chandrakasan
`16.1 Introduction
`16.2 Power-Aware Node Architecture
`16.3 Hardware Design Issues
`16.3.1 Processor Energy Model
`16.3.2 DVS
`16.3.3 Leakage Considerations
`16.4 Signal Processing in the Network
`16.4.1 Optimizing Protocols
`16.4.2 Energy-Efficient System Partitioning
`16.5 Signal Processing Algorithms
`16.5.1 Energy-Agile Filtering
`16.5.2 Energy-Agile Data Aggregation
`16.6 Signal Processing Architectures
`16.6.1 Variable-Length Filtering
`16.6.2 Variable Precision Architecture
`16.7 Conclusions
`References
`
`17 The Pleiades Architecture
`Arthur Abnous, Hui Zhang, Marlene Wan, George Varghese, Vandana Prabhu, Jan Rabaey
`17.1 Goals and General Approach
`17.2 The Pleiades Platform — The Architecture Template
`17.3 The Control Processor
`17.4 Satellite Processors
`17.5 Communication Network
`17.6 Reconfiguration
`17.7 Distributed Data-Driven Control
`17.7.1 Control Mechanism for Handling Data Structures
`17.7.2 Summary
`17.8 The Pleiades Design Methodology
`17.9 The P1 Prototype
`17.9.1 Pl Benchmark Study
`17.10 The Maia Processor
`17.10.1 Control Processor
`17.10.2 Address Generator Processor
`
`
`
`290
`290
`299
`293
`293
`293
`294
`294
`294
`294
`294
`295
`295
`297
`298
`298
`
`299
`
`299
`300
`302
`303
`304
`306
`311
`312
`313
`317
`318
`319
`320
`321
`322
`324
`324
`
`327
`
`327
`329
`331
`352
`334
`338
`339
`342
`345
`345
`348
`350
`352
`353
`S53
`
`IPR2022-00602
`Apple EX1053 Page 8
`
`IPR2022-00602
`Apple EX1053 Page 8
`
`

`

`Contents
`xier
`
`17.10.3 MemoryUnits
`17.10.4 Multiply-Accumulate Unit
`17.10.5 Arithmetic/Logic Unit
`17.10.6 Embedded FPGA
`17.10.7 Maia Results
`17.11 Summary
`References
`
`18 Application Specific Instruction Set Architecture Extensions for DSPs
`Jean-Pierre Giacalone
`18.1 The Need for Instruction Set Extensibility in a Signal Processor
`18.2 ISA Extension Capability of the TMS320C55x Processor
`18.2.1 Control Modes
`18.2.2 Dataflow Modes
`18.2.3 Typical C55x Extension Datapath Architecture
`18.2.4 Integration in Software Development Tools
`18.3 Domains of Applications and Practical Examples
`18.4 ISA Extensions Design Flow
`References
`
`19 The Pointing Wireless Device for Delivery of Location Based Applications
`Pamela Kerwin, John Ellenby and Jeffrey Jay
`19.1 Next Generation Wireless Devices
`19.2 The Platform
`19.3 New Multimedia Applications
`19.4 Location Based Information
`19.5 Using Devices to SummonInformation
`19.6 Pointing to the Real World
`19.7 Pointing Greatly Simplifies the User Interface
`19.8 Uses of Pointing
`19.9 Software Architecture
`19.9.1 Introduction
`19.9.2 Assumptions
`19.9.3 Overview
`19.9.4 Alternatives
`19.10 Use of the DSP in the Pointing System
`19.11 Pointing Enhanced Location Applications
`19.11.] Pedestrian Guidance
`19.11.2 Pull Advertising
`19.1].3 Entertainment
`19.12 Benefits of Pointing
`19.12.1 Wireless Yellow Pages
`19.12.2 Internationalization
`19.12.3 GIS Applications
`19.12.4 Entertainment and Gaming
`19.12.5 Visual Aiding and Digital Albums
`19.13 Recommended Data Standardization
`19.13.1 Consideration of Current Standards Efforts
`19.13.2 Device Data Types and Tiered Services
`19.13.3 Data Specifications
`19.13.4 Data Format
`19.13.5 Is it Sufficient?
`19.14 Conclusion
`
`Index
`
`354
`354
`354
`354
`355
`357
`358
`
`361
`
`361
`362
`
`366
`367
`
`372
`376
`377
`
`379
`
`379
`379
`379
`380
`380
`380
`381
`382
`382
`382
`382
`383
`383
`383
`384
`385
`386
`386
`387
`387
`387
`387
`
`388
`388
`388
`388
`389
`391
`393
`393
`
`395
`
`IPR2022-00602
`Apple EX1053 Page 9
`
`IPR2022-00602
`Apple EX1053 Page 9
`
`

`

`iric Systems Applied To Mobile Communications
`
`piome
`
`225
`
`ple 13.1 Verification resources example
`
`Je
`
`Ta
`
`ROM
`
`RAM
`
`MIPS
`
`EER
`
`-
`verification task
`ag distance telephone, ten continuousdigits
`
`Long
`
`* SSeS
`
`8K program IK search
`
`8
`
`2.1%
`
`is reduced to three. If a text-dependent verification system
`robabilities for the model
`contains a set of speaker-independent word HMMsthat serve as the basis for allowable
`verification phrases, then the parameters of the speaker-independent HMMs maybe used.
`For example, it may only be necessary to estimate the parameter 1,,,, while obtaining all
`ther parameters from the speaker-independent HMMs. Other parameters, such as the
`variance estimates of the Gaussian mixture components may be shared within a model, or
`even between models. Using these simplifications, typical verification models for text-depen-
`dent embedded applications require about 100 parameters per spoken word. However, more
`complex signal processing algorithms have been developedthat retain performance with as
`jowas 20 parameters per spoken word [12].
`Program memory storage necessary for speakerverification will depend on the particular
`speaker verification algorithm used. For a typical text-dependent application, the speaker
`verification code will be a small addition to the speech recognition code. Processing require-
`ments for the front-end feature processing will be similar to the speech recognition code.
`Text-dependent recognition requires calculation of the maximumlikelihood path through the
`sequence of HMMs making up the spoken phrase. However, unlike speech recognition
`applications, speaker verification uses the a priori knowledge of the spoken phrase. This
`implies that processing resourceswill be less than those reported for speech recognition. As
`an example, as shown in Table 13.1, except for front-end feature processing, the resources for
`text-dependent speaker verification using ten digit phrases will be about one-tenth ofthat
`reported for digit recognition as reported in Chapter 10.
`
`13.3 Live Fingerprint Recognition Systems
`
`13.3.1 Overview
`
`Theability to implementfingerprint ID systems in mobile devices hinges on the confluence of
`two technology developments: the recent commercial availability of very small, low power,
`high quality fingerprint sensors and the introduction of a new generation of fast, powerful
`DSPs into mobile devices.
`In this section we review the engineering elements of designing fingerprint systemsinto the
`next generation mobile devices. Webriefly characterize the unique aspects of mobile finger-
`print systems, develop the concept of operations for mobile fingerprint systems, and then
`examine the critical performance metrics used to control the system design and ensure its
`adequacy. The fingerprint system is then decomposedinto its basic elements. Eachof these is
`described along with some possible design approaches and implementation alternatives.
`Lastly, we describe a prototype system architecture based on the Texas Instruments’
`OMAParchitecture, and discuss the design and implementation of a demonstration system
`constructed using this architecture.
`
`IPR2022-00602
`Apple EX1053 Page 10
`
`IPR2022-00602
`Apple EX1053 Page 10
`
`

`

`The Application of Programmable DSPs in Mobile Communications
`226
`OOOO
`
`13.3.2 Mobile Application Characterization
`
`13.3.2.1 End-User Benefits
`Live fingerprint recognition on mobile devices makesbasic security and device personaliza.
`tion convenient for the user. Entering usernames, passwords, or PIN numbersinto portable
`devices is inconvenient enough that most people today don’t use the security and persona.
`lization functions in their portable devices. With live fingerprint recognition,a single touch of
`the sensor device is all that is required to determine the user’s identity, configure the device
`for personal use, or authorize access to private resources.
`
`13.3.2.2 Expected Usage Patterns
`
`A portable device typically has a small group of between one and five users. When an
`authorized user picks up the device and presents his/her finger to the sensor,
`the device
`should recognize the user and immediately switch its operation to conform to his/herprofile,
`
`13.3.2.3 Unique Aspects of the Application
`
`Mobile devices require fingerprint sensors that are significantly smaller than any previously
`used. This requirement propagates into two aspectsofthe fingerprint system design.Thefirst
`challenge is to build an adequate quality sensor small and light enough for mobile devices,
`The second challenge comes from the fact that smaller sensors generate images ofsmaller
`sections of skin. This meansless datais available for comparison than with the largersensors
`typically used for fingerprint recognition. To successfully match smaller fingerprint images
`the sensor must generate higher quality and more consistent images, and the matcheralgo-
`rithm must be designed to take advantage of the higher quality data.
`Alternatively, some systems require the userto slide his finger slowly acrossthe sensor, to
`increasethearea offinger surface imaged. This motionis called swiping. While this approach
`generates imagery of a larger area ofskin, it seriously distorts the skin and hassignificant
`operational and performanceliabilities.
`The prototype application discussed later in this chapter uses an AuthenTec AES-4000
`sensor with a sensing area just under | cm”. Systems using even smaller sensors are under
`developmentat several fingerprint system suppliers.
`
`13.3.3 Concept of Operations
`
`The operational concepts underpinning most fingerprint authentication systems revolve
`around three classes of user events: enrollments, verifications, and identifications. Each of
`these eventclasses is described below from ahigh-level process view. The proceduresundet-
`lying these processes are discussedlater in this chapter.
`
`13.3.3.1 Enrollment
`
`Enrollmentis the process of authorizing a new person to use the mobile device. Ina typical
`scenario, the owner of the device authorizes a person to use the device by: authenticating
`himself/herself to the device as the owner, creating a new user profile with the desired
`
`IPR2022-00602
`Apple EX1053 Page 11
`
`IPR2022-00602
`Apple EX1053 Page 11
`
`

`

`piometric Systems Applied To Mobile Communications
`
`227
`
`rivileges on the device, and then training the device to recognize the new user’sfingerprints.
`Typically the system1s trained to recognize twoorthreefingers for each personin case injury
`makes one finger unavailable.
`The process of training the fingerprint system to recognize a new finger can be broken
`downlogically into the following steps:
`e Collection of system training data samples
`Feature quality analysis
`¢ Template generation
`e Template storage
`
`Collection of Training Data Samples
`The systemcollects several viewsof a finger, prompting the newuserto lift and replace their
`finger on the fingerprint sensor several times. Each finger placementis considered as one
`view. Each view may consist of a sequence of image framesthat taken together define a view.
`
`13.3.3.2 Feature Quality Analysis
`
`The collected samples (called views) are analyzed to extract the features that will be used for
`matching. The system then assesses the quantity and quality of matchable feature data present
`in the views, and estimates the probable robustnessof that feature data. The results of this
`analysis determine whetherthe system canusethis set of views for enrollment, or if more, or
`better, data are needed. If the data are insufficient, the system may request more viewsofthe
`same finger or request the new userto presenta different finger.
`
`Template Generation
`If the data is sufficient for enrollment, the system assemblesthe best of the available data and
`formats it into a template that will be used as the reference for subsequent matching ofthis
`finger.
`
`Template Storage
`Theresulting template is then stored under an appropriate encryption schemefor recall during
`subsequent verification and identification operations. Templates can be stored on any media
`that can hold digital data. On mobile devices templates are typically stored in flash memory.
`
`13.3.3.3. Verification (Claimed Identity Verification)
`
`Verification is the process of authenticating a claimed user identity. A verification event
`occurs when: (1) a user indicates his/her identity to the system (usually by typing in a
`username) and (2) the system verifies the claimed identity by comparing the user’s live
`fingerprint to the template stored for that username. This type of comparisonis often called
`a One-to-one comparison becauseonly one stored template is compared to the live fingerprint.
`Verification processes generally require significantly less computational horsepower to
`perform than identification processes, and may be more reliable. However, verification is
`generally less convenient for the user as the username must be entered manually. Given that
`user convenience is a primary requirementfor fingerprint systems on mobile devices, veri-
`
`IPR2022-00602
`Apple EX1053 Page 12
`
`IPR2022-00602
`Apple EX1053 Page 12
`
`

`

`The Application of Programmable DSPs in Mobile Communication,
`228
`$s
`
`fication processes are probably inappropriate and identification processes (discussedin the
`next section) are preferred.In situations where only one personuses a device (which may be,
`significant percentage of devices) the identification process essentially devolvesto a simple
`verification, so the performance penalty is minimal.
`
`Data Collection
`The system typically collects one view of the finger, which may consist of a sequenceof
`image frames. For extremely small sensors, it may be necessary to collect multiple views of
`the finger to accumulate enough data to perform the fingerprint match.
`
`Feature Analysis
`The collected images are analyzed using various forms of pattern recognition algorithmsto
`extract the features to be used for matching.
`
`Matching to a Template
`The data from the live finger is comparedto the stored template for the claimed identity anda
`probability that the claimed identity is true is estimated from the match results. The system
`returns a binary result. The claimed identity is either true orfalse.
`
`13.3.3.4 Identification (Unassisted Identification)
`
`Identification is the process of finding the current user’s identity from a list of possible
`identities, based solely on the user’s live fingerprint. Identification processes do notrequire
`the user to enter a usernameor any other co-joined authentication. Instead, a single touch of
`the fingerprint sensor is sufficient. Identification processes typically require significantly
`more computational power than verification. Additionally, in identification processes both
`the accuracy andthelatency ofthe process are not constant, as they are functionsofthesize of
`the reference dataset being searched.
`From the process perspective, identification is similar to verification with two notable
`exceptions: (1) no usernameis entered, and (2) the system must perform an indexed search
`ofall of the possible enrolled templates to find the matching templateif it exists in the dataset.
`The result of the processis either the selected ID or an indication that the presentedfingeris
`not in the dataset.
`The identification process, with its one-step usage paradigm,is significantly better suited to
`convenient personalization than the verification process.
`
`13.3.4 Critical Performance Metrics
`
`13.3.4.1 Biometric Performance
`Biometric performance measures evaluate how well the system does the job of recognizing
`and differentiating people. At the system level, there are two generally accepted classes of
`problemsto be avoidedin these recognition systems. Thefirst class of problems occurs when
`the system cannot acquire a reasonable quality image ofthe finger. These usability problems
`are mostly associated with the sensoritself, and are called “failure to acquire”errors. In some
`systems available today, acquisition failure errors dominate the behavior of the system. The
`secondclass of problems occurs whenthe system has adequate imagery but makesanerror in
`
`IPR2022-00602
`Apple EX1053 Page 13
`
`IPR2022-00602
`Apple EX1053 Page 13
`
`

`

`Biometric Systems Applied To Mobile Communications
`
`229
`
`rforming the pattern recognition. This second class of problems is more generally asso-
`ciated with the pattern matching software, and can be categorized in classical terms as false
`accept errors (Type 2) and false reject errors (Type 1).
`
`Usability — Ability to Acquire
`Mobile communication devices are rapidly becoming a ubiquitous part of our everyday
`environment. As such, the fingerprint systems will have to work for everyone’s fingers.
`Failure to operate with fingers that fall outside of a norm — such as elderly, sweaty or dry
`fingers — will not be acceptable. The systems will have to work in a wide range of environ-
`ments; not just the office and the car, but also the tennis court, the garage, and the ski lodge.
`Manyfingerprint sensors are extremely sensitive to the condition of the finger skin that they
`must image. Somesensorsavailable today successfully image young healthy fingers, but are
`unable to image elderly fingers or fingers with dry skin. Some are unable to function in
`environments more demanding than a clean office. And yet, some sensors can adequately
`handle all of these conditions.
`The Ability-to-Acquire metric measures a system’s ability to capture usable fingerprint
`imagesover the range of population demographics, usage patterns, and environmental condi-
`tions appropriatefor that intended application. It can be represented as the expected percentage
`of successful finger imaging events over the ranges appropriate for a particular application.
`For general-purpose mobile communications and information devices we believe that the
`fingerprint system’s ability to acquire fingerprints should be in the range of 99.99%, over a
`population demographic that represents the entire world population and includes both clean
`and slightly contaminated fingers, and over a wide range of both indoor and outdoorenvir-
`onments.
`
`Identification/Verification Accuracy
`Identification and verification accuracies are usually represented as the percentage of identi-
`fication/verification events in which the system delivers an inappropriate response; either
`incorrectly rejecting a finger that should match(false reject), or incorrectly accepting a finger
`that should not match (false accept). Identification accuracy and verification accuracy (while
`using the same type of error metrics) are best treated for this discussion as two different kinds
`of specifications that are associated with two different implementationsoffingerprint authen-
`tication systems, as discussed earlier in this chapter.
`
`Verification Accuracy
`There are two classes of measurementtraditionally used to quantify identification/verification
`accuracy. These are the False Accept Rates (FARs)and the False Reject Rates (FRRs). We
`believe that mobile communications device applications when used in verification mode
`require FARs of 0.1% or less (which is sometimes considered similar to the probability of
`someoneintelligently guessing a user-selected four-digit PIN).
`Forthis type of live fingerprint recognizer, false reject errors come in two varieties. Some-
`times a valid user will place his finger on the sensor and beinitially rejected, but will
`immediately try again, repositioning the finger on the sensor, and be accepted. This is called
`a nuisancereject. It is a special case that only occursin live fingerprint systems, where the
`user can retry immediately. Our informal experience suggests that nuisance reject rates
`exceeding the 5-7% range can degrade the user experience and should be avoided. The
`
`IPR2022-00602
`Apple EX1053 Page 14
`
`IPR2022-00602
`Apple EX1053 Page 14
`
`

`

`
`
`230 The Application of Programmable DSPs in Mobile Communicationsaeins
`
`second and far moreserioustype of false reject is the denial-of-service event.In this type of
`event, the system fails to accept a valid user’s fingers even after several retry attempts,
`Clearly, for user satisfaction reasons, the denial-of-service reject should be avoided. We
`believe that denial-of-service FRRs of 0.1% or less will be required for ubiquitous deploy-
`ment.
`
`Identification Accuracy
`False Accept Errors
`Whena system performsanidentification event, it must in essence comparethelivefinger.
`print to all of the templates that are considered possible matches for that live finger. For
`comparison, when an non-enrolled finger is presented to the system,a verification function
`comparesthe finger to only one template. Henceit has only one oppo