text message, etc). Date/time tyoe EFRs are typically not
`application distribution (e.g. email,
`
`
`
`used 9850gis usedsince expression field for qualifying this information anyway. Thereafter, if
`
`
`
`
`
`lication
`lock 10194 determines the MADR does indeed match the requirements of the
`which invoked FIG. 20 processing, then block 10116 invokes a presentMADR procedureof FIG.
`25A with parameters passed for: the MADR object (e.g. or pointer thereof), a constant of
`“offline”, sender, recipient and CLOC information if available, and processing continues back to
`block 10108. If block 10194 determines one or more EFRs do not match parameters, then
`rocessing continues
`back to block 10108. If block 10114 determines
`the
`expression evaluated
`
`to False,
`
`then processing
`
`leaves
`
`block 10114 for
`
`block 10108. When zero
`
`or more COM-L
`
`MADRs
`
`are
`
`pr
`
`lock 10018
`
`checks
`
`t
`
`if the application event is an outbound
`
`event.
`
`otherwise
`processing continues to block 10020,
`distribution,
`If the event is an outbound
`processing continues to block 10148 where the application context invokeris returned to.
`Block 10120 accesses active COM-R MADRsfor the particular application (e.g. COM-R-email
`and places them into a recognizable list a user can act upon, block 10122 presents the list to
`the user of the MS of FIG. 20 processing, block 10124 interfaces with the user forlist selections,
`and block 10126 waits for particular user actions. Block 10120 preferably sorts the MADRs
`based on a selectsetof field settings in the MADRs. Block 10120 also uses Boolean variables
`tin FIG. 19
`to determine
`if any MADRs
`should b
`arched at all. One MADR is
`preferabl
`
`prior t
`any MADR data
`xamine
`found. The user m
`ut none m
`foun
`are
`any
`if
`found
`making a selection at block 10124, for example by actions processed at block 10132. If no
`MADRsare found, thelist is an emptylist where the user can interface at block 10124 for
`exiting via block 10128. When an action is detected, block 10126 continues to block 10128. If
`block 10128 determines the user selected to exit processing, then the application context event
`etecting
`invoker
`of FIG. 20
`is returned to
`at
`block 10148,
`otherwise processin
`ntinues t
`
`(e.g. COM-R
`or more MADR(s)
`block 10130. If block 10130 determines the user selected one
`then processing continues to block 10134, otherwise any other action at block 10124 is
`appropriately handled at block 10132 and processing continues back to block 10124.
`
`Block 10134 starts an iterative processing
`
`loo
`
`etting each selected MADR and
`
`continuin
`
`to
`
`block 10136.
`
`If block 101
`
`eterminesall selected COM-R MADRshave not
`
`vet
`
`been
`
`ntinues t
`therwise processin
`block 101
`ntinues to
`rocessin
`rocesse
`block 10148 for returning to the invoker of FIG. 20. Block 10138 interfaces with the user to
`specify
`and/or confirm delivery
`criteria (field 9850 h) for where to send/present the message
`field 9850 c information (e.g. remote MS). When accessed at block 10120, COM-R MADR
`fields 9850 h may be setto null, populated with a useful default or starter data, or may already
`contain complete
`information. The user can confirm or specify
`different
`delivery
`criteria
`at
`
`block 10138. In
`
`some
`
`embodiments,
`
`the user
`
`can alter any MADR fields
`
`prior t
`
`ndin
`
`referably
`
`as governe
`
`ermissions. lf the user
`
`decided
`
`to exit out of MADR processing
`
`at
`
`back to block 10134
`continues
`ing
`then pro
`block 10138 as detected at block 10140,
`send parametersat
`preparing
`by
`otherwise the MADR is delivered to the remote MS(s)
`block 10142 and invoking send processing of FIG. 13A at block 10144 before continuing back to
`block 10134. Depending on settings in the application distribution for outbound processing, FIG.
`13A may need to be invoked for a plurality of recipient MSs, therefore an iterative loop 10146is
`appropriately incorporated around
`blocks 10142 and 10144 for handling multiple recipients, an
`for handling attempts for a prioritized retry. An alternate embodiment may handle multiple
`recipients in send processing invoked at block 10144 depending on a transport interface used.
`Parameters are prepared at block 10142 so a MADR is deliveredin its entirety for processing at
`
`EXHIBIT 1013 - PAGE 0251
`
`APPLE
`
`

`

`be utilized. FIG. 20 focuses on MADR
`the receiving MS(s). Other transport mechanisms may
`processing. Various embodiments may not assumethe inbound or outbound application
`istribution associated with COM processing
`is processe
`ropriately outside of FI
`20 processing. Preferably, a CLOC parameter is passed wheneverpossible, including via
`block 10144 (e.g. block 10142
`esses MS whereabouts for setting
`the CLOC dat
`
`Another embodiment may not interface with the user at block 10138 and instead use delivery
`field 9850 h to deliver the MADR for processing at the specified receiving MS(s). In this
`embodiment, the receiving MSs are assumedto be the targets for presentation/message
`information of the MADR sent.
`
`With reference now to FIG. 7513A, depicted is a flowchart for describing a preferred
`embodimentof a procedure for sending data to a remote MS, for example to perform a remote
`
`action oHinkablePipcede-Gof presenting theastavekedfrem-bleck6162-F167/5/-is-preferably
`
`MADR information to a user. Parent applications contain discussions relevant to MADR
`processing. The purposeis for the MS of FIG. 4513A processing (e.g. a first, or sending, MS) to
`
`
`
`transmit MADRdata to other MSs3s
`
`
`
`(@.g. at least aa second, orreceiving, MS)forexomplean action
`
`for remote processing of the MADR information. The receiving
`
`MS
`
`may
`
`receive MADR dat
`
`wirelessly by
`
`being within wireless
`
`range of the sending
`
`MS
`
`(i.e. no intervening data processin
`
`systems), or may receive over aa peerttopeer ¢connection by way 0of dataa processing system(s) exemplificationsshownin-FIGS.-SOA-throughSOC--respectivelyrange data flow. Processing begins at
`
`
`block 7502, continues to block 7504 where the caller parameter(s) passed to FIG.
`7513A processing {e-g-actier
`amote
`e
`utien}-are used for
`
`sending at least one data packet containing aroperly formatted data for sending, and for being
`
`properly received and interpreted. Block 7504 may reformat parameters into a suitable data
`packet(s) format so the receiving MS can process appropriately (see FIG. 7513B). Depending
`
`on thepresentdiselesure-embodiment, any reasonable supportedidentity 4#O4DFyseHis a valid
`
`target (e.g.asandmaybederived from a+ecipient-ersystemparameterthe delivery criteria).
`Thereafter, block 7506 waits for an acknowledgementfrom the receiving MS if the
`communication embodimentin useutilizes that methodology. In one embodiment, the send data
`packetis an unreliable datagramdatagram(s) thatwill mostlikely be received by the target MS. In
`another embodiment, the send data packetpacket(s) is reliably transported data which requires
`
`ana_final acknowledgementthat it was received in good order. In any case,
`block 7506 continues to block 7508.
`
`EXHIBIT 1013 - PAGE 0252
`
`APPLE
`
`

`

`
`potentlytenceThetargetedMS should recognizethatthe data iis meantfor "and
`
`p
`
`p
`
`Block 7504 formats the data for sending in accordancewith the specified delivery method, along
`with necessary packet information (e.9. source identity, wrapperddata, GLOe, etc), and sends
`dataa appropriately.
`bread d Ad, bleck 7504 breaded Aa
`
`
`
`
`
`
`
`receivesit.
`
`eh
`
`Block 7506 waits for a synchronous acknowledgementif applicable to the send of
`block 7504 until either receiving one or timing out. Block 7506will not wait if no ack/responseis
`
`anticipated, in which case block 7506 sets status for block 7508 to ‘got it”.Habroadeastwas be accounted beforedeemingasuccessfulfor ackThereatfter, if block 7508 determines aan
`
`
`
`
`applicable ack/response wasreceived (i.e. data successfully sent/received), or none was
`anticipated (i.e. assumegotit), then processing continues to block 7510 for potentially
`processing thea response. Block 7510 will process the responseif it was anticipated for being
`received as determined by data sent at block 7504. Thereafter, block 7512 performs logging for
`success-e.g-+o-8xHistory36}. If block 7508 determines an anticipated ack was notreceived,
`
`then block 75127514 logs the attemptte-.gtoL8xhistery39}. An alternate embodimentto
`block 7514 will log an error and may require a user action to continue processing so a user is
`confirmed to have seen the error. Both blocks 7512 and 7514 continue to block 7516 where the
`catertinvoker} is returned to for continued processing (e.g. back to block 616210144).
`
`With reference now to FIG. 4513B, depicted is a flowchart for describing a preferred
`embodiment of Processing for receiving execution data from another MS, for example actien
`ena MADR object. FIG. 75138 processing describes a3 Receive Execution Datafa (RXED)pprocess worker thread,andis-of
`PiPeede6. Theree may be many worker threads for the RxED Process, just
`
`asdescribedFora18%
`
`EXHIBIT 1013 - PAGE 0253
`
`APPLE
`
`

`

`
`
`
`iselatedtomeodularreceiveprecessing.. Parent applications contain discussions relevant to MADR
`
`dataprocessing.
`
`A RxED thread processing begins at block 7552, continues to block 7554 where thea process
`workerthread count RxED-Ct is accessed and incremented by 1 (using appropriate semaphore
`access (e.g. RxED-Sem)), and continues to block 7556for retrieving from a receive queue 26the
`
`
`
`sent datatusinginterfaceinterfacetike1948}, perhaps a special termination request entry, and
`only continues to block 7558 when athe MADR data, or record of data (e.g. action for remote
`execution, particular atomic command,or terminationn
`
`record)|is retrieved. Hrene-embediment;
`
`Block — stays blocked on retrieving from a receive queueuntil data is retrieved, in which
`terminate was notfound|in the receive queue, processingcontinues to block £280. Block
`
`in
`
`ntinues
`
`to block 7558. If block 7558 determin
`
`ial
`
`entry
`
`indicating
`
`t
`
`7556
`
`Block 7560 validates incoming data for this targeted MS before continuing to block 7562. A
`preferred embodiment of receive processing already validated the data is intended for this MS
`by having listened specifically for the data, or by having already validatedit is at the intended
`MS destination (e.g. block 7558 can continue directly to block 7564 (no block 7560 and
`block 7562 required)). If block 7562 determines the data is valid for processing, then
`
`EXHIBIT 1013 - PAGE 0254
`
`APPLE
`
`

`

`
`block 7564 checksthe datafor its purpose (remote action-er, particular command,_or MADR
`
`IfIf block 7564 determines the data received is for‘ Processing a remote action, then
`processing).
` processing continues to block 7566 2as describediin1 the parentapplications.|If
`
`block 7564 determines that the execution data is for processing MADR data, then processing
`continues to block 7578 where the MADR is prepared for subsequent processing.
`Block 7578 accesses MADR fields and block 7579 evaluates expression field 9850 g in context
`f the receiving MS. Privileges should
`be
`essed
`at
`block 7579 for special terms which
`
`far
`then the MADR datai
`True,
`to
`Valuates
`5
`expression fiel
`rmission. If
`requir
`assumedto be privileged for further processing. If the expression evaluates to False, or is not
`able to be evaluated because of an undefined or unprivileged term, then the MADR data is
`assumed to NOTbeprivileged for further processing, and block 7580 need not pursue access
`further to privilege data+eceived. Thereafter, block 75687580 accessesprivileges fereach-ofthe
`remete-actien-partsfeommand,-operand,parameters}(i.e. if Expression evaluated to True) for
`eligible MADR processing to ensure the source hasproperprivileges for runningprocessing the
`
`
`actionMADR data at the MS of FIG. 7513B Processing. DependingembodimentsBlockof
`
`7568
`field 9850 g set to null implies a True evaluation result.
`
`actionprivilegeverificationalreadyinFIG.discussed61,Block 7580 recognizes the MADR for not
`
`
`
`
`
`eing
`privileged
`if expression fiel
`5
`id not evaluate to True at
`block 7579. Expression
`
`EXHIBIT 1013 - PAGE 0255
`
`APPLE
`
`

`

`
`
`Thereafter, if block 7582 determines the command{Command,Oserand,Parameters}MADRdata
`
`
`for execution is acceptable (and perhaps-privileged-orpriviiegedpersource-ortherewas-ne-check
`necessary), then block 7584 performsthecommandtocallyinvokes the presentMADR procedure
`of FIG. 25A at the MS of FIG. 75A13B processing_with parameters passed for: the MADR(e.g. a
`ointer thereof),
`a constantof “offline”, and optionally
`(may be null) sender, receiver and CLOG
`
`information which may.useful forpresentation. Thereafter, block 7586 checksif a responseis
`84. If
`A
`nc_command
`Amancdte
`needed-as-aresult-e
`
`block 7586 determines a response is to be sentback to theoriginating MS,
`
`
`block 7574 completes a responseto the originating MS of the data received at block 7556, and
`block 7576 sends/broadcasts the response, before continuing back to block 7556 for the next
`
`incoming execution request data. Ble
`5-5
`bread
`he-+respense-containingas
`
`
`
`If block 7586 determines a responseis not to be sent backto the originating MS, then
`processing continuesdirectly back to block 7556. If block 7582 determines the dataMADR for
`processing is not acceptable/privileged, then processing continues back to block 7556.+e+
`
`
`EXHIBIT 1013 - PAGE 0256
`
`APPLE
`
`

`

`
`
`Referring back to block 7562, if it is determined that the data is not valid for the MS of FIG.
`7513B processing, processing continues backto block 7556. Referring back to block 7558, if a
`workerthread termination request was found at the receive queue-26, then
`block 7586 decrements the RxED worker thread count by 1 (using appropriate semaphore
`access(e.g. RxED-Sem)), and RxED thread processing terminates at block 7588.
`Block 7586 mayalso check the RxED-Ct value, and signal the RxED processparentthread that
`
`all worker threads are terminated when RxED-Ctequals zero (0).
`
`For other acceptable receive processing, methods are well knownto thoseskilled in the art for
`“hooking” customized processinginto application processing of sought data receivedjustas
`
`diseussec-with£16448-abeve(e.g. mail application, callback function API, etc). Thus, there are
`well known methodsfor processing data in-context-ofthis-disclesurefor receiving remete-actions
`and/oratemiccemmanddateMADRsfor processing from an originating MS+e-a+receiingMS, for
`example whenusing email. Similarly, as described above, SMS/text messages can be used to
`communicate data, albeit at smaller data exchange sizes. The sending MS maybreakup larger
`portions of data which can be sent as parse-able text to the receiving MS. It may take multiple
`SMS/text messages to communicate the datain its entirety. Various embodiments will send
`MADR(s)
`along with an associated distribution.
`
`Regardlessof the type of receiving application, those skilled in the art recognize many clever
`methodsfor receiving data in context of a MS application which communicatesin a peer to peer
`
`
`
`
`
`
`
`
`eatibacic function(s}, AP in-an appropriate loop whichfashion with anotherMSte-g-interfacescan
`
`EXHIBIT 1013 - PAGE 0257
`
`APPLE
`
`

`

`received-cretherapplicableprocessing}. FIGS. 7513A and 7513B are an embodiment of MS to MS
`communications, referred to with the acronym MS2MS._Various MS2MS communication
`embodiments mayinclude: reliable transport protocol involving a one or more packets (sends
`and acknowl
`ment:
`tween
`tems
`for a singl
`nd: unreliable
`transport
`protocol
`involving one or more packets (sends and acknowledgements) between systemsfor a single
`send: or on-going
`communications
`processing which is subsequentto an initiation send of
`data
`
`peer to peer application processing. In some embodiments, the LBX
`etween systems (e.g.
`service propagation architecture is leveraged for hopping data to the targeted peer MS wherein
`distance between anoriginating MS and a targeted MS is increased by intermediary MS(s)
`“middle-manning’ the transmission.
`
`COM-R event processing provides a user with useful confirmation of delivery status by sending
`a MADRobject to a target remote system with an expression for checking presence of a
`previously sentdistribution. If the previously sent distribution has been delivered, acted upon, or
`used
`asindicate
`applicable AppTerm variables, the sending user
`can
`be
`delivered a
`messagein any of the variety of presentation types for the confirmation of delivery status.
`Similarly, a confirmation of delivery status for a previously sent distribution not having been
`seen for a period of time, as indicated by applicable AppTerm variables, can be provided to the
`sending userin any of the variety of presentation types for the confirmation of delivery status. In
`some embodiments, processing of blocks 10120 through 10146 can be invoked at any time bya
`user,
`preferably with convenient user parameters for which MADRsto presentin the list (e.g. b
`application and/or use and/or any selections of MADR field(s) values). All COM-R event
`rocessing can be accomplished with AD tyoe MADR objects which are shared to target
`
`triggered according to a plethora of configurable event options. COM-R processin
`systems and
`is provided for convenience within context of a particular application event.
`
`6221A depicts a flowchart for describing a preferred embodimentof a-precedurefor
`FIG.
`
`
`
`
`
`
`
`
`
`OCMprocessing. Processing begins at
`:
`block 10202 where a call is made from, or received to, the MS of FIG. 21A processing, and
`
`continues to block 10204 where the user interfaces to the connected call. Thereafter, when
`focusing on this disclosure, block 10206 monitors and waits for OCM actions invoked by a user
`to the MS userinterface during the active call. Block 10206 also waits for call termination. When
`
`a monitored OCM action is detected or the call is terminated, block 10208 checksif an OCM
`ction was invoked.
`If block 10208
`determines
`an OCM was not invok'
`rocessing continues
`to block 10210. If block 10210 determines the call was not terminated, block 10212 handles
`other user
`actions which may
`have caused leaving
`block 102
`nd
`processing continues
`
`back
`
`closed)
`line
`(e.g. active
`call termination
`usual
`to block 10204, otherwise
`block 10214 and FIG. 21A processing terminates at block 620610216.
`
`i
`
`rformed at
`
`If block 10208 determines an OCM action was invoked during the active call by a user of the MS
`f FIG. 21A pr
`ing,
`block 10218
`checks
`origination of
`th
`ll. If
`block 1021
`rmin
`the user of the MS of FIG. 21A processing made the active call, processing continues to
`block 10220. If block 10220 determines a OGM delimiter (e.g. beep,
`period of silence
`
`EXHIBIT 1013 - PAGE 0258
`
`APPLE
`
`

`

`combination thereof, etc) should be waited for prior to automatically leaving a message,
`lock 10222 sets a PROMPTvariable to True and
`processing continues to block 1022
`
`continues to
`processing
`otherwise block 10224 sets the PROMPTvariable to False and
`block 10226. There are different embodiments for indication whether or not to wait for an OGM
`delimiter such as: user action indicated to wait, or not wait
`(user waits for OGM and prompt); MS
`assumes delimiter should be waited for, MS assumes a delimiter should not be waited for; MS
`determines at block 10204 whetheror not the MS useralready waited for a delimiter; or
`block 10220
`u
`a prior
`experience with the
`callee
`together with block 10204 processing
`
`for
`
`
`
`
`
`whether or not there isadelimiter to wait for. Block 10226 mutes the user's input (e.g. mutes
`
`audio
`
`to
`
`the
`
`MS microphone)
`
`and
`
`pr
`
`in
`
`ntinues
`
`to
`
`block 10228. Referring
`
`back t
`
`received the call
`that the user of the MS of FIG. 21A processing
`determined
`if it is
`lock 10218.
`then processing continues to block 10228 (e.g. to play MADR(s) during the active call for the
`benefit of the caller and/or callee without call termination).
`
`Block 10228 prepares needed parameters for spawning FIG. 21B thread processing
`concurrently in the background while FIG. 21A processing continues. Thereafter,
`block 10230 spawns the background thread with parameters and block 10232 determines what
`to_ do with the active call. If block 10232 determines the MS user of FIG. 21A processing is the
`originator of the call
`(i.e.
`caller),
`processing
`continues to block 10214 for
`
`losing/terminating EIG. 21A processing
`
`(e.g. includin
`
`ess to the
`
`active
`
`call line which
`
`block 10216.
`at
`terminates
`nd FIG. 21A processing
`kground thre
`active for the
`remains
`Whenarrived to by block 10232, block 10214 closes accessto the active call channel, but does
`not terminate the active call. The MS user is now free to use any MS application while the FIG.
`21B thread is running in the background. If block 10232 determines the user of the MS of FIG.
`21A processing is not the caller, processing continues back to block 10204 where the active call
`continues, for example to have the caller and callee observe the presentation of MADR
`information by the background thread.
`
`FIG. 21A is invoked in context of a specific active call including in a MS which supports a
`plurality of simultaneous active calls. FIG. 21A may be processed for each active call, and
`interfaces with the userfor that particular active call.
`
`FIG. 21B depicts a flowchart for describing a preferred embodiment of OCM background thread
`processing as invoked by block 10230. Thread processing begins at block 10250, continues to
`lock 10252 where parameters
`passed are determined
`(e.g. user action, PROMPT variable
`
`caller
`
`and
`
`callee
`
`information,
`
`active
`
`call line
`
`resource
`
`(channel) handle,
`
`etc),
`
`block 10254 for
`
`initialization processing
`
`(e.g. open accessto active call line resource), and to block 10256.
`
`action was invoked (best fit MADR
`passed for what user
`rameter
`Block 10256 will us
`default MADR, specific MADR) and will get the distinct MADR if requested, otherwise
`block 10256 accesses MADRsthat may
`berelevantto the active call
`(e.g. with a use
`field 9850 od containing OCM) and preferably
`sorts the MADRs found based on any
`ofthe field
`settings in the MADRs. Block 10256 may also use caller and/or callee information for
`termining
`relevant MADRs,
`for
`example
`in a MADR embodimentwhich inclu
`aller/call
`
`fields
`
`for
`
`rch. One MADR is
`
`preferably
`
`found
`
`if
`
`any
`
`are
`
`found,
`
`but none m
`
`found.
`
`should b
`any MADRs
`if
`to determine
`tin FIG. 19
`Boolean variabl
`Block 10256 also u
`searchedat all. Processing continues to block 10258. Block 10258 gets the next MADR for
`r
`ing
`and
`block 102
`heck
`if
`all MADRs
`found hav
`npr
`_lfall
`
`MADRs
`
`are
`
`_n
`
`r
`
`r
`
`in
`
`ntin
`
`lock 10262. Block 10262
`
`nothin
`
`EXHIBIT 1013 - PAGE 0259
`
`APPLE
`
`

`

`if the user selected a distinct MADR for processing,
`
`otherwise block 10262 determines the
`
`
`
`Boolean result for expression field referably in real-timebyevaluating the expression985
`
`
`
`block 10264 determines
`if
`le terms. Thereafter,
`li
`ess to
`an
`stack processing
`using
`expression field 9850 g evaluated to true or a default was found, then processing continues to
`block 10292, otherwise, processing continues back to block 10258. Block 10292 accesses any
`joined EFRs to the MADRin process. Block 10292 determines: a) no EFRsare joined; b) one or
`more EFR(s)
`joined do not match criteria and/or CLOC information for the active call: orc) all
`EFR(s)
`joined match the criteria and CLOC information for the
`active
`call.
`
`Block 10292
`
`compareslocation type EFRs to
`
`the CLOC
`
`parameterif needed,
`
`an
`
`mpares the
`
`for a
`event handling
`clarify
`EFRs
`_ Location t
`criteria ifn
`call
`to
`EFRs
`t
`r
`k
`certain location match. Keyword(s) types EFRsclarify event handling for certain associated
`keywords, for example as detected in a text stream produced after converting voice to text of
`the active call
`(e.g. caller uses voice command information to clarify MADR selection).
`Thereafter, if block 10294 determines the MADR does indeed match the requirements of OCM
`processing, then block 10266 checks the PROMPTvariable to see if an OGM delimiter(e.g.
`riod of
`silence,
`combination thereof,
`etc)
`should be waited for.If
`block 10266 determines an OGM prompt should be detected, then block 10268 waits fora
`promptindication or times out after a reasonable wait period, and processing continues to
`k 10270, Block 10270 resets
`the PROMPTvariable to
`False
`in
`there
`ar
`luralit
`MADRsfor processing. Thereafter, block 10272 invokes the presentMADR procedure with
`parameters for the MADR in process (or pointer thereof), the active call line resource (e.g.
`channel) information, and optionally sender(caller), recipient (callee) and CLOC information,
`and processing continues back to block 10258. If block 10266 determines a prompt should not
`e waited
`for, then processin
`ntinues
`directly
`to
`block 10270. If
`block 10294 determines
`one
`
`or more EFRs do not match parameters, then processing continues backto block 10258.
`
`EXHIBIT 1013 - PAGE 0260
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0261
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0262
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0263
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0264
`
`APPLE
`
`

`

`
`
`
`
`EXHIBIT 1013 - PAGE 0265
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0266
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0267
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0268
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0269
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0270
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0271
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0272
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0273
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0274
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0275
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0276
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0277
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0278
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0279
`
`APPLE
`
`

`

`
`
`EXHIBIT 1013 - PAGE 0280
`
`APPLE
`
`

`

`
`
`
`
`Referring back to block 681910260,if it is determined thatthe systemfororecessingistheMSof
`FIG-68A-precessinglast MADR was processed (or no MADRsfound), then processing continues
`
`
`
`to blockK 6846 for checkingwhich“Operanc™was passec:10274o274. If block 6816102740274 determines the
`“OQnearand”indicatactoinvokeflaunch)ananpronrigtaanplicationfortha ane
`
`
`
`
`
`
`EXHIBIT 1013 - PAGE 0281
`
`APPLE
`
`

`

`
`
`ller is the user
`of the MS
`of FIG. 21B processing,
`block 10276 un-mutes
`an
`muting by block 10226. block 10278 performscall termination processing, and the FIG.
`21B background
`thread terminates at
`block 10280,
`otherwise block 10274 continues to
`
`lock 10282 where thread accessto the active call line (e.g. ooened at
`terminated and block 10280 terminates processing
`
`block 10254)
`
`is
`
`A video
`
`call is anal
`
`usly
`
`proc
`
`d to
`
`audio
`
`by FIGS. 21A and 21B. Eor a video
`
`call
`
`lock 10226 mutes the user's input
`
`(audio
`
`and
`
`video)
`
`to
`
`the vide
`
`ll,
`
`presentWADR
`
`MADR can
`that a vi
`channel (i.e. not mixit
`all
`ing will interrupt a vi
`r
`presented withoutinterference. FIGS. 21A&B enable a MS user to automatically leave an audio
`and/or video messageat a callee's system, thereby freeing the MS user to perform other MS
`tasks while background processing automatically leaves the message. Further, a MS user does
`not have to wait for a callee's system OGM in orderto leave the message. The background
`thread waits for the appropriate moment to leave the message.
`
`With reference now to FIG. 22A, a simplified block diagram depicts a to preferred embodiment
`for mixing/multiplexing MADR data (e.g. an audio recording),
`or interrupting a video call with
`MADR video data, for an active call in progress. Output is mixed, multiplexed and/or switched at
`the MS for generating an output interface using one or more channels. Examplesof relevant
`.5,
`patents include:
`U.S. Pat. No.
`11,161 (“System and method for merging multiple
`audio
`
`5
`
`5
`
`“Multi-stream audio
`
`sampling
`
`rate conversion circuit and method”, Rosefield
`
`et al):
`
`. Pat. No. audio mixer’, Lemeetal):6,154,161 (“Integrated _ Pat. No. 7,119,267
`
`
`
`al): U.S. Pat. No. 7,039,178 (“System and method for generatin
`treams”, Anderson et
`simultaneous mixed audio output through a single output interface”, Case et al); U.S. Pat. No.
`7,369,071 “Anal and Gi
`ital si nal mixer” Polletet al); U.S. Pat. No. 5,647 008 -Method and
`
`No. 5,729,225 (“Method and apparatus for asynchronous digital mixing”, Ledzius); U.S. Pat. No.
`(“Portable mixing recorder and method and program for controlling the same”, Hirade et al):
`
`Pat, No. 6 744 815 (“Method for synchronizing audio and video streams”, Sackstein et al): U.S.
`Pat. No. 6,330,338 (“Process and device for mixing digital audio signals”, Von Owetal): U.S.
`Pat. No. 6,373,954 (“Single-chip audio circuitry, method, and systems using
`the same”
`Malcolm, Jr. et al); U.S. Pat. No. 5,694,467 (“Integrated sound/telephone headset system”,
`Young,
`Ill); U.S. Pat. No. 7,671,886 (“Video-phone terminal apparatus,
`image-shooting method
`
`
`
`
`
`
`
`
`
`nd_computer product” Wi : U.S. Pat. No. 7,555,141 (“Vi hone”, Mori); U.S. Pat. No.
`
`
`
`
`
`phone”, Du); U.S. Pat. No.
`protocol
`mmunication method of internet
`7,408,924 (“Vid
`6,704,040 (“Cellular phone set and remote control method used therefore”, Sato): U.S. Pat. No.
`6,788,322 (“Wireless imaging device and system”, Cook); and U.S. Pat. No. 6,192,257
`(“Wireless communication terminal having video image capability”, Ray); The preferred system
`and method preferably utilizes processing in context of FIG. 3 discussions, for example to
`ntrol
`adjusting
`volume
`or
`gain levels
`of
`r
`i
`in
`r
`ntrol flow of
`
`to channel(s) with appropriate data conversions, operate mixing and switching data streams, etc
`but other embodiments may be used.
`
`EXHIBIT 1013 - PAGE 0282
`
`APPLE
`
`

`

`1
`A MADR processing thread
`video play. Further
`determined
`
`determinesit is for audio play and/or
`2 accesses a MADR and
`is that the audio/video is to be
`presented on an active channel
`
`4 owns
`1
`progress. While control logic
`in
`(audio or video)
`call
`phone
`active
`for example an
`system resources, coordinates data processing of the MS, and provides the foundation for
`interoperation of data processing system components, MADR processing thread 10302 is
`driving disclosed OCM processing. Control logic interfaces to MADR processing
`thread 10302 (e.g. process/thread execution by
`the CPU), data gate 10306, data qate 10308
`data gate 10332, data gate 10334, mixer 10310
`and channel(s)
`10312 by way
`of appropriat
`
`ntrol
`
`paths 10314, for example bus/switch 54. MS microphone 10316
`
`also
`
`provides
`
`an
`
`audi
`
`urce,
`
`for
`
`example
`
`to
`
`talk over an active
`
`phon
`
`llin progress. Audio
`
`data from th
`
`pass through a converter, for example an analog to digital
`source may
`microphone 10316
`converter prior to being communicated to data gate 10308. Similarly, the MADR processing
`thread 10302 may interface to data gate 10306 over a path including a converter, for example a
`digital format converter for proper bit sampling, decompression, or other format changes. In
`some embodiments, mixer 10310 includes functionality eliminating the need for a converter.
`Data gat
`.g. 10306, 10308, 10332, 10334)
`are controlled for allowing
`the audio/video
`information to pass through to the mixer 10310, or prevent the audio/video information from
`passing through to the mixer 10310. Data gates can be used for “muting” data. Each data gate
`
`
`
`
`
`
`has canbeuits own FIFO queue buffer which for relayin ta when controlled to allow
`
`
`
`
`data to pass, and its own ability to discard data when controlled to prevent data from passing.
`Depending on embodiments, data gates may operate on analog ordigital data. Mixer 10310 can
`allow a single audio path to flow for output, or mixes a plurality of audio sources for a mixed and
`improved single audio signal containing the sources. Mixer 10310 preferably adjusts the relative
`volumeor gain levels when mixing multiple audio data sources. Audio is output to other data
`processing system 72 by way of at least one channel 10312. In some embodiments, another
`converter maylie on the data path between mixer 10310 and channel(s) 10312, or a converter
`may
`be downstream from the channel(s)
`10312 interface prior to transmission to
`tem 72. In
`some embodiments, converters shown are not necessary, for example whendigital data is in
`use throughout a data path. In some embodiments, converters may be used between a data
`gate and the mixer 10310. In some embodiments, mixer 10310 contains conversion capability
`€.g.
`remove need for converter(s)). In some embodiments, data gates are located at strategic
`interfaces in different components than shown. Depending on embodiments, a converter may
`
`
`igital-to-analog converter or any ofavariety of format converters (e.g. single data channel
`into a multiplexed data channel, or visa-versa). While two separate audio signal paths have
`been exemplified, those skilled in the art appreciate the basic de