-
`
`,
`
`.
`
`\
`\ \
`,x-
`\ -- \W
`
`“m“
`:
`
`.
`
`‘
`
`1,
`
`‘
`
`mmmmm
`\WW\§\W
`_
`f
`' "‘5 ‘1.
`\V‘.
`§\
`§_
`x}
`
`"
`
`$§§A
`
`15 N ”'13: D KIA HES DE PA R’EE‘M 11-3
`
`’1" (l1? (‘0 M M I-lRi ‘1‘)
`
`United States ’l-‘ment mid "I‘mdemurk Office
`
`September 21, 2017
`
`3.x
`" '
`'
`‘
`\
`xxx
`‘RWWW e~m§gm§§§kmxx
`'\
`W‘XW '
`\
`'
`'3
`W -
`Q:
`' w 1
`‘
`\33-\\:\3
`§
`§ \ ES §t §§§ § § V - >§
`WNWWW $31me '
`
`Patent anti ‘fi'adenmrk Office
`
`THIS IS TO CERTIFY THAT ANNEXED HERETO IS A TRUE COPY FROM
`THE RECORDS OF THE UNITED STATES PATENT AND TRADEMARK
`OFFICE OF THOSE PAPERS OF THE BELOW IDENTIFIED PATENT
`
`APPLICATION THAT MET THE REQUIREMENTS TO BE GRANTED A
`FILING DATE.
`
`APPLICATION NUMBER: 62/399,313
`FILING DATE: September 23, 2016
`RELATED PCT APPLICATION NUMBER: PCT/USI 7/5081 7
`
`THE COUNTRY CODE AND NUMBER OF YOUR PRIORITY
`
`APPLICATION, TO BE USED FOR FILING ABROAD UNDER THE PARIS
`CONVENTION, IS US62/399,313
`
`(Tm-filial by
`
`Under Secretary (11' Cmnmtrm:
`for '1 nieilwiua‘l Pmpcrt)
`and Director m‘the United Suites
`
`

`

`PATENT
`
`NETWORK TIMING SYNCI-[RONIZATION
`
`Page 1 of 34
`
`

`

`BACKGROUND
`
`Technical Field
`
`[0001] This disclosure relates generally to computing devices, and, more specifically, to
`
`synchronizing time among multiple devices communicating over a network.
`
`Description of the Related Art
`
`[0002] Various protocols have been developed to synchronize computers clocks over a
`
`computer network.
`
`In many instances, a protocol may facilitate distributing timing
`
`information provided from a source that maintains a master clock such as the atomic clock
`
`at the US. Naval Observatory. For example, one of the most notable protocols is the
`
`network time protocol (NTP) used to synchronize time among computers connected to the
`
`Internet. The timing information received via this protocol may be used to not only to
`
`display time on a user’s computer, but also coordinate performance of various operations
`
`such as distributing software updates to computers, displaying notifications, invoking a
`
`backup utility, time stamping emails, etc. As timing demands have changed, other network
`
`protocols have been developed to enable more precise time synchronization. For example,
`
`IEEE 802.1AS may be used in various audio and video applications to synchronize time
`
`among devices that are consuming or producing audio and video content.
`
`Page 2 of 34
`
`

`

`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0003] Fig. l is a block diagram illustrating an example of a network that implements time
`
`synchronization.
`
`[0004] Fig. 2 is a block diagram illustrating an example of a hardware security module
`
`configured to facilitate time synchronization.
`
`[0005] Figs.
`
`3A and 3B are diagrams
`
`illustrating examples of messages
`
`in
`
`a
`
`synchronization communication,
`
`[0006] Figs. 4A-4C are diagrams illustrating examples ofmessages in a propagation delay
`
`exchange.
`
`[0007] Fig. 5 is a diagram illustrating an example of an integrity check value field included
`
`in one or more of the messages.
`
`[0008] Fig. 6 is a diagram illustrating an example ofa nonce field included in one or more
`
`of the messages.
`
`[0009] Figs. 7A and 7B are communication diagrams illustrating examples of a time
`
`synchronization that
`
`includes a propagation delay exchange and a synchronization
`
`communication.
`
`[0010] Fig. 8 is a flow diagram illustrating an example of a bounds check for a
`
`synchronization communication.
`
`[0011] Figs. 9A and 9B are flow diagrams illustrating examples of other methods that may
`
`be performed by network components.
`
`Page 3 of 34
`
`

`

`[0012] Fig. 10 is a block diagram illustrating an exemplary computer system.
`
`[0013] This disclosure includes references to “one embodiment” or “an embodiment.” The
`
`appearances of the phrases “in one embodiment” or “in an embodiment” do not necessarily
`
`refer to the same embodiment. Particular features, structures, or characteristics may be
`
`combined in any suitable manner consistent with this disclosure.
`
`[0014] Within this disclosure, different entities (which may variously be referred to as
`73
`LL
`
`“units,
`
`circuits,” other components, etc.) may be described or claimed as “configured”
`
`to perform one or more tasks or operations. This fonnulation—[entity] configured to
`
`[perform one or more tasks]—is used herein to refer to structure (i.e., something physical,
`
`such as an electronic circuit). More specifically, this formulation is used to indicate that
`
`this structure is arranged to perform the one or more tasks during operation. A structure
`
`can be said to be “configured to” perform some task even if the structure is not currently
`
`being operated. A “node configured to communicate traffic over a network” is intended to
`
`cover, for example, a device that has circuitry that performs this function during operation,
`
`even if the device in question is not currently being used (e.g., a power supply is not
`
`connected to it). Thus, an entity described or recited as “configured to” perform some task
`
`refers to something physical, such as a device, circuit, memory storing program instructions
`
`executable to implement the task, etc. This phrase is not used herein to refer to something
`
`intangible. Thus, the “configured to” construct is not used herein to refer to a software
`
`entity such as an application programming interface (API).
`
`[0015] The term “configured to” is not
`
`intended to mean “configurable to.” An
`
`unprogrammed FPGA, for example, would not be considered to be “configured to” perform
`
`some specific function, although it may be “configurable to” perform that function and
`
`may be “configured to” perform the function after programming.
`
`[0016] Reciting in the appended claims that a structure is “configured to” perform one or
`
`more tasks is expressly intended n_ot to invoke 35 U.S.C. § 112(f) for that claim element.
`
`Accordingly, none of the claims in this application as filed are intended to be interpreted
`
`Page 4 of 34
`
`

`

`as having means-plus-function elements. Should Applicant wish to invoke Section 112(f)
`
`during prosecution, it will recite claim elements using the “means for” [performing a
`
`function] construct.
`
`[0017] As used herein, the terms “first,” “second,” etc, are used as labels for nouns that
`
`they precede, and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.)
`
`unless specifically stated. For example, an exchange between a first circuit and a second
`
`circuit may include the communication of multiple messages. The terms “first” and
`
`“second” can be used to refer to any two of messages in the exchange. In other words, the
`
`“first” and “second” messages are not limited to the initial two messages in the exchange,
`
`for example.
`
`[0018] As used herein, the term “based on” is used to describe one or more factors that
`
`affect a determination. This term does not foreclose the possibility that additional factors
`
`may affect a determination. That is, a determination may be solely based on specified
`
`factors or based on the specified factors as well as other, unspecified factors. Consider the
`
`phrase “determine A based on B.” This phrase specifies that B is a factor is used to
`
`determine A or that affects the determination of A. This phrase does not foreclose that the
`
`determination of A may also be based on some other factor, such as C. This phrase is also
`
`intended to cover an embodiment in which A is determined based solely on B. As used
`
`herein, the phrase “based on” is thus synonymous with the phrase “based at least in part
`77
`OH.
`
`[0019] As used herein, the term “synchronize” refers to calculating, for a first clock, an
`
`adjustment to be made to the first clock to determine the value of a second clock. While
`
`this term encompasses adjusting the value in the first clock to have the same time as the
`
`second clock, this term, for the sake of this disclosure, also encompasses not adjusting the
`
`value in the first clock. For example, “synchronizing” a first clock with a second clock
`
`may include calculating an adjustment for the first clock and applying the adjustment to a
`
`time value output by the first clock in order to determine a time value of the second clock.
`
`Page 5 of 34
`
`

`

`DETAILED DESCRIPTION
`
`[0020] A problem with traditional time synchronization schemes is that they typically do
`
`not take security concerns into consideration. This design failure may allow a malicious
`
`person to potentially interfere with a computing device’s perception of the current time,
`
`For example, the person may attempt to spoof a trusted time source by providing a message
`
`announcing an incorrect time. The person might also attempt to a replay message after a
`
`delay and have the recipient consider the replayed messages as being fresh (e.g., replaying
`
`the message with a delay that corresponds to the receiver’s offset clock).
`
`In doing so, the
`
`person may be able to interfere with scheduled operations performed by the device and
`
`potentially cause other problems.
`
`[0021] The present disclosure describes various techniques for improving the security of a
`
`time synchronization system.
`
`In various embodiments described below, a computer
`
`network implements a protocol for managing clock synchronization among network nodes
`
`having respective local clocks. This protocol may be used to periodically announce the
`
`current time of a network clock via communicating a first set of one or more messages
`
`from a master clock to a slave clock as well as determine offsets between the clocks via
`
`communicating a second set of messages in order to account for the propagation delays
`
`introduced by the network in distributing the announced current time. To improve the
`
`security of these messages, in some embodiments, an integrity check value (ICV) is added
`
`to each message. As used herein, the term “integrity check value” is to be interpreted
`
`according to its understood meaning in the art, and includes a value that is usable to verify
`
`the integrity of data and calculated by applying a keyed function (i.e., a function that uses
`
`a cryptographic key) to the data.
`
`Inclusion of this value may allow messages to be
`
`authenticated and resistant to tampering by a malicious entity. In some embodiments, one
`
`or more nonces are included in an exchange that is used to determine a propagation delay
`
`and then repeated in a subsequent announcement of the current time. As used herein, the
`
`term “nonce” is to be interpreted according to its understood meaning in the art, and
`
`includes an arbitrary and unpredictable number that is only used once in an operation or a
`
`set of operations. For example, a nonce may be determined using a random or pseudo
`
`Page 6 of 34
`
`

`

`random number generator algorithm.
`
`Inclusion of a nonce in this manner may prevent
`
`older announcements from being replayed by a malicious entity attempting to interfere with
`
`a node’s local time.
`
`In some embodiments, a network node may collect information for
`
`determining a propagation delay that is also usable to determine an expected offset between
`
`a local clock and the network clock. The node may then compare this expected offset with
`
`an offset calculated from a timestamp identifying the current time in a subsequent
`
`announcement. If the expected offset differs significantly from the calculated offset, the
`
`node may discard the announcement as it may be illegitimate.
`
`[0022] In some embodiments, messages associated With time synchronization may also be
`
`processed by secure circuits coupled to network nodes using the timing information (as
`
`opposed to the nodes themselves handling the processing). As used herein, the term
`
`“secure circuit” refers to a circuit that protects an isolated, internal resource from being
`
`directly accessed by an external entity. As will be described below,
`
`in various
`
`embodiments, a secure circuit may maintain the local clock for a network node and keys
`
`used to generate or verify ICVs included in messages.
`
`In some instances, using secure
`
`circuits to process messages (as opposed to the nodes) may provide additional security to
`
`the network.
`
`[0023] The present disclosure begins with a description of components in a secure network
`
`that implements time synchronization in conjunction with Figs.
`
`1 and 2. Network
`
`communications between nodes are described with respect to Figs. 3A—7B. Methods
`
`performed by network components are described with respect to Figs. 8—9B. Lastly, an
`
`exemplary computer system that may be used to implement one or more network
`
`components is discussed with Fig. 10.
`
`[0024] Turning now to Fig. 1, a block diagram of a network 100 configured to implement
`
`time synchronization is depicted.
`
`In the illustrated embodiment, network 100 includes a
`
`switch 110 coupled to multiple nodes 120A—C via links 112. Nodes 120A—B, in turn, are
`
`coupled via links 122 to respective hardware security modules (HSMs) 130A-B, which
`
`include local clocks l32A—B. As shown, node 120C is coupled to a grand master (GM)
`
`Page 7 of 34
`
`

`

`140, which includes a network clock 142. Switch 110 is also coupled to an HSM 130D,
`
`which includes a local clock 132D.
`
`In various embodiments, network 100 may be
`
`implemented differently than shown. Accordingly, in some embodiments, more (or less)
`
`switches 110 and/or nodes 120 may be present.
`
`In some embodiments, GM 140 may be
`
`coupled to a switch 110 (as opposed to a node 120).
`
`In some embodiments, functionality
`
`described herein with respect to HSMs 130 may be implemented by circuitry in nodes 120.
`
`[0025] Network 100, in some embodiments, is a local area network (LAN) configured to
`
`communicate network traffic among nodes 120.
`
`In various embodiments, network traffic
`
`is routed among nodes 120 by switches l 10 Accordingly, switches l 10 may be confi gured
`
`to queue received messages (i.e., data frames) from nodes 120 and analyze the source and
`
`destination addresses specifi ed by the frames in order to appropriately send the messages
`
`on to their specified destinations.
`
`In some embodiments, switches 110 are configured to
`
`route data frames in accordance with IEEE 802.3 (i.e., Ethernet Frames), however, in other
`
`embodiments, other networking protocols may be supported.
`
`[0026] Nodes 120 may correspond to any suitable devices configured to communicate over
`
`a network.
`
`In some embodiments, nodes 120 may be devices within a home or office
`
`network such as desktop and laptop computers, mobile devices, smart television, smart
`
`appliances, etc. In some embodiments, nodes 120 are machines within a fabrication plant
`
`that are configured to perform various operations.
`
`In some embodiments, nodes 120 are
`
`electronic control units (ECUs) in a vehicle such as an aircraft, boat, automobile,
`
`recreational vehicle, etc. As used herein, the term “electronic control unit (ECU)” is to be
`
`interpreted according to its understood meaning in the art, and includes an embedded
`
`system (e. g., microcontroller) that controls one or more operations of a vehicle.
`
`In some
`
`embodiments, nodes 120 may include, for example, a motor ECU that communicates
`
`torque—control messages and wheel—speed messages in order to control operation of a
`
`motor, a brake—system ECU that communicates brake control messages in order to apply
`
`braking, a LIDAR ECU that processes data received from one or more LIDAR sensors, a
`
`flight yoke ECU that communicates angles of the yoke controls, etc.
`
`Page 8 of 34
`
`

`

`[0027] In various embodiments, nodes 120 (and switch 110) are configured to use
`
`synchronized time for various purposes, including coordinating communication of traffic
`
`over network 100. For example, an ECU receiving streams from multiple LIDAR sensors
`
`may generate a holistic view of a vehicle’ 3 surroundings that is dependent on synchronized
`
`time among the ECU and its sensors delivering the streams. That is, first and second
`
`LIDAR sensors may timestamp their respective streams so that the ECU can determine
`
`what every sensor is viewing at a given time.
`
`If these timestamps are generated by
`
`unsynchronized clocks, the ECU might have a problem determining whether an object is
`
`present in front of the vehicle when one LIDAR sensor is indicating the presence of the
`
`object while another sensor gives the appearance that nothing is present in front of the
`
`vehicle because it
`
`included the wrong timestamp. As another example,
`
`in some
`
`embodiments, traffic communicated over network 100 is coordinated in accordance with a
`
`schedule that indicates when particular nodes 120 are supposed to communicate particular
`
`traffic.
`
`If time is not synchronized across nodes 120, node 120A might begin
`
`communicating network traffic during a time slot allocated to node 1203, for example.
`
`This collision could result in a node getting the wrong information or significantly delayed
`
`information, which may be associated with a time-critical task.
`
`[0028] Hardware security modules (HSMs) 130, in one embodiment, are secure circuits
`
`configured to provide time information 134 to nodes 120 for various purposes.
`
`In some
`
`embodiments, each HSM 130 includes a local clock 132 configured to maintain a value of
`
`the local time at that node 120, which is synchronized with network clock 142 by the HSM
`
`130. As will be described below in greater detail,
`
`in various embodiments,
`
`time
`
`synchronization for network 100 may be performed such that a given HSM 130
`
`synchronizes its local clock 132 with a neighboring node 120’s or switch 110’s HSM 130
`
`that resides on the path between the given HSM 130 and grand master 140, and is a hop
`
`closer to grand master 140. Accordingly, HSMs 130A and HSMs 130B may synchronize
`
`their local clocks 132A and 132B with local clock 132D in HSM 130D being coupled to a
`
`node 120 one hop closer to node 120C and grand master 140. HSM 130D, in turn, may
`
`synchronize with network clock 142 in grand master 140 as there are no other closer nodes
`
`120 in Fig. 1. Said differently, time synchronization of network 100 may be viewed as a
`
`Page 9 of 34
`
`

`

`tree structure in which a given child node synchronizes with its parent node, and grand
`
`master 140 functions as the root node in the tree structure.
`
`[0029] When synchronization between clocks of two HSMs 130 is performed, the HSM
`
`130 providing the timing information (e.g., HSM 130D) may be referred to as the master
`
`while HSM 130 receiving the timing information (e.g., HSMs 130A and 130B) may be
`
`referred to as the slave. As part of performing a synchronization, in various embodiments,
`
`a slave may determine a propagation delay between it and its master (e.g., the delay
`
`between HSM 130A and HSM 130D). The slave may also determine an offset between its
`
`local clock 132 and master’s clock (e.g., the offset between clocks 132A and 1321)). Once
`
`these values have been determined, the HSM 130 may then provide information 134 to its
`
`node l20, where the timing information includes a time value associated with the current
`
`value of network clock 142 and is calculated based on the value of local clock 132, the
`
`propagation delay, and the determined offset.
`
`In various embodiments, HSMS 130 are
`
`relied on for maintaining local time and synchronizing time (as opposed to nodes 120) as
`
`HSMs 130 may employ various security techniques making them more secure than nodes
`
`120 as will be described below with respect to Fig. 2.
`
`[0030] Grand master (GM) 140, in one embodiment, is a circuit configured to maintain
`
`network time for network lOO—i.e., the reference time to which local clocks 132 are
`
`synchronized. That is, while HSM 132D may act as a master to HSM 130A and HSM
`
`130B, GM 140 is the ultimate authority (i.e., “grand master”) in the illustrated embodiment.
`
`As noted above, in the illustrated embodiment, GM 140 includes a network clock 142
`
`configured to track the current value of network time. In some embodiments, GM 140 may
`
`be one of several master clocks and is selected as the grand master through an election
`
`process. As noted above, in some embodiments, GM 140 may be coupled to switch 110
`
`(as opposed to node 120E as shown in Fig. 1) and/or may implement functionality of an
`
`HSM 130 discussed below.
`
`In various embodiments, GM 140 is also configured to
`
`facilitate
`
`time
`
`synchronization with HSM 130D via
`
`synchronization
`
`(sync)
`
`communications 152 and propagation delay (Pdelay) exchanges 154.
`
`(Although
`
`communications 152 and exchanges 154 are shown only between GM 140 and HSM 130D
`
`Page 10 of 34
`
`

`

`in Fig. l to simplify the diagram, communications 152 and exchanges 154 are also provided
`
`from HSM 130D to HSMs 130A and 130B as well.)
`
`[0031] Sync communications
`
`152,
`
`in the illustrated embodiment,
`
`are messages
`
`communicated by a master (e. g., GM 140 or HSM 130D) to announce the current value of
`
`network time as identified by its synchronized clock 132 or network clock 142. In various
`
`embodiments, a given sync communication 152 includes a timestamp identifying the
`
`current value of network time when a sync communication 152 is sent from the master and
`
`is usable by a given slave to determine the offset (i.e., difference) between the slave’ 5 local
`
`clock 132 and the master’s clock 132 or clock 142.
`
`In various embodiments,
`
`the
`
`determined offset is both 1) the time difference between the master’s clock and the slave’s
`
`clock and 2) the frequency difference between master’s clock and the slave’s clock as they
`
`may have slightly different frequencies in some instances. Sync communications 152 may
`
`be sent on a periodic basis (e.g., eight times a second) as the individual frequencies of
`
`clocks 132 and 142 may fluctuate with temperature changes.
`
`In some embodiments, a
`
`given sync communication 152 includes two messages: a sync message (discussed below
`
`with Fig. 3A) that is sent at a particular time and a follow up message (discussed below
`
`with Fig. 3B) that includes a timestamp specifying the value of network time
`
`at the
`
`particular time—i.e., the time at which the sync message was sent. In some embodiments,
`
`these messages include the content of IEEE 802.1AS frames.
`
`[0032] Pdelay exchanges 154, in the illustrated embodiment, are an exchange between a
`
`master and slave in order to determine a propagation delay between the master and slave.
`
`That is, because it takes time for a message to traverse network 100, a timestamp specified
`
`in a sync communication 152 is delayed by the time it arrives at an HSM 130. To account
`
`for this, the HSM 130 determines the propagation delay and residence time to appropriately
`
`adjust the timestamp in the sync communication 152 so that it accurately reflects the
`
`current network time.
`
`In some embodiments, an exchange 154 includes a slave sending a
`
`message to a master (referred to below and in Fig. 4A as a Pdelay request) and receiving a
`
`corresponding response from the master (referred to below and in Fig. 4B as a Pdelay
`
`response). The propagation delay may then be determined by 1) calculating the difference
`
`Page 11 of 34
`
`

`

`between the arrival time of the first message and its departure time, 2) calculating the
`
`difference between the arrival time of the second message and its departure time, and 3)
`
`determining an average of the differences. Notably, the arrival and departure times of both
`
`messages are used in order to account for the fact that the arrival and departure timestamps
`
`are supplied by the master’s clock and the slave’s clock, which may be unsynchronized as
`
`noted above. For example, HSM 130D may record the departure time of the first message
`
`and the arrival time of the second message based on local clock 132D, but rely on GM 140
`
`to provide timestamps for the arrival time of the first message and the departure time of the
`
`second message, which are based on network clock 142. In some embodiments, the second
`
`message may include both ofthese timestamps. In other embodiments, the second message
`
`includes the timestamp for the arrival time of the first message, and a third message
`
`(referred to below and in Fig. 4C as Pdelay follow-up) is sent that specifies the timestamp
`
`for the departure time of the second message.
`
`In some embodiments, Pdelay exchanges
`
`154 are performed periodically (e.g., once per second) as the propagation delays may
`
`change as the temperature oflinks 112 and 122 change.
`
`In some embodiments, messages
`
`communicated in an exchange 154 include the content of IEEE 802.1AS frames.
`
`[0033] To improve the security of sync communications 152 and Pdelay exchanges 154,
`
`in various embodiments, HSMs 130 and GM 140 are configured to perform various
`
`techniques to make them more resistant to tampering by a malicious entity.
`
`In some
`
`instances, these techniques may protect an HSM 130 if node 120 becomes compromised
`
`and begins spoofing that HSM 130’s master or GM 140 by broadcasting its own sync
`
`communications 152. These techniques may also protect against man—in—the—middle
`
`attacks such as switch 1 10 becoming compromised and attempting to replay messages from
`
`an old sync communication 152 or Pdelay exchange 154.
`
`[0034] First, in some embodiments, an integrity check values (ICV) is added to one or
`
`more messages sent between GM 140 and HSMs 130 during sync communication 152 and
`
`a Pdelay exchange 154. This ICV may be calculated using any of various algorithms. For
`
`example, in one embodiment, GM 140 and HSMs 130 are configured to calculate ICVs
`
`using a message authentication code (MAC) algorithm such as the Hash—based MAC
`
`Page 12 of 34
`
`

`

`(HMAC) algorithm.
`
`In another embodiment, ICVs are digital signatures calculated using
`
`the Digital Signature Algorithm (DSA). In still another embodiment, ICVs include a MAC
`
`encrypted using the Advanced Encrypted Standard (AES) in cipher block chaining (CBC)
`
`mode. In some embodiments, the key used to determine the ICV is unique to a given HSM
`
`130 participating in the communication and known to that HSM 130 and its master. For
`
`example, if HSM 130D is sending a first message to HSM 130A and a second message to
`
`HSM 130B, the ICV in the first message is calculated using a first key, and the ICV in the
`
`second message is calculated using a second key different from the first key. As a result,
`
`HSM 130A and 130B may be able to verify not only the integrity of a received message,
`
`but also authenticate the message because HSM 1301) is the only other entity with the key
`
`unique to that HSM 130. Accordingly, if an HSM 130 receives a message including an
`
`ICV and is unable to determine that the ICV is valid, the message is suspect and may be
`
`disregarded by the HSM 130. In doing so, HSMs 130 and GM 140 may protect against an
`
`entity attempting to spoof messages from GM 140 or a particular HSM 130. Examples of
`
`messages including an ICV are discussed below with respect to Figs. 3A-5.
`
`[0035] Second, in some embodiments, an HSM 130 is configured to provide a nonce
`
`during a Pdelay exchange 154, which is repeated by its master during one or more
`
`sub sequent sync communications 152. (In some embodiments, the master may also include
`
`an additional nonce in the exchange as will be discussed with Fig. 7B.) In doing so, HSM
`
`130 and GM 140 may prevent a malicious entity from replaying sync communications 152
`
`sent prior to providing the nonce as they would lack the nonce. Accordingly, if the HSM
`
`130 receives a sync communication 152 including a nonce that is different from the earlier
`
`provided one, the HSM 130 may disregard the communication 152 as suspect. Examples
`
`of messages including a nonce are discussed below with respect to Figs. 3A, 3B, 4A, and
`
`6.
`
`[0036] Third, in some embodiments, HSMs 130 are configured to determine an expected
`
`offset during a Pdelay exchange 154 and use the expected offset to perform a bounds check
`
`on an offset calculated from a given sync communication 152 in order to determine whether
`
`that sync communication 152 is valid. Although a Pdelay exchange 154 is performed
`
`Page 13 of 34
`
`

`

`primarily to determine a propagation delay as noted above, it is possible to determine an
`
`offset between a master’s clock and a slave’s clock from information obtained in
`
`determining the delay.
`
`In particular, this offset may be determined by 1) calculating an
`
`average of the first message’s departure time and the second message’s arrival time, 2)
`
`calculating an average of the first message’s arrival
`
`time and the second message’s
`
`departure time, 3) determining the difference between the two averages such that the
`
`difference is the offset. In such an embodiment, HSM 130 is configured to determine this
`
`offset and compare it against an offset calculated based on a timestamp included in a
`
`subsequent sync communicate 152. If the calculated offset differs from what was expected
`
`by more than a particular threshold, the timestamp included in the sync communication
`
`152 is suspect, As a result, an HSl\/I 130 may disregard the sync communication 152 and
`
`not attempt to provide time information 134 based on the suspect timestamp.
`
`In doing so,
`
`HSMs 130 may protect against spoofing and/or a man-in-the-middle attack. An example
`
`of a method for performing a bounds check is described in greater detail with respect to
`
`Fig. 8.
`
`[0037] Turning now to Fig. 2, a block diagram of an HSM 130 is depicted. As noted above,
`
`in some embodiments, HSMs 130 are used for timing keeping and synchronization in lieu
`
`of nodes 120 because HSMs 130 may be more secure for the reasons discussed below. As
`
`also noted, in some embodiments, GM 140 may implement functionality described with
`
`respect to HSM 130 in order to improve GM 140’s security. In the illustrated embodiment,
`
`HSM 130 includes a network interface 210, one or more processors 220, a read only
`
`memory (ROM) 230, local clock 132, a cryptographic accelerator 240, and a key storage
`
`250 coupled together via an interconnect 260. ROM 230 also includes firmware 232. Key
`
`storage 250 also includes one or more ICV keys 252.
`
`In some embodiments, HSM 130
`
`may include more (or less) components than shown.
`
`[0038] Network interface 210,
`
`in one
`
`embodiment,
`
`is
`
`configured to facilitate
`
`communication with a node 120. Accordingly, interface 210 may perform encoding and
`
`decoding data across a link 122, which,
`
`in some embodiments,
`
`is a serial peripheral
`
`interface (SPI) bus.
`
`In various embodiments, interface 210 is also configured to isolate
`
`Page 14 of 34
`
`

`

`internal components 220-250 from an external entity such as node 120, by filtering
`
`incoming read and write operations.
`
`In some embodiments, HSM 130 presents a limited
`
`attack surface by supporting only a small number of commands. For example, in one
`
`embodiment, HSM 130 supports a command allowing node 120 to request the current time
`
`and a command to provide data for a sync communication 152 or Pdelay exchange 154. If
`
`interface 210 receives data from a node 120 that is not a supported command, interface 210
`
`may prevent the data from entering HSM 130.
`
`In doing so, HSM 130 prevents, for
`
`example, node 120 from being able to access local clock 132 directly.
`
`[0039] Processor 220, in one embodiment, is configured to execute program instructions
`
`to implement various operations described herein with respect to HSM 130.
`
`In some
`
`embodiments, processor 220 is hardwired to fetch from a specific address range at boot in
`
`order to boot firmware 232 from ROM 230. Notably, because memory 230 is a ROM (as
`
`opposed to some other type of memory that can easily be written to), firmware 232 is
`
`resistant to modification, and thus, being tampered with. As a result, HSM 130 can be
`
`restored to a default, trusted state by merely causing processor 220 to reboot, which, in the
`
`illustrated embodiment, can be initiated by asserting reset signal 202. Thus, processor 220
`
`may further serve to isolate components in HSM 130.
`
`[0040] Cryptographic accelerator 240,
`
`in one embodiment,
`
`is circuitry configured to
`
`perform cryptographic operations for HSM 130. Cryptographic accelerator 240 may
`
`implement any suitable encryption algorithm such as Data Encryption Standard (DES),
`
`Advanced Encryption Standard (AES), Rivest Shamir Adleman (RSA), HMAC, etc.
`
`In
`
`the illustrated embodiment, accelerator 240 is configured to use keys stored in key storage
`
`250, which accelerator 240 may isolate from being accessed by other components of HSM
`
`130. That is, in some embodiments, accelerator 240 may allow an ICV key 252 to be
`
`updated by processor 220, but not allow the key 252 to be read from storage 250 by
`
`processor 220.
`
`[0041] In various embodiments, cryptographic accelerator 240 is configured to use ICV
`
`key 252 to verify ICVs included sync communications 152 and Pdelay exchanges 154.
`
`Page 15 of 34
`
`

`

`Accordingly, accelerator 240 may be configured to receive a message, calculate an ICV
`
`using ICV key 252, and compare the calculated ICV with the ICV included in the received
`
`message. If the two ICVs do not match,