JOURNAL OF AEROSPACE INFORMATION SYSTEMS
`Vol. 12, No. 3, March 2015
`
`Simultaneous Tracking of Multiple Ground Targets
`from a Multirotor Unmanned Aerial Vehicle
`
`Nathaniel Miller∗
`Texas A&M University, College Station, Texas 77843
`and
`Jonathan Rogers†
`Georgia Institute of Technology, Atlanta, Georgia 30332
`
`DOI: 10.2514/1.I010345
`
`An algorithm for simultaneous tracking of multiple ground targets by an unmanned aerial vehicle is presented. The
`algorithm is specifically tailored toward multirotor vehicles, and it consists of a particle filter to predict target motion,
`a reference trajectory generator, and a finite-horizon model predictive controller for trajectory tracking. Two
`versions of the algorithm are proposed: for a vehicle equipped with a gimbaled camera, and for a vehicle equipped
`with a fixed camera. Furthermore, a target rejection algorithm is included to prune targets that inhibit accurate
`tracking of the majority of the target set. The tracking algorithm and multirotor vehicle dynamic model are first
`described, followed by example simulations for both the gimbaled and fixed-camera cases. Trade studies are
`presented, analyzing the effects of controller tuning parameters and camera field of view, as well as performance of the
`target rejection algorithm. In simulation experiments, real-world target data are used to improve simulation fidelity.
`Overall, results show that the algorithm is effective in capturing a set of ground targets within the field of view
`simultaneously when using either gimbaled or fixed-camera configurations, but performance is somewhat degraded
`in the fixed-camera case when target dynamics occur on timescales similar to tracking vehicle dynamics.
`
`, alon
`alat
`i
`i
`alon
`, alat
`i
`i
`~cdx, ~cdy
`FDx, FDy, FDz
`g
`H
`IB, JB, KB
`II, JI, KI
`Mx, My, Mz
`MC
`m
`p, q, r
`Q, R
`ri
`S
`T
`vx, vy, vz
`v∕I
`v0
`i
`vi
`xCG, yCG, zCG
`xcam, ycam
`~x, ~y, ~z
`~xcam, ~ycam
`
`i …0†, y i …0†
`x
`wx, wy, wz
`w1, w2, w3
`zi
`ρ
`σlon
`i
`σv
`i
`
`, σlat
`i
`
`Nomenclature
`lateral and longitudinal accelerations of the ith target
`=
`estimated longitudinal and lateral accelerations of target i
`=
`= multirotor induced drag coefficients
`=
`induced drag force components in the inertial frame
`=
`gravitational acceleration
`=
`prediction horizon
`=
`unit vectors of the body-fixed reference frame
`=
`unit vectors of the inertial reference frame
`=
`body-frame components of the total external moment acting on the multirotor
`=
`total external moment on the multirotor about its mass center
`= mass of the multirotor vehicle
`=
`body-frame components of the angular velocity of the body frame with respect to the inertial frame
`= model predictive control weighting matrices
`distance from jth target to centroid of other targets
`=
`=
`rotor disk area
`=
`thrust force
`=
`inertial-frame components of the velocity of the mass center with respect to the inertial frame
`=
`velocity of the multirotor mass center
`estimated velocity of target i at beginning of planning horizon
`=
`= measured velocity of the ith target
`= multirotor mass center position in inertial frame
`=
`position of center of camera frame in inertial frame
`=
`desired multirotor position
`=
`desired position of center of camera frame in inertial frame
`= measured position of target i at beginning of planning horizon
`= wind velocity components in inertial frame at the current multirotor position
`= weighting values for target rejection score
`=
`center of discretized altitude interval
`=
`air density
`estimated standard deviation of uncertainty in alon
`=
`i
`estimated standard deviation of uncertainty in v0
`=
`i
`
`and alat
`i
`
`Presented as Paper 2014-2670 at the AIAA Atmospheric Flight Mechanics Conference, Atlanta, GA, 16–20 June 2014; received 6 October 2014; revision
`received 19 February 2015; accepted for publication 20 February 2015; published online 17 March 2015. Copyright © 2015 by the American Institute of
`Aeronautics and Astronautics, Inc. All rights reserved. Copies of this paper may be made for personal or internal use, on condition that the copier pay the $10.00 per-
`copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923; include the code 2327-3097/15 and $10.00 in correspondence with the
`CCC.
`*Graduate Research Assistant, Department of Aerospace Engineering.
`†Assistant Professor, Woodruff School of Mechanical Engineering.
`
`345
`
`Downloaded by Michael Daniels on August 3, 2016 | http://arc.aiaa.org | DOI: 10.2514/1.I010345
`
`Yuneec Exhibit 1014 Page 1
`
`

`
`346
`
`MILLER AND ROGERS
`
`c
`
`σψ
`i
`τϕ, τθ, τT
`ϕ, θ
`ϕ
`c, θ
`ϕg, θg
`~ϕ, ~θ
`ψ
`i…0†
`ψ
`ωB∕I
`
`i
`
`i…0†
`
`estimated standard deviation of uncertainty in ψ
`=
`= multirotor vehicle response time constants
`= multirotor Euler roll and pitch angles
`=
`commanded multirotor Euler roll and pitch angles
`=
`gimbal angles
`=
`desired multirotor roll and pitch angles
`course over ground of the ith target
`=
`estimated heading of target i at beginning of planning horizon
`=
`=
`angular velocity of the body with respect to inertial frame
`
`I.
`
`Introduction
`
`T HERE is increasing interest in the ability to task low-cost airborne assets to autonomously track ground targets: specifically through video
`
`feed. Such autonomous tracking capabilities may allow air vehicles to perform monitoring and surveillance activities without operator
`control, which can be especially burdensome in cases involving large numbers of targets and/or tracking vehicles. Consider a scenario in which
`multiple ground targets must be tracked and surveilled simultaneously by a fleet of unmanned air vehicles. Such a scenario may arise in urban
`combat settings, where a fleet of aircraft seeks to monitor enemy armored vehicles throughout a city. In an alternative scenario, a group of aircraft
`may wish to provide overhead surveillance for a convoy of ground vehicles. In these scenarios, one air vehicle may be responsible for simul-
`taneously tracking multiple targets. This tracking problem primarily becomes one of vehicle guidance and path planning, since the vehicle flight
`path must permit onboard sensors to track multiple ground targets that may be uncooperative. As ground targets move farther apart, the aircraft
`must increase altitude to ensure the targets remain within sensor fields of view. In addition, the air vehicle must continuously search for the optimal
`position that minimizes the probability of targets being obscured or unexpectedly maneuvering out of the field of view. Thus, the vehicle path-
`planning problem and the tracking solution must be compatible with sensor characteristics, vehicle dynamic constraints, and runtime limitations.
`Few solutions currently exist for this type of multitarget tracking problem, and tactical unmanned aerial vehicles (UAVs) are often restricted to
`tracking and engaging one target at a time.
`An extensive body of literature exists on automated airborne target tracking algorithms. Video-based tracking of a single target has been well
`studied [1,2] but such methods do not easily generalize to the simultaneous multitarget case. Similarly, the problem of tracking multiple targets
`from multiple aerial platforms has been studied somewhat extensively during the past decade (see, for instance, [3–7]). In many ways, these
`algorithms are generalizations of single-target single-tracker algorithms, in which the number of trackers needed is approximately the same as the
`number of targets. For high-altitude missions tasked with tracking multiple targets in a confined area, such as the predator scenario discussed in
`[8], the multitarget tracking problem from a single vehicle can be easily solved by flying standard surveillance patterns and determining optimal
`pointing solutions for onboard sensors. The problem becomes fundamentally different and far more difficult, however, when tracking is done with
`tactical low-altitude UAVs with onboard low-cost sensors of limited range and resolution. In this case, the vehicle must constantly climb, descend,
`and reposition to ensure adequate visual range and resolution with low-cost sensors. Limited work has also addressed the multiple target tracking
`problem when simultaneous tracking of all targets is not a priority [9,10]. Such solutions may fail if targets move unpredictably while not under
`observation.
`This paper presents a guidance and control algorithm for tracking an arbitrary number of ground targets from a single multirotor. For the
`purposes of algorithm development, it is assumed that target vehicle positions are known, and the control system is tasked to keep all targets within
`the video field of view to the maximum extent possible given dynamic constraints and uncertainty in target motion. Another critical assumption
`underlying the proposed work is that the tasks of pointing the sensor and determining target motion from the sensor feed are assumed to be
`completed by external algorithms separate from the controller derived here. Both of these topics have been well studied [11–14]. Given these
`assumptions, the algorithm described here focuses on vehicle guidance and control for the multirotor aircraft. The tracking algorithm is con-
`structed by first generating a reference trajectory given predicted target motion. Based on the reference trajectory, a receding-horizon optimal
`control problem is formulated and a linear model predictive control algorithm is generated assuming linearized vehicle dynamics. A particle filter
`is used to predict target motion over the receding control horizon. In addition, a target rejection algorithm limits the tracking vehicle altitude below
`a prescribed ceiling, ensuring that tracking resolution of the total target set is not sacrificed in order to track a handful of rogue targets.
`The paper proceeds as follows. First, the multitarget tracking problem is formulated and a linear dynamic model for the multirotor vehicle is
`described. The proposed guidance algorithm is outlined including the reference trajectory generator, model predictive controller, and target
`motion filter. A method of target rejection is also proposed in which a rejection score for each target is calculated as a weighted sum of various
`metrics. Following a brief discussion of system identification for the simulation model, example simulation results are presented using experi-
`mentally obtained data for walking, biking, and driving targets. Simulation results are provided for both gimbaled and fixed-camera cases. Finally,
`trade studies are performed examining the effects of camera field of view, target rejection, and gimbaled vs fixed-camera performance. Overall,
`results demonstrate that the algorithm enables simultaneous multitarget tracking performance for a broad range of ground target types and is
`robust to uncooperative target motion.
`
`II. Tracking Algorithm
`Figure 1 depicts a general multiagent, multitarget tracking problem formulated by considering M agents and N targets. Classical solutions to
`this problem typically involve two stages: a so-called asset assignment stage, in which targets are assigned to (or selected by) agents for tracking;
`
`A3
`
`A1
`
`A2
`
`T6
`
`T2
`
`T3
`
`T9
`
`T5
`
`T1
`
`T7
`
`T8
`
`T4
`
`T10
`
`T11
`
`Fig. 1 General multiagent, multitarget tracking problem. An asset assignment algorithm (not considered here) is used to assign targets Ti to agents Ai.
`
`Downloaded by Michael Daniels on August 3, 2016 | http://arc.aiaa.org | DOI: 10.2514/1.I010345
`
`Yuneec Exhibit 1014 Page 2
`
`

`
`MILLER AND ROGERS
`
`347
`
`and the tracking stage, in which agents actually carry out the tracking task. The asset assignment problem has been well studied, and numerous
`methods now exist to perform target assignment in a decentralized manner, including market-based approaches [15], distributed optimization
`[16], and game theory [17], among others. In this paper, it is assumed that target assignment has already taken place (or continues to take place
`externally), leaving the individual vehicle to track its specific set of targets. Thus, the focus of this paper is the control algorithm onboard each
`agent that allows it to track multiple targets simultaneously. An implicit assumption invoked here is that asset assignment and target tracking can
`be completely decoupled. In practice, some coupling between these two layers may be advantageous, since targets that are difficult to track by one
`agent may be transferred to another, improving overall tracking of the target set. Such coupling considerations are not explored in this work.
`The tracking algorithm proposed here consists of four components. First, a particle filter estimates target motion over a finite horizon using a
`simple constant-acceleration kinematic model. Given these predicted target paths, a trajectory generator constructs a desired three-dimensional
`reference trajectory for tracking by the airborne agent. A finite-horizon linear model predictive control (MPC) scheme is used to track this
`trajectory. Finally, a target rejection algorithm determines if any targets are driving the tracking vehicle to altitudes in excess of a prescribed ceiling
`or reducing tracking performance for the majority of targets. Such rogue targets are removed from the target set through a mathematical rejection
`criterion. A diagram of this control architecture is shown in Fig. 2. Detailed descriptions of each algorithm component are provided in this section
`following a discussion of the multirotor vehicle dynamic model.
`
`equations of motion for the multirotor are given by
`
`A. Nonlinear Multirotor Dynamic Model
`The multirotor dynamic model considered in this paper is a five-degree-of-freedom model consisting of three translational degrees of freedom
`and two rotational degrees of freedom. Due to the inherent symmetry of the multirotor vehicle, yaw is neglected in the equations of motion.
`Figure 3 shows a diagram of the reference frames used in development of the dynamic model, namely, the inertial frame I and the body-fixed frame
`B. The inertial frame makes use of a flat-Earth approximation, and the B frame is centered at the multirotor mass center. The B frame is obtained
`from the I frame by rotating first about JI by the pitch angle θ, and then, about IB by the roll angle ϕ. This geometry is depicted in Fig. 3.
`Three forces are assumed to act on the multirotor: weight −mgKI, thrust TKB, and drag. Let the velocity of the mass center with respect to the I
`frame be given by v∕I ˆ vxII ‡ vyJI ‡ vzKI, and let the angular velocity of the B frame with respect to I be given by ωB∕I ˆ pIB ‡
`
`qJB ‡ rKB. Drag acts at the mass center and is given by FD ˆ FDxII ‡ FDyJI ‡ FDzKI. Furthermore, assume that control is available through
`pure torques generated by differential thrust, leading to moments MC ˆ MxIB ‡ MyJB ‡ MzKB about the mass center. Therefore, the nonlinear
`8<
`8<
`9=
`9=
`: vx
`: _xCG
`; ˆ
`;
`
`_yCG
`_zCG
`
`vy
`vz
`
`(1)
`
`
`
`
`
`
`
`_ϕ
`_θ
`
`ˆ
`
`p ‡ sin…ϕ† tan…θ†q ‡ cos…ϕ† tan…θ†r
`cos…ϕ†q − sin…ϕ†r
`
`
`
`(2)
`
`Y
`
`Reference
`Trajectory
`Generator
`
`Model
`Predictive
`Controller
`
`ku
`
`Multirotor
`Plant
`
`k
`
`,
`x y
`i
`
`i
`
`for
`
`t
`
`∈
`
`[
`t
`
`,
`
`t +
`k K
`
`k
`
`]
`
`Target State
`Estimator
`(Particle Filter)
`
`Fig. 2 Multitarget tracking guidance and control architecture.
`
`BK
`
`BJ
`
`BI
`
`IK
`
`IJ
`
`II
`
`zCG
`
`yCG
`
`xCG
`xcam
`Fig. 3 Reference frame definitions for multirotor dynamic model.
`
`ycam
`
`Downloaded by Michael Daniels on August 3, 2016 | http://arc.aiaa.org | DOI: 10.2514/1.I010345
`
`Yuneec Exhibit 1014 Page 3
`
`

`
`348
`
`8<
`: _vx
`_vy
`_vz
`
`9=
`; ˆ 1
`
`m
`
`MILLER AND ROGERS
`
`24
`
`9=
`; ˆ ‰IBŠ−1
`
`8<
`: _p
`
`_q
`_r
`
`8<
`: Mx
`
`My
`Mz
`
`(3)
`
`(4)
`
`35
`
`9=
`;
`
`9=
`;
`
`q r
`
`35
`
`−r
`0
`q
`0 −p
`r
`−q p
`0
`
`24
`
`8<
`: T cos…ϕ† sin…θ† ‡ FDx
`−T sin…ϕ† ‡ FDy
`T cos…ϕ† cos…θ† ‡ FDz − mg
`8<
`9=
`: p
`; −
`‰IBŠ
`
`where IB is the moment of inertia matrix of the vehicle about its mass center expressed in the B frame.
`For multirotor vehicles, the primary contribution to drag in forward flight is caused by the thrust imbalance on the advancing rotors, giving rise
`to a phenomenon known as induced drag [18]. Induced drag is typically modeled as a linear function of translational velocity components in the
`8<
`9=
`8<
`x ≡ vx − wx, with analogous definitions for vwy and vwz . Following [18], induced drag is given by
`
`body frame. Define vw
`
`z † ‡ cdy…s2
`: cdx…c2
`
`x ‡ sϕsθcϕvwy ‡ s2
`z †
`− sθcθvw
`ϕs2θvw
`
`; ˆ −T
`: FDx
`ϕsθcθvw
`θvw
`x
`
`cdy…sϕsθcϕvwx ‡ c2
`
`y ‡ sϕcθcϕvwz †
`ϕvw
`FDy
`
`z † ‡ cdy…s2
`
`x ‡ sϕcθcϕvwy ‡ s2
`z †
`cdx…−sθcθvwx ‡ s2
`θvw
`ϕsθcθvw
`ϕsθcθvw
`FDz
`where cdx and cdy are the induced drag coefficients in the IB and JB directions, and sα ≡ sin…α†, cα ≡ cos…α†. The trigonometric quantities in
`
`
`
`Eq. (5) arise from the fact that vwx , vwy , vwz and FDx, FDy, FDz are defined as components in the inertial frame for compatibility with the model in
`Eq. (3). Although other aerodynamic effects act on the vehicle (i.e., parasite drag on the vehicle body, etc.), these components are usually neglected
`in multirotor modeling, as they are typically dominated by high-gain control action [18]. Note that several authors have demonstrated high-
`performance control of quadrotor vehicles using a simple static thrust model without accounting for any sources of drag [19,20].
`For the case of a fixed (nongimbaled) camera, one additional quantity of interest for the tracking control law is the location of the center of the
`camera field of view projected in the ground plane. As shown in Fig. 3, this location is given by the coordinates (xcam, ycam), such that
`xcam ˆ xCG − zCG tan…θ†
`
`(5)
`
`(6)
`
`9=
`;
`
`The next section linearizes this model about a given vehicle state, allowing for synthesis of a linear model predictive controller for multiple
`target tracking.
`
`ycam ˆ yCG ‡ zCG tan…ϕ†∕ cos…θ†
`
`(7)
`
`B. Linearized Multirotor Dynamic Model
`To obtain linearized equations of motion for the multirotor, first define nondimensionalized thrust as ~T ≡ T∕mg. Then, Eq. (3) becomes
`8<
`9=
`: g ~T cos…ϕ† sin…θ† ‡ FDx
`; ˆ
`−g ~T sin…ϕ† ‡ FDy
`g ~T cos…ϕ† cos…θ† ‡ FDz
`
`9=
`;
`
`m
`
`m
`
`m
`
`− g
`
`8<
`: _vx
`_vy
`_vz
`
`(8)
`
`Several assumptions are invoked during the linearization process as follows:
`1) The vehicle is equipped with an inner-loop stabilization system capable of tracking commanded Euler angles with a first-order lag. Thus,
`_ϕ ˆ τϕ…ϕc − ϕ† and _θ ˆ τθ…θc − θ†, where τϕ and τθ are the time constants of the stabilization controller and the plant.
`− ~T†, where τ
`_~T ˆ τT… ~Tc
`2) Vehicle thrust ~T responds to commanded thrust ~Tc according to a first-order lag such that
`T is a time constant
`accounting for motor controller and rotor inflow dynamics.
`3) The vehicle operates at relatively small pitch and roll angles. Thus, it is assumed that sin…θ† ≈ θ, cos…θ† ≈ 1 and tan…θ† ≈ θ with similar
`expressions for ϕ. Likewise, terms involving products of Euler angles are assumed to be negligible.
`Define the vehicle state vector and control vector as
`x ˆ ‰ xCG
`
`yCG
`
`zCG ϕ θ
`
`vx
`
`vy
`
`vz ϕc
`
`θc
`
`~T ŠT
`
`(9)
`
`u ˆ ‰ _ϕ
`
`c
`
`_θc
`
`~Tc ŠT
`In Eqs. (9) and (10), two control inputs ϕc and θc are included as elements of the state vector rather than the control vector. The reason for this, as
`shown in Sec. II.E, is that both the control and control rate can be penalized separately within the model predictive controller.
`Given the aforementioned assumptions, Eq. (8) can be linearized about a hover flight condition given by ~T0 ˆ 1 and ϕ0 ˆ θ0 ˆ 0, where hover
`means that the mass center velocity with respect to the wind is zero. Let Δϕ, Δθ, Δ ~T, Δvx, Δvy, and Δvz represent small perturbations about this
`9=
`8<
`9=
`8<
`flight condition. This leads to the linearized translational dynamics given by
`: gΔθ − gcdx
`Δvx
`: Δ _vxΔ _vyΔ _vz
`; ˆ
`;
`−gΔϕ − gcdyΔvy
`gΔ ~T
`
`(10)
`
`(11)
`
`Downloaded by Michael Daniels on August 3, 2016 | http://arc.aiaa.org | DOI: 10.2514/1.I010345
`
`Yuneec Exhibit 1014 Page 4
`
`

`
`MILLER AND ROGERS
`
`For the remainder of the paper, the Δ symbols are dropped for convenience. Defining ~cdx
`8<
`8<
`9=
`9=
`: gθ − ~cdxvx
`: _vx
`; ˆ
`;
`−gϕ − ~cdyvy
`_vy
`g ~T
`_vz
`
`≡ gcdx and ~cdy
`
`≡ gcdy, Eq. (11) becomes
`
`349
`
`(12)
`
`Given Eqs. (1) and (12), and the assumptions outlined previously regarding closed-loop dynamics, the following linear dynamic model can be
`established:
`
`(13)
`
`(14)
`
`(15)
`
`377777777777777775
`
`0
`0
`0
`0
`0
`0
`0
`0
`τθ
`0
`0
`0
`0
`0
`0
`g
`0
`0
`0
`0
`0 −τT
`
`35
`
`T
`
`0
`0
`
`T
`
`_x ˆ Ax ‡ Bu
`
`0
`0
`0
`0
`−τθ
`g
`0
`0
`0
`0
`0
`
`1
`0
`0
`0
`0
`− ~cdx
`0
`0
`0
`0
`0
`
`0
`1
`0
`0
`0
`0
`− ~cdy
`0
`0
`0
`0
`
`0
`0
`0
`0
`0
`1
`0 τϕ
`0
`0
`0
`0
`0
`0
`0
`0
`0
`0
`0
`0
`0
`0
`
`ϕ
`
`0
`0
`0
`
`0
`0
`0
`0−τ
`0
`0
`0
`0
`0 −g
`0
`0
`0
`0
`0
`0
`0
`0
`
`0 0
`0 0
`0 0
`
`0 0 0 0 0
`0 0 0 0 0
`0 0 0 0 0
`
`0 1 0
`0 0 1
`0 0 0τ
`
`0 0
`0 0
`0 0
`0 0
`0 0
`0 0
`0 0
`0 0
`0 0
`0 0
`0 0
`
`266666666666666664
`
`24
`
`B ˆ
`
`A ˆ
`
`where
`
`Although the state vector dynamics in Eq. (13) are the same for both the gimbaled and fixed-camera cases, two separate output vectors are
`defined for each of these vehicle configurations. These output vectors are given by
`zCG ϕ θ ŠT
`
`yG ˆ ‰ xCG
`
`yCG
`
`(16)
`
`yNG ˆ ‰ xCG
`
`yCG
`
`zCG ϕ θ
`
`xcam ycam ŠT
`
`(17)
`
`where yG and yNG represent the gimbaled and fixed-camera cases, respectively. To linearize the output equations for xcam and ycam in Eqs. (6) and
`(7), a small angle approximation is invoked. Furthermore, a class of linear models is created for 10 logarithmically spaced altitude intervals
`centered at altitudes zi, where i ˆ 1; : : : ; 10. The linear output equations for the gimbaled and fixed-camera configurations are given,
`
`(18)
`
`(19)
`
`377775
`
`x ˆ CGx
`
`1 0
`0 1
`0 0
`0 0
`0 0
`
`0 0 0 0 0
`0 0 0 0 0
`1 0 0 0 0
`0 1 0 0 0
`0 0 1 0 0
`
`0 0 0 0
`0 0 0 0
`0 0 0 0
`0 0 0 0
`0 0 0 0
`
`266664
`
`respectively, by
`
`yG ˆ
`
`3777777775
`
`x ˆ Ci
`NGx
`
`1 0 0
`0 1 0
`0 0 1
`0 0 0
`0 0 0
`1 0 0
`0 1 0
`
`0
`0
`0
`0
`0
`0
`1
`0
`0
`1
`0 −zi
`0
`zi
`
`0
`0
`0
`0
`0
`0
`0
`
`0 0 0 0 0
`0 0 0 0 0
`0 0 0 0 0
`0 0 0 0 0
`0 0 0 0 0
`0 0 0 0 0
`0 0 0 0 0
`
`2666666664
`
`yNG ˆ
`
`where zi is the center of the discretized altitude band.
`Collectively, Eqs. (13), (18), and (19) represent the linearized multirotor model for both the gimbaled and fixed-camera configurations. These
`continuous equations are transformed to a discrete-time system using a zero-order-hold transform [21] with a time step of 0.2 s. The resulting
`discrete-time equations of motion are given in Eqs. (20–22):
`xk‡1 ˆ Adxk ‡ Bduk
`
`(20)
`
`Downloaded by Michael Daniels on August 3, 2016 | http://arc.aiaa.org | DOI: 10.2514/1.I010345
`
`Yuneec Exhibit 1014 Page 5
`
`

`
`350
`
`MILLER AND ROGERS
`
`yG;k ˆ CGxk
`
`yNG;k ˆ CiNGxk
`
`
`
`(21)
`
`(22)
`
`C. Ground Target Model
`The proposed guidance algorithm requires predictions of target motion over a finite time horizon H. Targets are modeled as point masses and,
`–JI plane, as shown in Fig. 4. A constant acceleration model is assumed, where
`for the purposes of this paper, their motions are restricted to the II
`targets hold constant longitudinal and lateral accelerations over the planning horizon. It is further assumed that lateral acceleration is reasonably
`small, such that changes in speed due to lateral acceleration of the target are neglected. For all cases in this paper, a planning horizon of H ˆ 5 s is
`used. Let _xi…t†, _yi…t†, and ψ
`i…t† be the velocity components and heading of target i, respectively. Then, assuming v0
`i is the target speed at the
`beginning of the planning horizon (t ˆ 0),
`
`(23)
`
`(24)
`
`(25)
`
`_xi…t† ˆ …alon
`
`i
`
`
`
`t ‡ v0i† sin ψ
`
`i…t†
`
`_yi…t† ˆ …alon
`
`i
`
`
`
`t ‡ v0i† cos ψ
`
`i…t†
`
`‡ alat
`
`i
`
`t†
`
`ln…v0i ‡alon
`
`alon
`i
`
`i
`
`if alon
`i
`if alon
`i
`
`≠ 0
`≈ 0
`
`8<
`: ψ i…0† − alat
`i…0† ‡ alat
`
`i
`
`ln v0
`i
`alon
`i
`i t
`v0
`i
`
`ψ
`
`dτ
`alat
`i
`i ‡ alon
`v0
`
`i
`
`τ ˆ
`
`Zt
`
`0
`
`The heading angle ψ i…t† is given by
`
`ψ i…t† ˆ ψ i…0† ‡
`
`For the purposes of this paper, it is assumed that measurements of the target’s heading and speed are available over a prior time horizon. Given
`i , and ψ i…0† may be obtained using online accumulation algorithms such as those given in [22] (the
`, alat
`, v0
`these measurements, estimates of alon
`i
`i
`accumulator equations are omitted here for brevity). Furthermore, assuming the uncertainty associated with each estimate is Gaussian, the
`standard deviation associated with each estimated quantity is also available through these accumulators. Thus, at the start of the planning horizon,
`i…0† for each target are assumed to be available. Let these values be denoted by alon
`i , and ψ
`, v0
`, v0
`the mean and standard deviation of alon
`, alat
`, alat
`i ,
`i…0† and σlon
`i
`i
`i
`i , σψ
`ψ
`, σlat
`, σv
`i , respectively.
`i
`i
`A particle filter is implemented for each target to predict target motion over the planning horizon. Using the mean and standard deviation for the
`four preceding quantities, Np particle trajectories are generated over the interval t ˆ ‰0; HŠ for each target in a Monte Carlo fashion using a
`
`i
`
`perturbed dynamic model given by
`
`(26)
`
`(27)
`
`(28)
`
`(29)
`
`
`_xj
`
`
`
`ij† sin ψ ji…t† ˆ … ~alonij t ‡ ~v0 i…t†
`
`
`
`
`
`ij† cos ψ ji…t† ˆ … ~alonij t ‡ ~v0 i…t†
`
`‡ ~alat
`
`ij
`
`ij t†ln… ~v0ij‡ ~alon
`
`
`
`~alon
`ij
`
`ij
`
`ln ~v0
`ij
`~alon
`ij
`ij t
`~v0
`ij
`
`if ~alon
`ij
`
`if ~alon
`ij
`
`≠ 0
`
`≈ 0
`
`dτ
`~alat
`ij
`ij ‡ ~alon
`~v0
`
`ij
`
`τ ˆ
`
`~ψ
`
`Zt
`
`0
`
`i…t† ˆ ~ψ
`ψ j
`
`ij…0† ‡
`
`
`
`_yj
`8<
`: ~ψ ij…0† − ~alat
`ij…0† ‡ ~alat
`for j ˆ 1; : : : ; Np, where the jth perturbed acceleration, velocity, and heading values are sampled according to
`∼ N… alon
`∼ N… alat
`i †;
`
`i ; σlati †;
`; σlon
`~alat
`~alon
`ij
`ij
`∼ N… v0
`
`i ; σvi†;
`
`~ψ ij…0† ∼ N… ψ i…0†; σψi †
`~v0
`ij
`
`i
`
`0i
`
`v
`
`lon
`
`ia
`
`iψ
`
`IJ
`
`lat
`
`ia
`
`II
`Fig. 4 Ground target motion diagram.
`
`Downloaded by Michael Daniels on August 3, 2016 | http://arc.aiaa.org | DOI: 10.2514/1.I010345
`
`Yuneec Exhibit 1014 Page 6
`
`

`
`MILLER AND ROGERS
`
`351
`
`Fig. 5 Method of determining desired multirotor location … ~xk; ~yk; ~zk†.
`
`An example of these Monte Carlo trajectories for four targets is shown in Fig. 5 in the form of black trajectory paths emanating from each target
`at the current instant in time. In Fig. 5, individual lines show the observed paths of the targets, and multiple lines emanating from the current target
`locations are possible target paths predicted by the particle filter. Circles represent the expected location of each target at a future time. A bounding
`rectangle is drawn that contains the expected locations of all targets at time step k. The desired multirotor location is the center of this square
`camera field of view. At the beginning of each planning time step, particles are completely regenerated from measurements rather than resampled
`from prior trajectories; thus,
`
`
`xj
`
`i…0† ˆ x i …0†
`
`(30)
`
`(31)
`
`
`yj
`
`i…0† ˆ y i …0†
`
`i …0† and y i …0† are the measured position of target i at the beginning of the planning horizon. Note that this scheme is equivalent to a
`
`
`where x
`particle filter in which the measurement error covariance is zero. However, the preceding algorithm requires significantly less computational effort
`than a complete particle filter, since resampling and likelihood computation is not performed in light of the assumption of perfect measurements. It
`should also be noted that the target prediction algorithm used here is provided strictly to evaluate performance of the proposed tracking algorithm.
`More complex target motion predictors can certainly be used and are unlikely to affect the performance of the tracking scheme.
`
`D. Reference Trajectory Generation
`
`Let the prediction horizon be divided into K equally spaced time intervals such that tk‡K − tk ˆ H. The goal of the reference trajectory
`
`generator is to create a sequence of system outputs such that the predicted expected values of all target locations remain within the camera field of
`view over the prediction horizon. It is further desired that the multirotor maintain the minimum altitude possible to keep all targets’ expected
`locations within the field of view. Let (xik, yik) be the ith target’s expected location at time step tk. Furthermore, let the surface of the ground be
`given in general form by z ˆ f…x; y† and the set of all points on the ground be given by set C ˆ f…x; y; z† ∈ R3jz ˆ f…x; y†g. Note that f may be
`either a smooth function in the case of terrain or a discontinuous function in the case of buildings, etc. Define the line-of-sight vector between
`target i and the multirotor commanded location as Li ˆ …xik
`− ~xk†II ‡ …yik
`− ~yk†JI ‡ …zik
`− ~zk†KI. The intersection of this line-of-sight vector
`with the ground surface is given by the set Si ˆ Li
`∩ C, which contains at least one point (the ith target location) but can potentially contain more
`in the case of targets obscured by buildings or terrain features. Finally, define all points that lie within the projection of the camera field of view on
`⊂ C. Then the commanded multirotor location at time step k can be found by solving the following constrained nonlinear
`the ground as CF
`optimization problem:
`
`Find… ~xk; ~yk; ~zk† that minimizes J ˆ j ~zkj
`subject to ~zk > f… ~xk; ~yk†;
`\ …xik; yik; zik† ˆ ∅ for all i;
`Si
`…xik; yik; zik† ⊂ CF
`for all i
`
`The solution of the optimization problem framed as shown ensures that the multirotor will track all targets from a position at minimum altitude
`such that line-of-sight vectors do not pass through obstacles (second constraint) and all targets remain within the camera field of view (third
`constraint). For general terrain, a nonlinear optimization solver may be used to determine the commanded multirotor location at each time step.
`
`Downloaded by Michael Daniels on August 3, 2016 | http://arc.aiaa.org | DOI: 10.2514/1.I010345
`
`Yuneec Exhibit 1014 Page 7
`
`

`
`352
`
`MILLER AND ROGERS
`
`For the purposes of this paper, flat terrain is assumed, and thus the line-of-sight vector constraint is satisfied for any choice of … ~xk; ~yk; ~zk†. To
`optimal multirotor location … ~xk; ~yk; ~zk† is computed as that which contains the expected location of all targets at minimum altitude. Figure 5
`
`solve this optimization problem for two-dimensional (2-D) terrain, bounding targets are defined at the edges of the camera field of view and the
`
`presents a graphical representation of this process at an example time step. Note that, for simplicity, it is assumed that the camera field of view is
`equal in the IB and JB directions, and that the vehicle yaw angle is always zero. Also, it is assumed that the vehicle pitch and roll angles are small
`enough that the skewing of the field of view is neglected.
`Optionally, a buffer zone can be designated at the edge of the camera field of view to keep targets from the approaching the edge, guarding
`against modeling errors and target prediction uncertainty. In this case, bounding targets are measured against the edge of the buffer zone rather than
`the edge of the field of view. Use of this buffer zone has the effect of increasing the multirotor’s altitude and reducing the probability that targets
`exit the field of view. At the same time, this altitude increase means that the targets are viewed from a longer distance, which is generally
`undesirable. A trade study involving this buffer distance is described in the Simulation Results section (Sec. IV).
`
`Let … ~xk; ~yk; ~zk† be the desired multirotor position and ~ϕk and ~θk be the desired roll and pitch angles, respectively, at time step k. The following
`
`stacked output vectors are defined over the prediction horizon for the gimbaled and fixed-camera cases, respectively:
`
`R ˆ ‰ ~xk
`YG
`
`~yk
`
`~zk
`
`~ϕ
`
`k
`
`~θ
`
`k
`
`: : :
`
`~xk‡K
`
`~yk‡K
`
`~zk
`
`~ϕ
`k‡K
`
`~θ
`
`k‡K ŠT
`
`(32)
`
`R ˆ ‰ ~xk
`YNG
`Throughout the rest of this work, ~ϕ
`k and ~θ
`k are set to zero, which has the effect of forcing the model predictive controller to track the desired
`velocity with as close to a level attitude as possible. Furthermore, in Eq. (33), ~xcam;k ˆ ~xk and ~ycam;k ˆ ~yk for all k, since, for the fixed-camera case,
`
`~yk
`
`~zk
`
`~ϕk
`
`~θk
`
`~xcam;k
`
`~ycam;k
`
`: : :
`
`~xk‡K
`
`~yk‡K
`
`~zk
`
`~ϕk‡K
`
`~θk‡K
`
`~xcam;k‡K
`
`~ycam;k‡K ŠT
`
`(33)
`
`it is desired that the multirotor be positioned directly above the target set as much as possible.
`
`E. Model Predictive Control
`A linear model predictive controller is constructed to track the reference trajectory. Given the control vector [Eq. (10)] at time step k, a stacked
`sequence of control vectors may be defined as
`
`U ˆ ‰ uT
`
`k
`
`uT
`k‡1
`
`: : :
`
`k‡K ŠT
`uT
`
`(34)
`
`For this control sequence, the estimated output Y over the planning horizon may be computed from the discrete system model using block
`matrices according to the following equations [23]:
`
`(35)
`
`(36)
`
`(37)
`
`Y ˆ kcaxk ‡ kcabU
`
`37775
`
`CdAd
`CdA2
`d
`.
`..
`CdAK
`d
`
`26664
`
`kca ˆ
`
`3777775
`
`0
`CdBd
`CdABd
`CdBd
`CdA2
`CdABd
`dBd
`.
`.
`..
`..
`
`d Bd CdAK−1d Bd
`CdAK
`
`0
`0
`CdBd
`.
`..
`: : :
`
`0
`0
`0
`
`: : :
`: : :
`: : :
`.
`. .
`0
`CdABd CdBd
`
`2666664
`
`kcab ˆ
`
`Given the predicted output vector Y and the desired output vec