IMAGES VIDEOS MAPS NEWS SEARCH HISTORY MORE MSN OUTLOOK.COM
`
`rWM
`
`Make Bing my homepage
`
`2*
`
`■y 250 Sign in •»
`
`0
`
`l> bing
`
`Large-Scale Reconfigurable Computing in a Microsoft Datacenter
`
`7
`
`. - k'
`
`*\l n
`
`w
`
`.
`
`• «»•
`
`, .
`
`V
`
`Andrew Putnam - Microsoft
`
`Hot Chips 26 - Aug 12, 2014
`
`© Image Credits
`
`C 2014 Microsoft | Privacy and Cookies | Legal | Advertise | About our ads | Help | Feedback
`
`IPR2018-01694
`
`EXHIBIT
`2065
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 1
`
`

`

`Microsoft Cloud Services
`fj
`
`io7 f
`106
`
`510 i
`
`4
`
`10
`
`:
`3 '
`10 r
`
`2
`
`10 [
`*
`10 r
`1 '
`
`o
`
`10
`
`a
`
`A
`
`A
`
`*
`
`l
`
`Transistors
`(thousands)
`
`Single-thread
`Performance
`(SpecINT)
`
`Frequency
`(MHz)
`- •; Typical Power
`(Watts)
`Humber of
`Cores
`
`i •
`
`a
`
`y*-""
`*
`
`■
`
`A
`
`A
`
`A
`
`» 4
`A A
`%
`a a
`
`A
`
`A
`
`•;
`
`a
`
`*
`4b
`
`■*
`
`A
`A
`
`•«
`• 9 \ s*
`* y_ M'm •
`
`a
`
`i
`
`a.
`
`• •
`• • %
`■
`»«
`«•
`£
`#
`a - .v>
`
`a
`
`A
`
`rA
`
`A A
`
`A
`
`A
`
`A
`
`A
`
`A
`
`A
`
`A;
`
`A
`
`±
`1975 1980 1985 1990 1995 2000 2005 2010 2015
`
`Original data col I acted and plotted by M Horowitz. F labonte, O Shacham. K Otjkotun. L Hammond and C, Batten
`Dotted toe extrapolations by C Moore
`
`𝑷𝒆𝒓𝒇𝒐𝒓𝒎𝒂𝒏𝒄𝒆/𝑾𝒂𝒕𝒕
`
`$
`
`Capabilities, Costs ∝
`
`Windows Azure
`XBOX
`
`I IX/pr t^:osmos ^55^
`msn>l
`Qa Outlook.com B o.Off ice 365
`
`Microscft"-SQLServer
`
`Ql.
`
`-
`
`OneDrive
`
`JZ3 fll Microsoft1'11
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 2
`
`

`

`More flexible...
`
`More efficient...
`
`Microprocessors
`
`DSPs
`
`10X
`
`Dedicated HW
`
`100X
`
`1000
`3
`E55 loo
`CLo
`— 10
`
`1
`
`LJ
`CD
`tCD
`g1 0.1
`LU
`
`US
`
`>>
`
`0.01
`
`1
`
`2
`
`3
`
`4
`
`5
`6
`7
`8
`9
`10 11
`12 13 14 15 16 17 18 19 20
`Processor Number (sorted by efficiency)
`
`Source: ISSCC Proceedings
`
`Increase Efficiency with Hardware Specialization
`
`V
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 3
`
`

`

`Datacenter Environment
`• Software services change monthly
`• Machines last 3 years, purchased on a rolling basis
`• Machines repurposed -Vi way into lifecycle
`• Little/no HW maintenance, no accessibility
`
`• Homogeneity is highly desirable
`
`The paradox: Specialization and homogeneity
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 4
`
`

`

`Efficiency via Specialization
`
`1000
`
`More flexible...
`
`FPGAs
`
`DSPs
`
`More efficient...
`
`Microprocessors
`
`ri
`
`Dedicated HW
`
`ASICs
`100X
`
`r
`
`2
`E< 100
`Q_o
`— 10
`
`u C
`
`1
`
`D
`o
`5=
`CD
`21 0.1
`CD
`LU
`
`0.01
`
`12 13 14 15 16 17
`10 11
`9
`8
`7
`6
`5
`Processor Number (sorted by efficiency)
`
`18 19 20
`
`Source: ISSCC Proceedings
`
`Source: Bob Broderson, Berkeley Wireless group
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 5
`
`

`

`One Application's Accelerator
`
`Flexibility
`
`Xeon CPU
`
`Efficiency
`
`NIC
`
`V
`Accelerator Opportunities
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 6
`
`

`

`One Application's Accelerator
`
`Flexibility
`
`Efficiency
`
`Xeon CPU
`^Application Changes
`
`Search Acc.
`(FPGA)
`
`Search Acc.
`(ASIC)
`
`NIC
`
`Xeon CPU
`B^Re-assigned Server
`
`HJPC
`Clysttr
`
`Xeon CPU
`
`Search Acc. v2
`(FPGA)
`
`Wasted Power,
`Holds back SW
`
`
`NIC
`
`Math
`Accelerator
`
`Wasted Power,
`NIC
`One more thing that
`can break
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 7
`
`

`

`Integrating FPGAs into the Datacenter
`
`■:
`
`"r
`m
`■J
`i
`m
`
`;
`3
`r
`
`:»
`
`:*
`vi
`
`:■
`
`:■
`
`■
`
`:•
`1
`:•
`
`HB?
`
`•lkJ
`
`- -
`
`:■
`
`j
`
`:
`
`•c
`#;
`
`I
`15:! :zS:
`HE I *
`:
`•:
`: =|;
`•:
`:
`:
`:
`
`:~5'!
`:dS-i r:2 •
`-jfj=|^ =|:
`3
`
`:
`
`»
`
`w
`
`»
`
`rA’
`
`:
`
`:
`
`■:
`
`r‘.
`
`:■
`
`;
`:
`•fi
`
`:■
`
`i'.
`j
`•:
`
`:■
`
`:■
`
`:
`:
`:•
`
`:■
`
`:■
`
`:•
`
`:■
`
`i
`
`■:
`
`I1
`
`•:
`
`;
`:
`:
`
`:
`
`*;
`■i
`
`h
`
`:
`
`*
`i:
`i:
`
`:\
`
`:
`s •
`j:
`
`■1
`
`■:
`
`I,»
`
`[:
`
`:
`;
`
`?
`
`:
`
`:
`;i
`X
`f:
`I:
`
`*:
`-!
`
`■:
`
`■:
`
`•:
`<!
`«•:
`'A
`
`:
`:•
`:*
`
`:f
`
`:]
`
`o-:
`
`1 $
`
`i:
`
`i:
`i:
`
`!;
`
`SI
`
`■;
`
`,
`
`-» i- -Ji J
`
`:i
`
`:•
`
`|:
`:
`:
`: •
`:
`:
`:
`
`I:
`
`!
`:
`
`■
`
`I
`
`.
`
`t.
`
`:■
`
`:
`:
`:*
`:•
`:•
`:•
`:
`:•
`:
`
`:
`:
`
`Distributed
`
`i?
`
`•t
`
`|:
`:
`P #
`[:
`:
`
`*■
`
`i
`i
`
`:-
`:
`1
`I
`:
`li
`• '!
`:
`:
`;
`I;
`:]
`:]
`:
`:
`
`:■
`
`:■
`
`:
`:•
`:
`:
`:]
`ii
`
`:■
`
`I;
`
`FPGAs
`
`:
`:
`:
`s
`
`:
`:
`:
`P *
`!
`:
`:
`
`:
`ti
`:
`■i
`•i
`•;
`
`:r
`
`:•
`1
`:•
`
`if
`
`:•
`a
`
`■
`
`I
`:
`=
`j:
`:
`:
`
`FPGAs
`
`1
`
`!l
`
`-
`
`■
`
`!
`;
`:
`I
`
`This looks easy - Plug into the network and go!
`
`Centralized
`
`:•!
`:4
`1
`II
`:
`3
`I
`
`r
`?i
`
`:*
`
`-
`I
`
`!i
`
`Cl
`f:
`
`i:
`
`■:
`
`*
`::
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 8
`
`

`

`~o
`03
`CQ
`CQ
`CQ
`
`O
`
`CD
`Q_
`
`Q_
`
`o
`o
`
`^1SICXSA90X
`9 - X31dlA
`
`•XNHIX
`
`•Mil
`
`imu
`
`!*(
`
`r
`
`'t
`
`___J
`
`fiii
`
`■
`
`in
`
`m
`
`m. m.m
`
`BFB-I RLVA
`
`-
`
`I t
`
`XC6VSX315
`
`T U
`
`XllINX.VIRTEX-6
`
`f
`
`I;
`
`II
`
`»
`
`MF«
`
`9-xaiaiA
`• XNHIX
`
`1SICXSA93X
`
`t
`
`•<«'
`
`,#« t
`
`• «(•
`
`* i
`
`n UNX
`VIRTEX-6
`XC6VSX315T“
`
`f?
`
`eu
`
`u
`
`9-X3iaiA
`
`rt,lSlCXSA9DX
`
`•'XNHIX
`
`f ^
`
`s !K
`
`oo
`\
`I
`
`'T
`
` ■
`
`*
`
`N
`
`XILINX*VIRTEX-6
`xcevsxaisi™
`^4>0M3U
`
`* •««
`
`Vt t«M4)4
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 9
`
`

`

`^1
`
`li
`
`CO
`
`3s-
`ill •
`
`LLij
`
`I IIS
`:1
`
`i
`
`I ISO
`,, 059} j •
`
`ii L
`
`II."
`
`■
`
`s
`
`ii-' K
`*C .»«r»
`
`15H1F
`
`S
`
`vjIRC
`
`ijjjl
`
`■©
`
`' i
`
`I
`
`O)
`
`r*
`£•
`sis’
`iso
`
`II
`
`5 ‘*1
`
`1 -1
`
`Iri
`
`"i i
`
`In
`
`<
`
`■
`
`05
`
`I Jr4
`
`X
`
`Prototype #1: BFB board
`Prototyped a 6-FPGA board
`3x2 GPIO mesh
`PCIe connecting all FPGAs, CPU
`Plugs into Supermicro GPU server
`Serves L2 scoring for 48-server pod
`
`• • • • •
`
`::
`i
`
`W
`
`r
`
`v—4
`
`5
`/-•
`
`:
`T
`
`:
`
`w
`
`:
`▼
`
`fjr
`
`t i
`
`v
`
`• 1U, 2U, or 4U rack-mounted
`
`• 1/2/4 x 10Ge ports
`
`• Up to 4 PCIe x16 slots
`
`• 2 sockets, 6-core Intel Westmere
`
`I ^
`s
`Iv.vXn.
`
`~
`
`r»
`
`i
`
`1
`
`t
`
`\
`
`V
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 10
`
`

`

`Centralized Model Unsuitable for Datacenter
`• Single point of failure
`• Complicates rack design, thermals, maintainability
`• Network communication for any use of FPGA
`• Definition of the Network In-cast problem
`• Precludes many latency-sensitive workloads
`• Limited elasticity
`• What if you need more than six FPGAs?
`
`-
`
`ji
`
`'A
`
`w.
`
`:l
`
`£■■
`
`l:
`
`-j
`
`■
`
`.i
`:
`at;
`
`1
`
`'k'
`
`:
`w :
`
`I
`
`I;
`
`'a*
`
`i
`
`I
`
`'a^
`
`l1i1
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 11
`
`

`

`Our Design Requirements
`
`Don't Burn Too Much
`Power
`
`k
`
`<10% Power Draw
`(25W max, all from PCIe)
`
`1. Specialize HW with
`an FPGA Fabric
`2. Keep Servers
`Homogeneous
`
`fi
`
`Don't Cost Too Much
`
`<30% Cost of Current
`Servers
`
`Don't Break Anythin
`
`Work in existing servers
`No Network Modifications
`Do not increase hardware failure rate
`
`k
`fr
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 12
`
`

`

`#
`
`\m ■
`Al
`
`t
`
`I,
`
`\
`
`r
`
`/
`
`w
`
`■j
`
`rl
`
`*
`k •
`k
`k
`s,
`S'
`
`A
`
`■*
`
`■».
`
`-
`J? f-*
`- x? *
`f r
`■P-
`- a_/L ^
`'S»v •,
`
`•v
`. H
`-
`‘U.H
`
`>-
`
`VV
`
`SVk
`
`*
`
`*i»
`
`► »
`' */,
`• ■ *
`!> .
`• «
`
`v
`
`Datacenter Servers
`• Microsoft Open Compute Server
`• 1U, ^ wide servers
`• Enough space &
`power for ^ height
`^ length PCIe card
`• Squeeze in a single FPGA
`• Won't fit (or power) GPU
`
`/
`
`\
`\
`http://www.globalfoundationservices.com/posts/2014/january/27/microsoft-contributes-cloud-server-specification-to-open-compute-project.aspx
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 13
`
`

`

`Microsoft Open Compute Server
`
`►
`
`i o
`
`XC
`
`i ♦
`
`o
`Xc
`6
`
`t
`
`i
`
`i• j
`
`‘l!
`
`I
`
`Air flow
`200 LFM
`Inlet
`68 ⁰C
`
`
`
`»
`
`*»
`
`%■>
`
`•’.i
`
`as
`
`k
`
`*
`
`>9
`
`I
`
`*!
`-i’
`I-
`* i
`■IPj;
`
`\ i
`■KN ■ ft.'
`m\ p| k
`
`f.
`
`1 L 1 / ■ I K
`
`•.r.
`
`§.
`
`J
`
`A #|i| |IIH
`r
`
`rT d
`
`, > «
`
`I5I ^L
`
`su
`
`ki
`
`tv
`*
`
`• Two 8-core Xeon 2.1 GHz CPUs
`• 64 GB DRAM
`• 4 HDDs @ 2 TB, 2 SSDs @ 512 GB
`• 10 Gb Ethernet
`• No cable attachments to server
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 14
`
`

`

`• Altera Stratix V GS D5
`• 172k ALMs, 2,014 M20Ks, 1,590 DSPs
`• 8GB DDR3-1333
`• 32 MB Configuration Flash
`Config
`Flash
`
`Stratix V
`
`• PCIe Gen 3 x8
`• 8 lanes to Mini-SAS
`SFF-8088 connectors
`• Powered by PCIe slot
`
`PCIe Gen3 x8
`
`4x 20 Gbps Torus Network
`
`8GB DDR3
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 15
`
`

`

`00
`oo
`CD X
`TD CO u
`03
`LD
`CDo
`cn
`CQ
`
`00 oo x
`03
`CD
`
`u
`
`Q
`
`E
`
`EEo
`cr
`LD
`on
`x E
`E E
`E rN
`LT) ■^-
`no \—i
`
`LD \n
`Q_
`CO
`CD
`
`E
`
`:-iiK
`
`Ini]
`
`p!n
`
`■izi
`
`-13
`
`».
`
`sflaMiiiliu
`il
`iH*
`wmm
`
`i I
`
`LP
`
`i
`
`>
`
`II
`
`Is■
`
`:
`
`Mezz Conn.
`
`i
`
`1U
`
`ti"^:
`
`3
`
`:
`
`11H||
`MsHB
`• ■MMME
`00 <
`LD
`
`■sis! li.
`
`’dwm■igaijjjEEfenr^mttflH
`
`mm■rn^m
`I^lH
`
`I
`
`li I f I! 1111! I il IJ! !E: II i Si HF:
`
`!'l
`,j ;T!|B||P:i
`
`sltlflli
`ilil
`
`;i'
`
`m
`
`Hi.Di
`i-l'llBllUl
`lus^
`
`!=!
`
`•mm
`
`ranaaisBESBfcSi^
`
`©
`®®
`
`
`^-■=:~-
`
`:■.
`
`IKHTri
`
`HplP m
`
`2SSS£ESS£S5BJ?§|^S
`lifiMill
`ps®«g? imtm
`ilkiliml
`:!
`8mil»ii—
`"'Mm
`
`FPGA
`
`V
`
`l
`
`Tjj
`K ::riTii:
`
`■iiRmaMiiUH'aitnii
`'i ipNi^-;11mMiM
`I ll^tefvwaita :-..ll II
`llBHlillMMi
`I mmmm
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 16
`
`

`

`Board / Server Integration
`
`Catapult Network Cables
`(Mini-SAS / SFF-8088)
`
`
`r,.
`
`S-
`
`12216
`
`i
`
`-v
`
`\MiiM
`//ll/l
`
`r
`
`|
`
`- -^a
`
`'
`
`*
`
`m
`
`it s
`
`• •
`
`I
`i' *1
`
`IBB I
`
`V
`
`Ky * *
`
`; o|
`
`Tl
`n.i^ , ■ il
`
`■
`
`1
`
`—-a
`
`}jv
`
`I
`
`i.
`
`Si!
`
`V
`
`V]
`
`□□
`o
`Q)
`Q_
`in
`
`no
`
`3
`3
`CD
`n
`CD
`Q_
`
`O
`IQ
`CD
`
`CD
`
`1
`
`Catapult Board
`Mezzanine Slot
`
`
`CD
`3
`Q_
`U
`CD
`CD
`
`C0
`
`o
`
`"O
`CD
`O
`GO
`
`Q_
`CD
`CD
`u
`
`CD
`
`>C
`
`D
`LO
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 17
`
`

`

`6x8 Torus in a 2x24 Server Layout
`HfiH21ni
`i i
`i
`i
`l
`I
`I
`
`I;
`;
`I
`ii
`i
`I!
`l
`i:
`it
`:
`:
`::
`:
`:
`:
`:
`:
`;
`:
`:
`:
`:::
`:::
`
`■I
`
`;
`
`:
`
`S:
`i
`:
`
`X;
`
`::
`
`;
`:
`
`;:
`
`:
`:
`:
`:
`:
`
`;I
`
`I
`
`:
`:
`:
`
`!!
`
`::!
`
`:
`:
`
`:
`:
`:
`:
`:
`:
`:
`:
`::
`:
`:
`:]
`:
`:
`:
`:
`:
`:
`:
`:
`;
`:
`
`:;
`
`A
`
`[i
`
`.
`
`■
`
`t
`
`rr
`
`j
`
`J
`
`\ i
`
`23<
`22
`21
`-20
`.19
`18
`17
`
`1°<
`’ 9 \1
`8
`7
`6
`5-
`4
`15
`3
`14-
`2
`3
`1
`8-Shell Cables
`
`WS,
`
`23 ^
`22__ S
`
`-
`
`^—10
`
`mjr
`Ei
`
`20
`19__ 2
`CT“ 6------------------18
`^5
`17^
`^ 4______________16___^
`15
`$
`14
`13
`6-Shell Cables
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 18
`
`

`

`Scalable Reconfigurable Fabric^
`•
`1 FPGA board per Server
`• 48 Servers per Vi Rack
`•
`6x8 Torus Network among FPGAs
`• 20 Gb over SAS SFF-8088 cables
`
`1 A
`
`3
`
`1 *
`
`i/.
`
`j»y ■
`
`►
`
`-t
`
`3
`
`U V-
`
`Tl U
`
`i
`
`◄
`
`3
`
`►
`
`❖
`
`rao
`
`3
`
`,
`
`i ?
`r- ■
`
`■.
`
`k.
`
`i
`
`•r^sa
`
`i-jpJB
`
`L
`
`vLJ
`
`Data Center Server (1U, ½ width)
`5 V
`l![
`r.
`I l‘
`£
`jx
`
`fejs
`
`.9^
`
`n • prm
`WF3
`
`el
`
`U
`Vi
`
`E|
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 19
`
`

`

`An Elastic Reconfigurable Fabric
`
`I
`
`I
`
`I
`
`T
`
`FPGA
`
`
`FPGA
`FPGA
`Web Search Pipeline
`
`
`FPGA
`
`Top-of-Rack Switch (TOR)
`
`PCIe (8.0 GB/s)
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 20
`
`

`

`An Elastic Reconfigurable Fabric
`
`Math Acceleration
`Service
`
`X I
`
`2
`
`/
`17
`
`x i
`
`2
`
`XT
`
`/ I
`
`FPGA
`FPGA
`Web Search Pipeline
`
`
` Physics
`Engine
`
`i
`
`I
`
`~ZA
` Comp.
`Vision
`Service
`
`
`zT
`
`FPGA
`
`FPGA
`
`PCIe (8.0 GB/s)
`SLIII (2.0 GB/s)
`400 ns latency/hop
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 21
`
`

`

`4 GB DDR3-1333
`ECC SO-DIMM
`
`4 GB DDR3-1333
`ECC SO-DIMM
`
`72
`
`72
`
`Shell
`
`DDR3 Core 0
`
`DDR3 Core 1
`
`Role
`
`8
`
`Host
`CPU
`
`x8 PCIe
`Core
`
`Application
`
`DMA
`Engine
`
`Inter-FPGA Router
`
`North
`SLIII
`
`South
`SLIII
`
`East
`SLIII
`
`West
`SLIII
`
`2
`
`2
`
`2
`
`2
`
`4
`
`256 Mb
`QSPI
`Config
`Flash
`
`Config
`Flash
`(RSU)
`
`JTAG
`
`LEDs
`
`Temp
`Sensors
`
`I2C
`
`xcvr
`reconfig
`
`SEU
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 22
`
`

`

`Selection as a Service (SaaS)
`
`Ranking as a Service (RaaS)
`
`Query
`
`SaaS 1
`IFM 1
`IFM 1
`IFM 1
`
`SaaS 2
`IFM 2
`IFM 2
`IFM 2
`
`SaaS 3
`IFM 3
`IFM 3
`IFM 3
`
`SaaS
`IFM 44
`48
`IFM 44
`IFM 44
`
`Selected
`Documents
`
`RaaS 1
`IFM 1
`IFM 1
`IFM 1
`
`RaaS 2
`IFM 2
`IFM 2
`IFM 2
`
`RaaS 3
`IFM 3
`IFM 3
`IFM 3
`
`RaaS
`IFM 44
`48
`IFM 44
`IFM 44
`
`10 blue links
`
`Ported to Catapult
`
`Selection-as-a-Service (SaaS)
`- Find all docs that contain query terms,
`- Filter and select candidate documents for
`ranking
`
`Ranking-as-a-Service (RaaS)
`- Compute scores for how relevant each selected
`document is for the search query
`- Sort the scores and return the results
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 23
`
`

`

`FE: Feature Extraction
`
`Query: “FPGA Configuration”
`
`NumberOfOccurrences_0 = 7 NumberOfOccurrences_1 = 4 NumberOfTuples_0_1 = 1
`
`W http://en.wikipedia.org -Mki/FPGA P ’ ^ C X | W Field-programmable gate a... X
`‘1 JH *
`
`1-^Lra.lfcrf^J'
`
`Log in / create account *n
`
`Features:
`
`(
`
`Document
`{Query, Document}
`i
`FE: Feature
`Extraction
`
`FFE: Free-Form
`Expressions
`
`i i
`
`MLS: Machine
`Learning Scoring
`\
`
`Score
`L2 Score
`
`Read Edit
`
`View history | Seardi
`
`f « y Article Talk
`Field-programmable gate array
`From Wikipedia.
`cyclopedia
`(Redirected froi
`A field-programmable gate array IFPGAJis an integrated circuit rtesinnsd tn hp nnnfigured by the customer or
`designer after manufacturing—hence "tie'lcrproqrammable". The FPGA Configuration is generally specified using a
`hardware description language (HDL), similar to that used for an application-specific integrated circuit (ASIC) (circuit
`diagrams were previously used to specify the configuration, as they were for ASICs, but this is increasingly rare).
`pPGAs|can be used to implement anv logical function that an ASIC could perform. The ability to update the
`Tuncnonality after shipping, partial re- configuration ofa portion of the design^ and the low non-recurring engineering
`costs relative to an ASIC design (notwitnstanomg the generally higher unit cost), offer advantages for many
`applications.121
`
`Q
`
`51 a
`WikipediA
`The Free Encyclopedia
`Main page
`Contents
`Featured content
`Current events
`Random article
`Donate to Wikipedia
`v Interaction
`Help
`About Wikipedia
`Community portal
`Recent changes
`Contact Wikipedia
`► Toolbox
`► Print/export
`Languages
`
`FPGAs|contain programmable logic components called "logic blocks", and a hierarchy of reconfigurable
`interconnects that allow the bh^ks tn hp "wired together"—somewhat like many (changeable) logic gates that can be
`d to perform complex combinationalinter-wired in (many) different configuration:. Logic blocks can be cQQgguif
`
`functions, or merely simple logic gates like AND and XQR. In mostFPGAa the logic blocks also include memory
`elements, which may be simple flip-flops or more complete blocks of memory.121
`
`TTfflT
`BwapCKM
`Boansch
`Catala
`
`In addition to digital functions, somelFPGAsI
`have analog features. The most common analog feature is
`programmable slew rate and drive strength on each output pin, allowing the engineer to set slow rates on lightly
`loaded pins that would otherwise ring unacceptably, and to set stronger, faster rates on heavily loaded pins on high-
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 24
`
`

`

`FFE: Free Form Expressions
`
`Features:
`
`Document
`{Query, Document}
`*
`FE: Feature
`Extraction
`
`FFE: Free-Form
`Expressions
`
`MLS: Machine
`Learning Scoring
`
`Score
`L2 Score
`
`NumberOfOccurrences_0 = 7 NumberOfOccurrences_1 = 4 NumberOfTuples_0_1 = 1
`
`FFE #1 =(2*NumberOfOccurrences_0 + NumberOfOccurrences_1)
` (2 * NumberOfTuples_0_1)
`
`i
`
`Metafeature #1 = 9
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 25
`
`

`

`Compressed
`Document
`
`PCIe
`
`Control/Data
`Tokens
`
`Stream
`Preprocessing
`FSM
`
`Distribution latches
`
`Free Form
`Expression
`(FFE)
`
`Feature
`Gathering
`Network
`
`• 196 feature families
`• 54 state machines
`• 2.6K dynamic features extracted in
`less than 4us (~600us in SW)
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 26
`
`

`

`1
`
`«
`
`s;f
`
`HHI
`
`P'^iSs
`ii|
`iiiiiiii;*
`
`h
`
`JH
`
`'J1**
`
`I!
`
`* Si!!
`^■’1 ■£
`t L-*.- y tic IHiSg
`n
`tlSiSP
`iHfiigi
`Cluster
`iffll
`rf -TSIS
`ttiiife
`0
`Him
`KgigSfSSSMW
`iiil
`mmm
`git
`
`55=
`
`■■
`
`'iW-%
`
`i'-i. I?..-'
`
`mngj^EiniK'ff^Tr:
`
`^aiggEistaiizarariEi!
`
`Core 0 Core 1 Core 2
`If
`
`:
`4tIII
`
`'ir
`
`mmm
`'h:|Bi
`
`FST
`
`ut
`Outp
`
`Complex
`[1
`Mi (iifs
`i pn
`'!}f
`Core 3 Core 4 Core 5
`Jll
`"tW Hffiifi; Wt\.r'
`iw
`
`iii;
`
`FFE Soft Cores
`• Soft processor for multi­
`threaded throughput
`• 4 HW threads per core
`• 6 cores share a complex ALU
`•
`log, divide, exp, float/int conv.
`• 10 clusters (240 HW threads)
`per FPGA
`
`Ranker
`Model
`File (INI)
`
`Model
`Compiler
`
`C+ +
`
`FFE
`Backend
`Compiler
`
`FFE
`Assembly
`
`¥
`
`Scheduler
`
`I-Mem
`
`Feature
`Store
`
`^ r
`Thread 0
`Thread 1
`Thread 2
`Thread 3
`
`^ r
`
`F
`
`D
`
`E M W
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 27
`
`

`

`Document
`
`
`
`FE: Feature Extraction
`
`
`FFE: Free-Form
`Expressions
`
`
`
`Score
`
`Compute
`Score
`Compute
`Score
`
`8-Stage Pipeline
`
`RaaS Servers
`
`Route to
`Head
`
`Route to
`Head
`
`Document
`Scoring
`Request
`
`Return
`Score
`
`Document
`Scoring
`Request
`
`Return
`Score
`
`FPGA 0
`
`FPGA 1
`
`FPGA 2
`
`FPGA 3
`
`FPGA 4
`
`FPGA 5
`
`FPGA 6
`
`FPGA 7
`
`Server
`
`Server
`
`Server
`
`Server
`
`Server
`
`Server
`
`Server
`
`Server
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 28
`
`

`

`<c
`
`i
`
`:•; j
`
`-f
`4
`
`i\l
`t'y-is
`-*
`-•
`»•
`-»
`-•
`-.•
`■'•S
`-•
`-!
`-•
`>«
`«•
`
`lifll’l
`; -Bi:3I:'
`i! ijil
`-iiiiti
`E S L -i
`-
`.* _i
`-^e
`li
`1-:3it?l^r3 toi
`L ^ ^4t?::M
`III!:<1
`ii
`
`-f 4 -- -*
`>.
`-- I
`•c:ri •<
`<:rv
`i
`
`4L
`
`*■'
`
`vl
`
`"I b■ '4 \ '4 -
`■'SHW&M:
`im mn
`i d: J 1
`\ f\-i' I zji ■
`
`A
`
`jk3;
`
`_*'
`
`; d | -5 ■ I
`.US’ Jl.—£
`-*1 -1-----4
`"~mm
`■ zf :f -f
`" -f 4::
`-4 v
`»
`v. ^
`a-
`p^:*4 i
`v
`P" *" o‘ ■ 1
`k-:| <
`fc
`pc -S 4:^4
`■£’ .' ^ .J. '< •-.
`Jl
`- I' ^ a.
`•xV
`
`I
`I
`
`1
`I
`I
`
`3
`
`I
`
`II
`
`5 i
`3 ; 4:i\i-
`i' i1
`3 i 5
`il-J:
`3 :! i.
`iar
`_r
`-fj
`
`-i ,
`
`•:
`
`1
`
`'X\ %
`.*
`-ir •
`r
`*•
`^ . J .i
`^ i
`
`'-S.
`
`^ *
`
`-f1
`* •
`>«f <y
`
`.
`
`K*
`
`>•
`
`1 1
`
`<
`
`V
`K.
`
`.
`
`rC
`
`e-.
`
`w
`
`■'*
`
`«•
`
`J*
`u--.
`
`•,
`
`.<?
`'.<?
`#
`
`■»>
`
`;\
`•r
`rW «.
`
`< V V
`
`• • ^
`
`V
`
`**
`r-*
`
`L
`
`k
`
`k.
`
`k.
`
`.*
`
`• .
`
`V
`
`1
`
`'i
`II . r1
`I n
`H
`i*!
`-
`;:;-ft
`
`1
`
`i
`
`-*
`T_
`-• .•
`-•
`*
`~*-•
`-/
`-•
`3
`
`V
`
`* -
`
`1,632 Server Pilot Deployed
`
`in a Production Datacenter
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 29
`
`

`

`Accelerating Large-Scale Services – Bing Search
`1,632 Servers with FPGAs Running Bing Page Ranking Service (~30,000 lines of C++)
`
`3
`
`95% Query Latency vs. Throughput
`
`T3
`OJ
`N 2.5
`
`2x Increase in
`Throughput
`
`29% Latency
`Reduction
`
`roE
`o 2C
`Q
`O 1.5u
`LU
`CO
`cc
`LU
`Cl
`CO
`LU
`CC
`111 0.5
`
`1
`
`Da
`
`SW + FPGA
`
`Reduced #
`of servers
`
`SW Only
`
`< 30% Cost
`t 25 W Power
`i HW Failures
`
`1.6 1.8
`
`2
`
`0
`
`0
`
`0.2
`
`More compute time for
`Production in Early 2015
`improving relevance
`
`—SWOnly —SW + FPGA
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 30
`
`

`

`FPGAs for Application Acceleration
`• Hardware is the "easy" part
`• Software changes fast
`• Services last across hardware generations
`
`/'
`
`\
`
`/
`
`>
`
`x'
`
`FPGA
`
`N
`
`■/
`
`Software &
`Language
`
`Compiler &
`Tools
`
`RTL&
`Hardware
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 31
`
`

`

`Cluster 0
`
`Core 0
`
`Core 1
`
`Core 2
`
`FST
`
`Complex
`
`ut
`Outp
`
`Core 3
`
`Core 4
`
`Core 5
`
`4 GB DDR3-
`1333 ECC
`SO-DIMM
`
`72
`
`4 GB DDR3-
`1333 ECC
`SO-DIMM
`
`72
`
`Shell
`
`DDR3 Core 0
`
`DDR3 Core 1
`
`Role
`
`Application
`
`Hos
`t
`CPU
`
`8
`
`x8 PCIe
`Core
`
`DMA
`Engine
`
`Inter-FPGA Router
`
`North
`SLIII
`
`South
`SLIII
`
`East
`SLIII
`
`West
`SLIII
`
`2
`
`2
`
`2
`
`2
`
`4
`
`256
`Mb
`QSPI
`Conf
`ig
`Flash
`
`Config
`Flash
`(RSU)
`JTAG
`
`LEDs
`Temp
`Sensor
`s
`I2C
`xcvr
`reconfi
`g
`SEU
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 32
`
`

`

`Key Needs for FPGA Computing
`• Huge need for high-productivity languages
`• C-to-gates tools did not do well on FE state machines
`• Domain-specific languages, OpenCL, BlueSpec both show promise
`
`66 CD
`CD ^
`sZ CD
`CD ^£
`CDO
`LO
`
`to
`O
`
`66
`CD
`
`ou
`
`CD
`66 £
`CD
`"O
`C£ CD
`
`• Faster compilation times
`• Fewer warnings... NO warnings on IP libraries
`• Better debugging integration
`
`• Hardened PCIe, DDR, JTAG debugging
`• Faster, more efficient DDR
`• Improved floating-point performance
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 33
`
`

`

`Conclusions
`• Hardware specialization is a (the?) way to gain
`efficiency and performance
`• The Catapult reconfigurable fabric offers a flexible,
`elastic pool of resources to accelerate services
`• Results for Bing: V2 the number of ranking servers,
`lower latency, reduced variance, proven scalability,
`proven resilience
`• Bing going to production in early 2015
`• Biggest future problem is programmability
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 34
`
`

`

`Top Row: Eric Peterson, Scott Hauck,
`Aaron Smith, Jan Gray, Adrian M.
`Caulfield, Phillip Yi Xiao, Michael
`Haselman, Doug Burger
`
`Bottom Row: Joo-Young Kim, Stephen
`Heil, Derek Chiou, Sitaram Lanka,
`Andrew Putnam, Eric S. Chung,
`
`Not Pictured: Kypros Constantinides,
`John Demme, Hadi Esmaeilzadeh,
`Jeremy Fowers, Gopi Prashanth
`Gopal, Amir Hormati, James Larus,
`Simon Pope, Jason Thong
`
`Huge thanks to our partners at
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 35
`
`

`

`IMAGES VIDEOS MAPS NEWS SEARCH HISTORY MORE MSN OUTLOOK.COM
`
`Make Bing my homepage
`
`2W
`
`'S' 250
`
`Sign in
`
`o
`
`MT
`
`**
`
`bing
`
`Enter your questions here
`
`2?
`f¥ >
`
`.m'.
`
`&
`
`Feedback
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 36
`
`

`

`Microsoft Research
`
`B Microsoft
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 37
`
`

`

`Microsoft
`
`© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
`The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on
`the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
`
`PATENT OWNER DIRECTSTREAM, LLC
`EX. 2097, p. 38
`
`