United States Patent (19)
`Kawashima et al.
`
`54 AUTOMATIC TRACKING LIGHTING
`EQUIPMENT, LIGHTING CONTROLLER
`AND TRACKING APPARATUS
`75 Inventors: Toshikazu Kawashima, Shijonawate;
`Minoru Yoshida, Katano; Satoshi
`Furukawa, Hirakata; Kenichi Hagio,
`Katano, all of Japan
`73 ASSignee: Matsushita Electric Works, Ltd.,
`Saka, Japan
`
`21 Appl. No.: 08/878,071
`22 Filed:
`Jun. 18, 1997
`e - V's
`O
`O
`Foreign Application Priority Data
`30
`Feb. 22, 1996
`JP
`Japan .................................... 8-O34770
`Jun. 19, 1996
`JP Japan ...
`SSS3,
`Jun. 19, 1996
`JP
`Japan .................................... 8-158640
`51) Int. Cl." .............................. G05B 1918; H04N 7/18
`52 U.S. Cl. ........................... 364/167.6; 700/57; 700/58;
`700/59; 382/103. 3827181. 34.8/1 69
`58) Field of Search
`s
`s
`700/56 57.58
`700/59 60,83,382/103, 181 190: 34s. i25.
`s s/ Yu's Y-1s
`s 143. 155. 169 170
`s
`s
`s
`
`56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`4,386.848 6/1983 Clendenin et all
`356/5
`5.023.709 6f1991 Kita et all
`-
`- - - - - - - - - - - - - - - - - - - - - - - 35s/125
`5,062,056 10/1991 Local...
`... 364,516
`5204749 4/1993 Toyama et al.
`... 358/227
`5,434,617 7/1995 Bianchi ............
`... 348/170
`5,506,912 4/1996 Nagasaki et al.
`... 382/103
`5,739,857 4/1998 Kaneda ................................... 348/349
`
`USOO6079862A
`Patent Number:
`11
`(45) Date of Patent:
`
`6,079,862
`9
`9
`Jun. 27, 2000
`
`FOREIGN PATENT DOCUMENTS
`64-33803 2/1989 Japan.
`1296,502 11/1989 Japan.
`G SE E.
`OTHER PUBLICATIONS
`Patent Abstracts of Japan vol. 18, No. 663 (M-1723), for
`JP-6–260002, to Kansei Corp., published Dec. 14, 1994.
`Patent Abstracts of Japan vol.97, No. 007, for JP-9-063314,
`to Matsushita Electric Works, Ltd., published Mar. 7, 1997.
`An English Language Abstract of JP 6-260002.
`An English Language Abstract of JP 1-2.96502.
`An English Language Abstract of JP 64-33803.
`Primary Examiner Paul P. Gordon
`Attorney, Agent, or Firm-Greenblum & Bernstein, P.L.C.
`
`ABSTRACT
`57
`Automatic tracking lighting equipment for automatically
`tracking a target to be illuminated. A spotlight is Supported
`for pivoting in horizontal and vertical directions on a ceiling
`Surface. A horizontal drive mechanism changes the horizon
`tal angle of the Spotlight and a vertical drive mechanism
`changes the vertical angle of the Spotlight. A CCD camera
`picks up the image of a target area to be illuminated. An
`image recognition unit processes the image from the CCD
`camera to recognize the target to be illuminated and to
`Specify its coordinates. A coordinate calculation unit calcu
`lates how far to move the Spotlight based on a distance of the
`target to be illuminated. A movable control unit converts the
`output of the coordinate calculation unit into drive signals
`for the horizontal drive mechanism and for the vertical drive
`mechanism and outputs drive signals to pivot the Spotlight in
`a desired direction.
`
`29 Claims, 43 Drawing Sheets
`
`
`
`
`
`
`
`Movable
`Control
`Unit
`
`Coordinate
`Calculation
`unit
`
`Image
`recognition
`unit
`
`VWGoA EX1007
`U.S. Patent No. 9,955,551
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 1 of 43
`
`6,079,862
`
`Fig. 1
`
`Movable
`Control
`unit
`
`
`
`
`
`
`
`Coordinate
`Calculation
`unit
`
`Image
`recognition
`unit
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun.27, 2000
`Jun. 27, 2000
`
`Sheet 2 of 43
`Sheet 2 of 43
`
`6,079,862
`6,079,862
`
`Fig.2
`
`
`
`Fig.2
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun.27, 2000
`Jun. 27, 2000
`
`Sheet 3 of 43
`Sheet 3 of 43
`
`6,079,862
`6,079,862
`
`Fig. 3
`Fig.3
`
`
`
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun.27, 2000
`Jun. 27, 2000
`
`Sheet 4 of 43
`Sheet 4 of 43
`
`6,079,862
`6,079,862
`
`Fig.4
`Fig.4
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 5 of 43
`
`6,079,862
`
`
`
`
`
`
`
`
`
`
`
`Movable
`Control
`unit
`
`
`
`Coordinate
`Calculation
`unit
`
`Image
`recognition
`unit
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 6 of 43
`
`6,079,862
`
`Fig.6
`
`(xsys Zs)
`
`S
`
`Fig. 7
`
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun.27, 2000
`Jun. 27, 2000
`
`Sheet 7 of 43
`Sheet 7 of 43
`
`6,079,862
`6,079,862
`
`
`
`
` >
`
`_
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun.27, 2000
`Jun. 27, 2000
`
`Sheet 8 of 43
`Sheet 8 of 43
`
`6,079,862
`6,079,862
`
`
`
`Fig.9
`
`(Xg,¥g:2g)
`
`\
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun. 27, 2000
`Jun. 27, 2000
`
`Sheet 9 of 43
`Sheet 9 of 43
`
`6,079,862
`6,079,862
`
`Fig. 10
`
`
`
`II|||!t!t|1i1!!!iii{
`
`i.
`
`Spnr rr ee ce escsen
`x ~~
`
`Lee eeee
`
`>
`
`I|ttt!{I'1Itt11I1J
`
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 10 of 43
`
`6,079,862
`
`Fig.11
`
`13
`
`3a
`
`N N
`-
`
`pi,
`
`12
`
`Movable
`Control
`
`
`
`13
`
`N
`
`8
`
`2- )
`
`9
`
`Movable
`Control
`unit
`
`7
`
`
`
`COOrdinate
`Calculation
`unit
`
`Storage
`unit
`
`11
`
`Image
`recognition
`unit
`
`5
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun. 27, 2000
`Jun. 27, 2000
`
`Sheet 11 of 43
`Sheet 11 of 43
`
`6,079,862
`6,079,862
`
`Fig.12
`
`
`
`13
`
`rcweee eaea
`
`_—ENo©—
`
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 12 of 43
`
`6,079,862
`
`Fig. 13
`
`
`
`(xsys Zs)
`
`(0,0,zm)
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 13 of 43
`
`6,079,862
`
`3 3a
`
`3 10a
`S
`
`N
`
`3 10b
`N
`
`N
`
`
`
`2f\-4a
`‘. 2
`5a
`2)
`N
`3b N Image
`recognition
`unit
`
`
`
`N
`
`4b
`
`Movable
`Control
`unit
`
`
`
`
`
`Movable
`Control
`unit
`
`5b.
`
`Coordinate
`Calculation
`unit
`
`15a
`
`12a
`
`1
`
`Unit
`
`
`
`Movable
`Control
`unit
`
`
`
`Coordinate
`Calculation
`Unit
`
`Three-dimentional
`COOrdinate
`Calculation
`unit
`
`
`
`14
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Image
`recognition
`unit
`
`COOrdinate
`Calculation
`unit
`
`15b.
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun. 27, 2000
`Jun. 27, 2000
`
`Sheet 14 of 43
`Sheet 14 Of 43
`
`6,079,862
`6,079,862
`
`1
`
`
`
`(Xm2.¥m22m2)!
`
`ran
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun. 27, 2000
`Jun. 27, 2000
`
`Sheet 15 of 43
`Sheet 15 of 43
`
`6,079,862
`6,079,862
`
`Fig. 16
`
`(Ym2.¥m2:¥m1)
`
`mee dee eeee
`
`7{
`
`o~SpoN
`
`as re,
`
`m - - - - - - - - - - - - - - - - - - - - -
`
`
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 16 of 43
`
`6,079,862
`
`Fig. 17
`
`Video signal
`
`A/D conversion
`
`Step 1
`
`Step2
`
`Characteristic value
`extracting
`
`Calculation of
`similarity
`
`Step3
`
`Step4
`
`
`
`
`
`
`
`Object coordinate
`determination
`
`Object coordinate
`
`
`
`Object characteristic
`Value
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun. 27, 2000
`Jun. 27, 2000
`
`Sheet 17 Of 43
`Sheet 17 of 43
`
`6,079,862
`6,079,862
`
`CO
`OO
`LO
`Ww)
`
`99
`GS
`
`rS
`
`les | ()| 9
`3qzg)PbS
`eeS~~
`9.bI4
`
`N
`
`ISSNS
`
`8 | -61-I
`
`|eve
`
`|
`
`gg
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun. 27, 2000
`Jun. 27, 2000
`
`Sheet 18 of 43
`Sheet 18 of 43
`
`6,079,862
`6,079,862
`
`
`
`Fig. 19
`
`Fig.19
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun. 27, 2000
`Jun. 27, 2000
`
`Sheet 19 of 43
`Sheet 19 of 43
`
`6,079,862
`6,079,862
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 20 of 43
`
`6,079,862
`
`Fig.21
`
`
`
`Lighting Space
`
`Right shoulder C
`
`

`

`~L
`
`O
`
`
`
`U.S. Patent
`U.S. Patent
`
`Jun. 27, 2000
`Jun. 27, 2000
`
`Sheet 21 of 43
`Sheet 21 of 43
`
`6,079,862
`6,079,862
`
`Fig.22
`Fig.22
`
`
`
`ne [mm]
`
`‘$2
`
`X B {dot]
`
`Xt
`
`Ve
`
`S3
`
`Vf
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 22 of 43
`
`6,079,862
`
`Fig.23A
`(0,0)
`
`Fig.23B
`
`
`
`22
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun. 27, 2000
`Jun. 27, 2000
`
`Sheet 23 of 43
`Sheet 23 of 43
`
`6,079,862
`6,079,862
`
`GG
`GS
`
`
`
`
`
`poble
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun. 27, 2000
`Jun. 27; 2000
`
`Sheet 24 of 43
`Sheet 24 of 43
`
`6,079,862
`6,079,862
`
`
`
`
`
`Fig.25
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun. 27, 2000
`Jun. 27, 2000
`
`Sheet 25 of 43
`Sheet 25 of 43
`
`6,079,862
`6,079,862
`
`co
`O
`ite)
`O
`
`/
`/ /
`
`fd
`. . .
`Oo
`OQ
`oO
`O
`O
`CS
`N.
`R.
`N.
`Do 8
`
`==
`®
`TO
`CD
`C
`N.
`F.
`S. P
`8 »
`
`O
`O
`
`Ww:
`
`LO
`
`Q t
`
`O
`n
`rea
`D
`
`2
`O
`CN
`O
`
`ws
`
`53aO48 5la° 51
`
`w
`
`Uys
`y
`
`Fig.26
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 26 of 43
`
`6,079,862
`
`2
`
`Fig.27B
`
`
`
`Designate original
`Center position
`
`E2
`A.
`(22
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 27 Of 43
`
`6,079,862
`
`99
`
`CitzG
`
`CIZGº eig { @ C|| GyQ
`
`SQ
`
`N
`
`N
`
`939<<--
`````N`N
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun. 27, 2000
`Jun. 27, 2000
`
`Sheet 28 of 43
`Sheet 28 of 43
`
`6,079,862
`6,079,862
`
`
`
`feubisBuryoesy
`
`0/
`OL
`
`BulyesadoO
`
`YOUMS
`
`ule
`
`|OJ1U09
`
`yun
`
`ebew|
`
`e6e10}s
`
`UOHoes
`
`aBel01S
`
`[O1]U09
`
`uol]0as
`
`ebeu|
`
`ebeiojs
`
`UNOS
`
`ONSWE}OeIeUD
`
`
`
`abesolsenjea
`
`uoioes
`
`69
`69
`
`09
`
`08v9
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Buisseoold_\
`Bulzweulg
`uoyoes|GATO
`
`99G9
`BursseooldoeL9
`uonoes|GAY!
`62big
`
`99
`
`peubis
`
`uonoes
`
`jeubisg
`
`e9cg09
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 29 Of 43
`
`6,079,862
`
`Fig. 30
`Step 1
`
`*-> YES
`
`NO
`
`Step3
`
`Has initial
`Setting been
`executed ?
`
`NO
`
`Step5
`
`Step7 YES
`R
`
`ge inp
`
`Step6
`Object characteristic
`Value Storino
`
`
`
`HaS marker
`been detected?
`
`YES
`
`Color image input
`Step 11
`Object extracting process
`Step12
`
`
`
`
`
`
`
`
`
`AX, Ay
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 30 0f 43
`
`6,079,862
`
`LJ
`
`O O CD O O O CO O CO O O CD OO N QO LO <r Cro CN +=
`
`-},
`
`[1|||||||||
`
`500
`
`600
`
`Wavelength (nm)
`
`
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 31 of 43
`
`6,079,862
`
`Fig.32
`
`
`
`Synchronous
`receiving
`e2S
`
`infrared LED
`drive means
`
`Synchronous
`extracting
`eaS
`
`Synchronous
`transmitting
`eaS
`
`COn Verter
`eaS
`
`Image
`storage
`eaS
`
`Image
`differentiating
`eaS
`
`

`

`6,079,862
`6,079,862
`
`U.S. Patent
`U.S. Patent
`
`Jun. 27, 2000
`Jun. 27, 2000
`
`Sheet 32 of 43
`Sheet 32 of 43
`
`
`
`
`
`veebly
`
`geebly
`
`g88 (61-)
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 33 of 43
`
`6,079,862
`
`Fig.34A
`
`91"
`
`O
`
`Fig.34B
`
`Fig.34C
`
`N
`
`|-N
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 34 of 43
`
`6,079,862
`
`Fig.35
`Step 1
`
`*-> YES
`
`NO
`
`Step3
`
`Has initial
`setting been
`executed?
`
`YES
`Step7
`IR image input
`
`NO
`
`Step5
`Color image input
`Step6
`Object characteristic
`value storing
`
`Step8
`
`HaS marker
`been detected?
`
`YES
`
`
`
`
`
`Step 14
`Color image input
`Step 15
`Object extracting process
`Step16
`Position detection
`
`
`
`
`
`Has a plurality
`of markers been
`detected?
`
`NO
`
`Color image input
`Step12
`Object extracting process
`Step 13
`Position detection
`
`Position detection
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 35 0f 43
`
`6,079,862
`
`Fig. 36
`Step 1
`
`*-> YES
`
`NO
`
`Step3
`
`
`
`setting been
`executed ?
`
`R
`
`Has marker
`been detected?
`
`YES
`
`Step13
`Color image input
`Step 14
`Object extracting process
`Step 15
`Position detection
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Step5
`Color image input
`Step6
`Object characteristic
`value Storing
`
`
`
`Step9
`IR position detection
`Step 10
`Color image input
`Step 11
`Object extracting process
`Step 12
`Position detection
`
`AX, Ay
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 36 of 43
`
`6,079,862
`
`Fig. 37
`Step 1
`
`Has initial
`setting been
`executed ?
`
`
`
`Has marker
`been detected?
`
`YES
`
`Step 13
`Color image input
`Step 14
`Object extracting process
`Step 15
`Position detection
`
`Step5
`
`Step6
`
`
`
`value Storino
`
`Step9
`IR position detection
`Step 10
`Color image input
`Step 11
`Object extracting process
`Step12
`
`AX, Ay
`
`Object characteristic
`value updatinC
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 37 of 43
`
`6,079,862
`
`08#799929
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`9
`
`09
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 38 of 43
`
`6,079,862
`
`Fig. 39
`Step 1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`HaS initial
`Setting been
`executed?
`
`
`
`
`
`O
`IR image input
`
`HaS marker
`been detected?
`
`YES
`
`Step 13
`Color image input
`Step 14
`Object extracting process
`Step 15
`Position detection
`
`
`
`
`
`
`
`Step5
`Color image input
`Step6
`Object characteristic
`value storing
`
`Step9
`IR position detection
`Step 10
`Color image input
`Step 11
`Object extracting process
`Step 12
`Position detection
`
`Rotating table control
`
`To rotating
`table
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun. 27, 2000
`Jun. 27, 2000
`
`Sheet 39 0f 43
`Sheet 39 of 43
`
`6,079,862
`6,079,862
`
`0/
`OL
`
`Buyeedo
`
`YOUMS
`
`08v9€9
`
`
`
`
`
`
`
`jeuBis
`
`ure! uoloes
`
`
`
`
`
`J04]U09
`
`yun
`
`abe
`
`ebelols
`
`UdIDaeS
`
`abelols
`
`JO1JUOD
`
`Buizueuig
`uol]oes
`Buisseoold
`
`uoHoes
`
`ONSA]OBIEUD
`
`
`
`ebesojsanjea
`
`UOl}0as
`
`ebew}
`
`jeuBis
`
`uoNoes
`
`UOl]OesS
`
`ebes0jis
`Bursseood
`
`ajge}Bu—y
`
`
`
`
`
`
`
`

`

`Jun. 27, 2000
`
`Sheet 40 of 43
`
`6,079,862
`
`U.S. Patent
`Fig. 41
`Step 1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`HaS initial
`Setting been
`executed?
`
`
`
`IR image input
`
`HaS marker
`been detected?
`
`YES
`
`Step 13
`Color image input
`Step 14
`Object extracting process
`Step 15
`Position detection
`
`
`
`
`
`Step5
`Color image input
`Step6
`Object characteristic
`value storing
`
`Step9
`IR position detection
`Step 10
`Color image input
`Step 11
`Object extracting process
`Step 12
`POSition detection
`
`Rotating table posture reading
`
`From rotating
`table
`
`Coordinate transformation process
`
`Rotating table control
`
`Lighting control
`
`To rotating
`table
`
`To lighting
`equipment
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 41 of 43
`
`6,079,862
`
`Fig.42
`
`Y
`
`105A
`
`N
`
`a A.
`
`102
`
`103A !
`104A
`1 O1 N First
`N
`recognition unit
`
`106A
`
`105B
`
`N
`
`N
`
`e
`
`2 1 O3B
`104B
`
`109E
`
`109A
`
`First movable
`Control unit
`
`Second movable
`w
`Control unit
`
`108B
`
`108B
`
`108A
`
`First coordinate
`Calculation Unit
`
`Second
`COOrdinate
`Calculation unit
`
`
`
`1 11
`
`Storage
`unit
`
`Third
`COOrdinate
`Calculation
`unit
`
`Three-dimentional
`COOrdinate
`calculation unit
`
`113
`
`112
`
`107
`
`106B
`
`Second image
`recognition unit
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`
`
`
`
`yunuomuBbooe:abew).erbi4
`
`
`BiauedJO}OOpugAq
`
`pesodoidayeynoyey
`
`suoisodsjoel[qo
`
`BIBUUBDIOJOO3s}Aq
`
`pesodoidajyeinoyjen
`
`suoisodsjoalqo
`Jo}oOoOpugpueys,Aqsuolsod
`
`
`sjoslgopesodoidayejnajen
`
`UONEUIQWUIODeBaes19091Sgdais€4a81S
`
`UONBUIQUIODB109|9S
`UONBUIQUIODBj98|9S
`
`Sheet 42 of 43
`
`
`UOITEUIQWIODBOa]ES
`
`
`
`
`
`
`
`Aywuojuo0ojseubiyjoAjtuojuosysayBiyjo
`
`Ayuuojucoysaybiyjo
`
`
`
`Alwuoju0oyseyBiyjo
`
`uollsodul
`
`uomsodul
`
`Buoweuomsodul
`
`Buoweuolsodul
`
`elaweoJO}OOpugAq
`
`
`BiaweoJojooyspAq,
`
`
`
`BIOWEDPAJBIjUlPUuZ
`
`
`sjosiqopesodoidpue
`
`
`sjoe[qopesodoidpure
`
`BIBWEDJO|OOPug
`
`pueys}Aqsiemieuw
`
`pesodoidBuowe
`
`JS}Aqsieysewpesodod
`
`BJOLWUEDPOJE’U!
`
`paeleyulpugAqsiayiewpesodoid
`Biaweo
`pue3s,Aqalgo
`
`pesodoidBuowe
`
`
`
`lunuoNe|noyeoseyeulp1009
`
`
`
`
`
`|EUONUBWIP-88U!yp)dayg[UONDeIEpUONISOdjulodpayoejesjo
`
`
`
`
`
`QJEUIPJOODUOlSOd
`
`
`
`6,079,862
`
`rmmreermeeeeeeeeeeeeeeeeeeeeeeeeee
`
`—_—— = a a ee ee
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 27, 2000
`
`Sheet 43 of 43
`
`6,079,862
`
`Fig. 44
`
`
`
`First infrared Camera
`
`SeCOnd infrared Camera
`
`

`

`1
`AUTOMATIC TRACKING LIGHTING
`EQUIPMENT, LIGHTING CONTROLLER
`AND TRACKINGAPPARATUS
`
`BACKGROUND OF THE INVENTION
`1. Field of the Invention
`The present invention relates to automatic tracking light
`ing equipment, lighting controller and tracking apparatus,
`which can automatically track a performer or another object
`in a Space and floodlight it in a Site Such as a banquet hall,
`a hall or a Stage.
`2. Description of the Prior Art
`Conventionally, as automatic tracking lighting equipment
`for automatically tracking a moving target to be lighted Such
`as a perSon and floodlighting it, for example, Japanese
`Patent Laid-Open Publication No. SHO 64-33803 discloses
`one constituted by making the person who is the target to be
`lighted carry a transmitter of a SuperSonic wave or a radio
`wave, providing about twenty receiver Sensors on, for
`example, a ceiling Surface of a lighting Space having an area
`of 15 mx7.5 m, Specifying the transmitter, i.e., the position
`of the perSon carrying the transmitter from a reception signal
`from the receiver Sensor and making the lighting direction of
`a lighting fixture to automatically track the perSon.
`In the automatic tracking lighting equipment having the
`above construction, in order to correctly Specify the position
`of the transmitter, it has been required to provide a number
`of receivers on the ceiling Surface of the lighting Space and
`clarify a positional relation between the receivers in the
`lighting Space, leading to the problem that an inferior
`workability results.
`By the way, as is commonly known, the lighting control
`ler is to control the direction of a lighting fixture for
`floodlighting a perSon or another object moving on a stage
`and making the lighting position track the position of the
`person who is the target to be lighted with the movement of
`the perSon.
`Among the conventional lighting controllers, there is a
`lighting controller having a constitution in which an operator
`is located in the vicinity of a space to be lighted (i.e., located
`in a position where the operator can get the direct view of at
`least the lighting position), operates the lighting with a
`remote controller or the like by getting the direct view of the
`Space desired to be lighted and continues operating So that
`the target lighting position and the lighting position of the
`lighting fixture coincide with each other.
`There is another lighting controller having a constitution
`in which a camera is fixed to the light body section of the
`lighting fixture So that the lighting direction of the lighting
`fixture and the image pickup direction of the camera are
`arranged in parallel to each other, and the operator continues
`operating So that the target lighting position and the lighting
`position of the lighting fixture coincide with each other not
`by getting the direct view of the Space desired to be lighted
`but by Viewing an image picked up by the camera by means
`of a touch panel or the like provided at the display device
`that is displaying the image.
`There is a further lighting controller having a constitution
`in which a camera of which image pickup direction is fixed
`is placed in the vicinity of a lighting fixture, and when an
`operator inputs a target lighting position while Viewing the
`image picked up by the camera, the lighting direction is
`moved from the current lighting position to a designated
`target lighting position.
`However, in the lighting controller operated by a remote
`controller among the aforementioned lighting controllers,
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6,079,862
`
`2
`the operator must be located in the vicinity of the Space
`desired to be lighted and the operator is required to continue
`operating while viewing the lighting position until the
`lighting position reaches the target lighting position. This
`arrangement has the problem that the operator is required to
`have a skill to a certain extent and labor and burden are
`imposed on the operator.
`In the lighting controller having the constitution in which
`the camera is fixed to the light body Section of the lighting
`fixture So that the lighting direction of the lighting fixture
`and the image pickup direction of the camera are arranged
`in parallel to each other, and the lighting direction is
`controlled based on the image picked up by the camera, the
`operator is not required to be located in the vicinity of the
`Space desired to be lighted, however, the operator is required
`to continue operating until the lighting position reaches the
`target lighting position similarly to the above case. This
`arrangement has the problem that labor and burden are
`imposed on the operator.
`In the lighting controller which controls the lighting
`direction by the image of the camera of which image pickup
`direction is fixed, the operator is not required to continue
`operating until the lighting position reaches the target light
`ing position once the target lighting position is inputted,
`however, the mounting positions of the Spotlight, camera
`and So forth are required to be inputted before executing the
`control. Therefore, the mounting positions must be
`measured, and this causes the problem that a considerable
`labor is required for the initial Setting.
`By the way, in recent years, a tracking apparatus for
`tracking an object to be tracked is used for the uses of
`automatically picking up the image of a bride or another
`object by means of a TV camera and automatically flood
`lighting the object to be tracked according to the movement
`of the object in a banquet hall Such as a wedding ceremony
`hall and a hotel. Tracking apparatuses of this kind have been
`roughly classified into two types of one which tracks an
`unspecified object to be tracked and the other one of which
`object to be tracked is predetermined.
`The former is to preparatorily Store the characteristics
`(e.g., color and shape) of the object to be tracked displayed
`on a Screen which displays an image picked up by an image
`pickup means and detect a position in which the highest
`Similarity to the aforementioned characteristics is achieved
`is detected from the images inputted Successively. AS a
`concrete method of this type, there is So-called the template
`matching. The template matching is to Store image data of a
`region including an object to be tracked as a template, Sort
`the Successively inputted images into blocks having the
`Same size as that of the template, calculate every pixel the
`absolute value of a difference between the template and each
`block, accumulate the absolute values and adopt the position
`of the block having the minimum cumulative value as the
`position of the object to be tracked.
`The latter is to detect the position of an object to be
`tracked by detecting the marker of an infrared light
`transmitter, a color chart or a reflection tape etc. attached to
`the object to be tracked by using position detecting Sensors,
`or by extracting them by So-called the binarizing process.
`For example, Japanese Patent Laid-Open Publication No.
`HEI 6-260002 discloses a constitution which detects the
`position of an object to be tracked by receiving the infrared
`light from an infrared light transmitter carried with the
`object to be tracked by light receiving element of the
`position detecting Sensor mounted to the body of the
`Spotlight, and control the lighting derection of the Spotlight.
`
`

`

`6,079,862
`
`3
`Further, Japanese Patent Publication No. HEI 5-46041
`discloses a constitution which detects the position of an
`object to be tracked, after attaching a reflection marker Such
`as reflection tape or the like to the object, by extracting high
`luminance points by binarizing the image from pickup
`CS.
`According to the prior art tracking apparatus utilizing the
`template matching, the difference between the values of
`pixels located at the relatively Same coordinates of the
`template and the block is taken. Therefore, when the shape
`of the object to be tracked changes (e.g., when the object to
`be tracked is a human being), the cumulative value of the
`absolute values of the difference in the correct position
`disadvantageously increases, and this has Sometimes led to
`a failure in tracking the object to be tracked. Furthermore,
`there has been the problem that a degraded tracking accu
`racy has resulted in an environment where a number of
`objects having a shape Similar to that of the object to be
`tracked exist.
`In order to Solve these problems of template matching, a
`method for extracting color information of the object to be
`tracked and having, for example, a characteristic value as a
`histogram has been proposed. This method has the charac
`teristic that the histogram does not depend on the shape, the
`characteristic that the color information has a relative tol
`erance to a change in lighting and So forth. However, the
`color information changes when the distance and angle
`between the object to be tracked and the light changes in the
`course of tracking, and therefore, the Stored color informa
`tion cannot be extracted, possibly leading to a difficulty in
`tracking.
`On the other hand, the method of tracking by detecting the
`position of the marker has the characteristic that an output
`power can be increased when the infrared light transmitter is
`used as a marker because the infrared light is invisible to the
`human being and tracking can be stably executed when the
`infrared light can be stably received. However, there has
`been the problem that the marker is Sometimes made invis
`ible by the object to be tracked itself or another object, and
`a number of markers (infrared light transmitters) must be
`attached to the object to be tracked in order to prevent the
`occurrence of it. Furthermore, depending on the environ
`ment of use, it is Sometimes the case where a light Source
`having an infrared component Such as distance measuring
`equipment or a halogen lamp of a camera exists other than
`the infrared light transmitter, when the infrared light from
`other than the infrared light transmitter is disadvantageously
`tracked, causing a failure in tracking the object to be tracked.
`Also, when a reflection marker Such as reflection tape or
`the like is employed, in case there exist a plurality of high
`luminance points in the lighting rigion by existence of
`another reflective object, the possibility of wrong recogni
`tion of the object to be tracked become higher, by using of
`the method which specifies the position of the object only by
`the binarizing data of luminance, and it causes lack of utility.
`The present invention has been developed in view of the
`aforementioned problems and has an object to provide
`automatic tracking lighting equipment of which workability
`is improved.
`Further, the present invention has been developed for
`improving the aforementioned problems and has another
`object to provide a lighting controller capable of fixing the
`target lighting position while viewing the lighting position
`on a display device without requiring the operator to be
`located in the vicinity of the lighting position and capable of
`moving the lighting position into the target lighting position
`
`4
`by designating once the target lighting position, thereby
`lightening the burden on the operator, obviating the need for
`inputting the mounting positions of the lighting fixture,
`camera and So forth and requiring no Substantial labor for the
`initial Setting.
`Furthermore, the present invention has been developed
`for the aforementioned reasons and has further object to
`provide a tracking apparatus capable of Surely tracking the
`object to be tracked with high accuracy.
`
`SUMMARY OF THE INVENTION
`In order to achieve the aforementioned object, and to
`Solve the aforementioned problems, the first aspect of the
`present invention comprises: a lighting means having a
`directivity; a drive means for changing a lighting direction
`of the lighting means, an image pickup means for picking up
`an image in a direction identical to the lighting direction
`while changing its image pickup direction together with the
`lighting direction; an image recognizing means for recog
`nizing a target to be lighted from an image from the image
`pickup means and Specifying coordinates of the target to be
`lighted; a calculating means for calculating a quantity of
`movement in the lighting direction from a quantity of
`movement of the target to be lighted in the image recogniz
`ing means, and a control means for controlling driving of the
`drive means based on a result of calculation of the calcu
`lating means. Therefore, this equipment can also cope with
`a target to be lighted having neither transmitter nor the like.
`Furthermore, there is no need to provide a number of
`receivers or the like on the ceiling Surface, and therefore, the
`workability can be improved. Furthermore, in an initial
`Setting Stage, the lighting direction of the lighting means can
`be visually made to coincide with the coordinates of the
`target to be lighted of the image recognizing means.
`Further, the Second aspect of the present invention com
`prises: a lighting means having a directivity; a drive means
`for changing a lighting direction of the lighting means, an
`image pickup means provided in a lighting Space So as to be
`able to pick up an image in the lighting Space of the lighting
`means, an image recognizing means for recognizing a target
`to be lighted from an image from the image pickup means
`and Specifying coordinates of the target to be lighted; a
`calculating means for calculating a quantity of movement in
`the lighting direction from coordinates of the target to be
`lighted in the image recognizing means and a positional
`relation between the lighting means and the image pickup
`means in the lighting Space, and a control means for con
`trolling driving of the drive means based on a result of
`calculation of the calculating means. Therefore, this equip
`ment can also cope with, for example, a lighting fixture of
`which lighting direction is changed through reflection on a
`mirror, and it can control a plurality of lighting means.
`Furthermore, the third aspect of the present invention
`comprises: a lighting means having a directivity; a first drive
`means for changing a lighting direction of the lighting
`means, an image pickup means provided in a lighting Space
`So as to be able to pick up an image in the lighting Space of
`the lighting means, a Second drive means for changing an
`image pickup direction of the image pickup means; a Storage
`means for Storing the image pickup direction of the image
`pickup means, an image recognizing means for recognizing
`a target to be lighted from an image from the image pickup
`means and Specifying coordinates of the target to be lighted;
`a Second calculating means for calculating a quantity of
`movement of the Second drive means from a quantity of
`movement of the target to be lighted in the image recogniz
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`

`

`6,079,862
`
`15
`
`25
`
`S
`ing means, a first calculating means for calculating a quan
`tity of movement in the lighting direction of the lighting
`means from a result of calculation of the Second calculating
`means, a positional relation between the image pickup
`means and the lighting means in the lighting Space and an
`image pickup direction of the image pickup means Stored in
`the Storage means, a first control means for controlling
`driving of the first drive means based on a result of calcu
`lation of the first calculating means, and a Second control
`means for controlling driving of the Second drive means
`based on a result of calculation of the Second calculating
`means. Therefore, a lens having not So wide angle can be
`used for the image pickup means, thereby allowing even the
`image of a target to be lighted located far away to be clearly
`picked up and allowing image recognition to be easily
`executed.
`Furthermore, the fourth aspect of the present invention,
`based on the third aspect of the present invention, comprises
`a plurality of the image pickup means, whereby the Second
`calculating means calculates three-dimensional coordinates
`of the target to be lighted in the lighting Space from image
`pickup directions of the plurality of the image pickup means.
`Therefore, the target to be lighted can be correctly captured
`even when the floor Surface of the lighting Space has an
`UCWCCSS.
`Also, in order to Solve the aforementioned problems,
`according to the fifth aspect of the present invention, there
`is provided a lighting controller including: a lighting fixture
`for radiating light from a light Source; a drive Section for
`pivoting the lighting fixture around an axis, an image pickup
`means being pivotable around an axis, for picking up an
`image in an image pickup region and outputting an image;
`a display device for displaying the image, and an input
`device for designating a point on the display image dis
`played on the display device, the lighting controller com
`35
`prising a pivoting quantity calculating Section for calculating
`a quantity of pivoting of the lighting fixture for floodlighting
`the designated point from a relative distance between the
`position on the display image designated by the input device
`and the position of origin of the display image, whereby the
`drive Section pivots the lighting fixture based on the quantity
`of pivoting.
`Further, according to the Sixth aspect of the present
`invention, there is provided a lighting controller including:
`a lighting fixture for radiating light from a light Source, a
`drive Section for pivoting the lighting fixture around a pan
`axis and a tilt axis, an image pickup means being pivotable
`around an axis, for picking up an image in an image pickup
`region and outputting an image; a display device for dis
`playing the image; and an input device for designating a
`point on the display image displayed on the display device,
`the lighting controller comprising a pivoting quantity cal
`culating Section for calculating a quantity of pivoting around
`the pan axis of the lighting fixture for floodlighting the
`designated point from a relative distance in a horizontal
`direction between the position on the display