Max - Pia nck - 1 nstitut
`fur biologi s che Kybernetik
`Arbeitsgruppe Bulthoff
`
`Technical Report No. 6
`
`June 1994
`
`Moving Cast Shadows and the Perception of Relative
`Depth
`
`Daniel Kersten, Pascal Mamassian & David C. Knill
`
`Abstract
`
`We describe a number of visual illusions of motion in depth in which the motion of an
`object's cast shadow determines the perceived 3D motion of the object. The illusory
`percepts are phenomenally very strong. We analyze the information which cast
`shadow motion provides for the inference of 3D object motion and experimentally
`measure human observers' use of this information. The experimental results show that
`cast shadow information overrides a number of other strong perceptual constraints,
`including viewers' assumptions of constant object size and a general viewpoint.
`Moreover, they support the hypothesis that the human visual system incorporates a
`stationary light source constraint in the perceptual processing of shadow motion. The
`system imposes the constraint even when image information suggests a moving light
`source.
`
`DK and PM were supported by the National Science Foundation (BNS-9109514) and the Max Planck Society.
`DCK was supported by the Air Force Office for Scientific Research (AFOSR 90-2074) and NIH (EY09383-
`01Al). We thank Albert Yonas and Deborah Rossen for their comments and suggestions. Correspondence should
`be sent to: Daniel Kersten, N218 Elliott Hall, Psychology Department, 7 5 East River Road, Minneapolis, MN
`55455, U.S.A .. Email: kersten@eye.psych.umn.edu.
`
`This document is available as /pub/mpi-memos/TR-6.ps.Z via anonymous ftp from ftp.mpik-tueb.mpg.de or by
`writing to the Max-Planck-Institut filr biologische Kybernetik, Spemannstr. 38, 72076 Ti.ibingen, Germany.
`
`16 June 199412:32 pm
`
`Legend3D, Inc. Ex. 2023-0001
`IPR2016-01243
`
`

`

`Introduction
`
`1.0
`
`Introduction
`
`The relative displacement between an object and its
`cast shadow in an image provides an important source
`of visual information about the spatial layout of
`objects. Leonardo da Vinci elucidated the principle
`relating shadow displacement and the perception of
`relative depth in his notebooks: " ... when representing
`objects above the eye and on one side--if you wish
`them to looked detached from the wall--show
`between the shadow on the object and the shadow i~
`casts, a middle light, so that the body will appear to
`stand away from the wall." ( da Vinci, 1970) Artists
`regularly exploit this principle in static drawings and
`paintings of 3D scenes, and psychophysical research
`has shown the salience of static cast shadow informa(cid:173)
`tion for judgments of depth (Yonas, 1978). Yonas et
`al. (1978) were able to show that the location of a cast
`shadow was able to influence the judged depth and
`height of an object above a ground plane in observers
`as young as three years old. The role of dynamic shad(cid:173)
`ows in human perception, however, has received no
`scientific study. Because movement due to shadow
`boundaries is almost always present in the retinal
`image, understanding how the visual system processes
`shadow motion is a fundamental issue in vision . In
`this paper, we report a set of controlled experiments
`and phenomenal demonstrations which show:
`
`the perception of motion in depth. The reason is that
`judgments based on static cues with long viewing
`times can involve conscious reasoning as well as per(cid:173)
`ceptual processing. Second, the computational prob-
`1 em of identifying shadows is known to be very
`difficult. The real-time requirements of identifying
`shadows in motion may be even harder. Although pro(cid:173)
`cessing of static shadows has received some study in
`computer vision (Waltz, 1972; Shafer, 1985), with few
`exceptions (Kender, J. R., & Smith, E. M., 1987) com(cid:173)
`puter vision has ignored moving cast shadows. Third,
`if vision's primary function is to determine the iden(cid:173)
`tity and spatial layout of surfaces and objects, one
`could argue that variation of intensity in the image due
`to illumination might be discounted early given the
`processing overhead required. A related argument that
`the visual system discounts variations in illumination
`in order to determine surface color has been discussed
`since Helmholtz.
`
`The computational difficulty lies in the fact that optic
`flow is determined by a complex interaction of causes.
`The form and evolution of optic flow is influenced by
`changes in the viewpoint of the observer, positions
`and shapes of the objects, and the illumination. Unlike
`the effect of shape, the effect of illumination on the
`image is not just local. Shadow boundaries are deter(cid:173)
`mined by the illumination, the casting object, the
`receiving object and the viewpoint. Unfortunately,
`there is no unambiguous local cue for a shadow edge.
`Nevertheless for human shape perception, static cast
`shadow boundary is useful for object shape perception
`as well as depth perception (Cavanagh, & Leclerc,
`1989). How are shadows identified? Cavanagh ( 1991)
`argues, based on work with images of faces, that the
`identification of shadow boundaries and utilization of
`shadow information may in factfollow the recognition
`of the category that a shape belongs to. From this
`point of view, it is not unreasonable to suppose that
`judgments involving static shadows may require pro(cid:173)
`cesses that are too slow to be useful in processing
`dynamic shadows for depth information. Yet, moving
`cast shadows are used routinely in cartoon animations
`and in video games; but does this merely enhance the
`realism of the pictures, or is this information useful
`for depth?
`
`Figure 1 illustrates the well-known effect of shadow
`displacement on the perception of relative depth in
`static images: the closer an object is to its cast shadow
`in an image, the closer it appears in depth to the back(cid:173)
`ground surface. We created a motion analog of this
`demonstration, in which the shadow cast by a station(cid:173)
`ary square moves back and forth relative to the square
`(figure 2). We then ran a simple psychophysical
`experiment (Experiment 1) to test whether subjects
`would see the square move in depth (see figure 2 cap(cid:173)
`tion for details). When the shadow was rendered real-
`
`•
`
`•
`
`the relative motions of objects and their cast shad(cid:173)
`ows in an image can produce remarkably strong
`percepts of 3D motion
`information provided by the motion of an object's
`shadow overrides other strong sources of informa(cid:173)
`tion and perceptual biases, such as the assumption
`of constant object size and a general viewpoint
`image features such as shadow darkness can be
`utilized, but are not necessary for the perception of
`depth from moving cast shadows
`• support for a prior assumption of a stationary light
`source constraint by the visual system.
`
`•
`
`2.0 The Phenomenon
`
`2.1 Experiment 1: Cast shadow motion is
`sufficient for the perception of motion
`in depth.
`
`The first question is whether shadow motion is in fact
`used for the perception of relative motion in depth.
`Although it is reasonable to assume that an affirmative
`answer would follow given the evidence from judging
`static shadows in pictures, it is not necessarily the case
`for at least three reasons. First, the fact that a pictorial
`cue is useful for judgments of depth does not neces(cid:173)
`sarily imply that variations of that cue will produce
`
`1
`
`Legend3D, Inc. Ex. 2023-0002
`IPR2016-01243
`
`

`

`The Phenomenon
`
`greater effects of cast shadow motion on observers'
`percepts of 3D motion. Unfortunately, one cannot
`remove the effect of the size constancy constraint
`from an experiment, since the image size of an object
`is an inherent property of a stimulus. What is possible,
`however, is to remove the effect of the general view(cid:173)
`point constraint by simply moving the object, as well
`as its cast shadow, in the image plane.
`
`2.2 Demonstration 1: Phenomenally strong
`illusion of motion in depth with acciden(cid:173)
`tal view removed.
`
`We generated a 3D graphics simulation which we call
`the ball-in-a-box animation (figure 3), in which we
`simulated a ball moving inside a box in such a way
`that it followed a diagonal trajectory in the image
`plane. As in Experiment 1, the size of the object's
`
`Fig. 2. Observers were asked to look at a fixation
`mark ( +) placed on a checkerboard plane which sub(cid:173)
`tended 6.6 x 1 o· of visual angle. Viewing distance was
`500 mm. At a position 4.1 • to the right of the fixation
`point, a foreground square was superimposed over a
`sharp shadow of the same size as the square. In a 500
`msec. animated sequence, the shadow oscillated for
`one cycle through a 0.34° displacement from the fore(cid:173)
`ground square. The foreground square remained sta(cid:173)
`tionary throughout the animation . Observers were
`asked to indicate whether the foreground square
`appeared to oscillate in depth or appeared to be sta(cid:173)
`tionary. Six different types of shadow were used for the
`experiment: three "dark" shadows simulated as film
`transparencies with transmittances of 12, 16, and
`36%; and three physically implausible "light" shadows
`corresponding to transmittances of 180, 284, and
`394% (i.e. light was added within the shadow). The
`background checkerboard had a mean luminance of
`17.4 cd I m2 with an 82% contrast between dark and
`light squares. Subjects were split into two groups of
`ten. The order of presentation of different shadow con(cid:173)
`ditions for one group, in terms of effective transmit(cid:173)
`tance, was: 16, 284, 12, 394, 36, and 180%. The other
`group saw the stimuli in the order 284, 16, 394, 12,
`180 and 36%. Each subject viewed three series of pre(cid:173)
`sentations, making a total of 18 trials . On 78% of the
`trials using dark shadows, observers reported seeing
`the foreground square as oscillating in depth--toward
`and away from the viewer. On only 40% of the trials
`using light shadows did subjects report seeing the
`square oscillating in depth (A Wilcoxon signed rank
`order test on the difference between light and dark
`shadows gave p= 0.001 ).
`
`Fig. 1. Increasing the displacement between the cast
`shadows and the three foreground squares tends to
`produce an impression of increasing depth (from left
`to right) relative to the background checkerboard .
`
`istically dark, subjects reported seeing the square
`move toward and away from the background surface
`78% of the time. When the shadow was implausibly
`lighter than its background, subjects only reported
`seeing the square move in depth 40% of the time. Sub(cid:173)
`jects who perceived the motion reported that the per(cid:173)
`cept was phenomenally strong and immediate.
`
`The result clearly shows an effect of cast shadow
`motion on observers' perception of 3D motion of an
`object. Moreover, a close analysis of the experimental
`stimuli reveals that for the observers who saw the
`motion in depth, the motion of the shadow overrode a
`number of conflicting cues which suggested that the
`square was stationary: the lack of any change in size
`of the square, and the lack of any 2D motion of the
`square in the image. That these features of the stimu(cid:173)
`lus would suggest object stationarity results from the
`human visual system's bias to assume, first, that
`objects do not change size over time (related to object
`size constancy, cf. Gogel, Hartman, & Harker, 1957) ,
`and second, that the viewer is viewing the scene from
`a non-accidental, or general viewpoint (Biederman,
`1985; Nakayama, & Shimojo, 1992) . The assumption
`of object size constancy would lead the visual system
`to interpret the non-changing size of the square as
`information that the square was stationary, since any
`change in depth of a rigid object would lead to a cor(cid:173)
`related change in the size of the object's image. The
`general viewpoint assumption would lead the system
`to interpret the lack of any 20 motion of the square
`also as information for stationarity, since for almost
`all viewpoints (except one "accidental" view in which
`the viewer is looking along the direction of motion),
`motion in depth of an object would cause a correlated
`2D motion of the object's image. The cues for station(cid:173)
`arity could well have led to the result that on 22% of
`the trials with dark shadows, subjects did not see the
`square move in depth. This raises the possibility that
`elimination of the stationarity cues would lead to
`
`2
`
`Legend3D, Inc. Ex. 2023-0003
`IPR2016-01243
`
`

`

`The Phenomenon
`
`Fig. 3. Three frames from animations made with the ball-in-a-box simulation. In a simulated world, a ball was
`placed in a small 132 x 132 mm box and viewed from a point 355 mm from the center of the box with an elevation
`of 21.8° relative to the floor of the box. The viewpoint was offset slightly to the right, as shown. Each animation
`was created in two stages: first, we rendered a scene with a moving ball without cast shadows. Second, we inde(cid:173)
`pendently added the ball's cast shadow to the images in an animation, so that we could manipulate the motion of
`the shadow independently of the ball's motion. The shading on the ball and in the room for all the animations,
`except those used in Experiment 3, was generated by simulating a light source at infinity with a slant of 63.4 •
`degrees relative to the floor of the box. In Experiment 3, we manipulated the shading on the ball as an indepen(cid:173)
`dent variable. In all the animations, the ball moved in a linear trajectory in the image at an angle tilted by 21.8°
`from the horizontal. Its velocity varied sinusoidally (period = 4 sec), so that the ball repeated its motion back and
`forth between its left- and right-most positions in the image. The shadow moved so that it remained vertically
`below the ball in the image. Only the distance between the shadow and the ball varied as the shadow and ball
`moved. The images shown here are copies of those used in the two animations for Demonstration 1. Figure 3a
`shows the left-most positions of the ball and shadow in both animations. Figure 3b shows the right-most positions
`in one of the animations and figure 3c shows the right-most position in the other. The demonstration animations
`were recorded on videotape, and observers were shown the taped animations. For the experiments (Experiments
`2 and 3), however, the animations were shown on the screen of a Stardent GS2000 graphics computer. Subjects
`were given the task of adjusting a line along the right wall (shown in 3b and c) to match the apparent height of the
`middle of the ball at the right-most point of its trajectory. Subjects adjusted the height of the line by moving the
`computer's mouse and indicated a match by pressing the mouse button . The motion of the ball and its shadow
`continued throughout the course of a trial.
`
`image, in this case that of a ball, remained fixed
`throughout the animation.
`
`The first demonstration using this simulation (Demon(cid:173)
`s tra ti on I) consisted of two different animation
`sequences: In the first, the ball's cast shadow followed
`a horizontal trajectory in the image (ending up at the
`position shown in figure 3b); in the second, it fol(cid:173)
`lowed a diagonal trajectory identical to that of the
`ball's image (ending up at the position shown in figure
`3c ). Despite the fact that the ball's image remained the
`same size and had an identical trajectory in the image
`plane in both animations, all observers reported the
`striking percept of seeing the ball rise above the
`checkerboard floor when the shadow trajectory was
`horizontal, and recede smoothly in depth along the
`floor when the slope of the shadow trajectory matched
`that of the ball. Because the size of the ball's image
`remained fixed, it is clear that the apparent depth from
`
`the moving cast shadow was sufficient to override the
`constant size constraint in this experiment.
`
`2.3 Demonstration 2: Apparent depth pro(cid:173)
`duced by cast shadows induces appar(cid:173)
`ent size change.
`
`If observers have an implicit perceptual assumption
`that objects do not change physical size, one would
`predict that when the slope of the shadow trajectory
`matched the ball, the ball would appear to grow in size
`as it recedes in depth. Indeed, several of our observers
`reported this perception. In Demonstration 2, every(cid:173)
`thing was as with Demonstration I, except that we tri(cid:173)
`pled the length of the box in world coordinates (figure
`4). For constant ball size, the image should decrease
`in size by about 50% if it were indeed receding to the
`back of the box. However, as before, the image of the
`ball was kept constant. The ball made a full excursion
`(in the image) from the lower left comer of the box to
`
`3
`
`Legend3D, Inc. Ex. 2023-0004
`IPR2016-01243
`
`

`

`The Stationary Light Source Constraint
`
`the upper right comer. All of our observers reported
`seeing the ball apparently inflating and shrinking
`when the trajectory of the shadow matched the ball,
`but remaining fixed in size when the shadow trajec(cid:173)
`tory was horizontal. In another study, we explicitly
`varied the image size of the ball together with the
`shadow trajectory slope and found a non-linear inte(cid:173)
`gration of the two sources of information in the per(cid:173)
`ception of the relative position of the ball
`(Mamassian, Kersten, and Knill, 1992).
`
`2.4 Demonstration 3: Moving cast shadow
`can produce the illusion of a non-linear
`object trajectory.
`
`A third demonstration (Demonstration 3) further
`shows the sophistication of human 3D motion percep(cid:173)
`tion from relative shadow motion. We modified the
`animations used for Demonstration 1 in the following
`way: the shadow was given a non-linear motion tra(cid:173)
`jectory in which it initially touched the ball's image,
`moved towards the front of the box, at mid-trajectory
`returned to touch the ball's image, and then swung to
`the front again (see figure Sa). The ball's image
`moved in the same straight, diagonal trajectory as
`before. All observers reported seeing the ball as mov(cid:173)
`ing in a non-linear 3D trajectory in which the ball
`appeared to come forward, retreat in depth, and then
`come forward again, as it moved from left to right in
`the box. Moreover, the observers reported seeing a
`singularity, or bounce, in the path of the ball when the
`shadow touched the ball's image and changed direc(cid:173)
`tion. Observers saw the bounce despite the fact that
`the ball's motion in the image was smooth at that point
`
`3.0 The Stationary Light Source
`Constraint
`
`Like many other monocular cues, the relative dis(cid:173)
`placement of an object's image and its cast shadow
`provides theoretically ambiguous information for spa(cid:173)
`tial layout. In order to interpret the cues, the visual
`system must use other information about the scene
`and make prior assumptions about the world. Since
`cast shadow displacement is a function of both object
`position and light source position (figure 6), the visual
`system must make implicit assumptions, or inferences
`from image data, about the position of the light source
`creating the shadows in order to infer the spatial posi(cid:173)
`tions of the casting objects. In this section, we present
`experimental data and phenomenal demonstrations
`which reveal the nature of the information and prior
`assumptions about light source position which the
`visual system brings to bear on the interpretation of
`cast shadow motion ..
`
`4
`
`For static images of objects with cast shadows, the
`visual system must either assume a single light source
`illuminating all the objects in a scene or estimate the
`positions of different light sources illuminating the
`different objects. The phenomenal demonstration in
`
`Fig. 4. The top and bottom panels show the extreme
`right position of the ball for the horizontal and diagonal
`shadow
`trajectories,
`respectively.
`In
`these static
`images, the effect of the shadow on the apparent size
`of the ball is small, but noticeable. In the dynamic case
`with diagonal trajectory,
`the ball has the striking
`appearance of inflating as it moves from left to right.
`For the horizontal trajectory, the ball appears to remain
`the same size.
`
`Fig. 6. A displacement L1S between an object and its
`shadow can be produced either by a change in light
`source position, L1L or by a change in depth of the
`object, L1D.
`
`Legend3D, Inc. Ex. 2023-0005
`IPR2016-01243
`
`

`

`The Stationary Light Source Constraint
`
`tion based on image data, or does it rely on prior
`assumptions about light source position?
`
`3.1 Experiment 2: A fixed light source con(cid:173)
`straint?
`
`In order to study these questions, we designed a psy(cid:173)
`chophysical paradigm to collect quantitative data on
`subject's perception of 3D motion from cast shadow
`motion. In the experiments, subjects viewed different
`ball-in-a-box animations and reported the height from
`the floor of the box to which the ball appeared to
`move at the right-most point of its trajectory (see the
`caption of figure 3 for a description of subjects' report(cid:173)
`ing method). We performed an initial, exploratory
`experiment to test whether subjects' performance
`could be fit by a model which based its estimates on a
`single, fixed position of the light source creating the
`ball's cast shadow. We tested four conditions, each
`corresponding to a different, linear shadow trajectory.
`The four trajectories had different slopes in the image
`plane, as shown in figure Sb. Figure 7 shows the
`results obtained for three observers. The height esti(cid:173)
`mates of all three subjects varied systematically with
`the slope of the shadow trajectory: smaller slopes, cor(cid:173)
`responding to larger divergences between the shadow
`and the ball, resulted in larger height estimates
`
`This reflects differences in the perceived 3D motion of
`the ball between that of receding along the floor (for
`large slopes) to that ofrising above the floor (for small
`slopes). If the observers based their setting on the
`actual light source position (which was at infinity), the
`settings would have fallen on the solid lines shown in
`the plots. While this was a good fit for only one
`observer (subject WB), we were able to obtain a better
`fit to each subject's data by finding what would
`amount to a perceptually implicit fixed light source
`position for the subject. These fits are shown with
`dashed lines. Observers behaved as if they had fabri(cid:173)
`cated a fixed illumination arrangement with which to
`interpret the scene. Any such fabrication, however,
`would have to have been unconscious, for when que(cid:173)
`ried after the experiment as to where the light source
`was, observers claimed to have not thought about it.
`
`The data from Experiment 2, while suggesting that the
`visual system uses a strategy in which it effectively
`accounts for light source position when interpreting
`cast shadow motion, does not directly answer either of
`the two questions we posed at the beginning of this
`section. We consider first the question of a fixed light
`source constraint and then turn to a consideration of
`whether and how the system estimates light source
`position. While the good fit of the fixed light source
`models to the data from Experiment 2 is consistent
`with the hypothesis that the visual system assumes a
`fixed light source constraint, observers could have
`
`Fig. 5. Two schematic diagrams of some of the trajec(cid:173)
`tories (in the image) followed by the ball and its
`shadow in the ball-in-a-box animations. Solid arrows
`in~icat~ the trajectory of the ball (constant in all the
`~rnmat1c:ins), and dashed arrows indicate the trajecto(cid:173)
`ries of its shadow. (a) A time-lapse diagram of four
`frames from the animation used for Demonstration 3
`(the n~n-linear motion). Observers reported the ball
`appeanrig to b~unce at the third position from the left
`shown in the diagram. (b) The four different shadow
`trajectories used. for Experi~en~ 2. Each trajectory cor(cid:173)
`responds to a different animation used in the experi(cid:173)
`ment.
`
`figure 1 suggests that, at least when no information
`about multiple light sources is provided in an image,
`the visual system relies on the assumption of a single
`light source (a constraint similar to the light source
`from above constraint used to explain certain effects
`in the perception of shape from shading (Gibson,
`1950; Pentland, 1982; Ramachandran, 1988) ). In
`order to explain the perception of motion in depth
`from moving cast shadows, we suggest that the visual
`system makes a different assumption about light
`sources: that the light source casting a shadow is
`fixed, at least on the time scale of the motion. We call
`this the stationary light source constraint. Such a con(cid:173)
`straint by itself supports only the qualitative percep(cid:173)
`tion of 30 object motion. In order to perceive the 3D
`motion of an object more exactly, the visual system
`must use image information or make assumptions
`about the exact position of the light source. Our dis(cid:173)
`cussion suggests two questions about the role of per(cid:173)
`ceived light source position in the visual system's
`interpretation of cast shadow motion: First, does the
`system rely on a fixed light source constraint? Second,
`in making quantitative estimates of object motion,
`does the visual system estimate the light source posi-
`
`5
`
`Legend3D, Inc. Ex. 2023-0006
`IPR2016-01243
`
`

`

`The Stationary Light Source Constraint
`
`40
`35
`30
`25
`20
`15
`10
`5
`0
`40
`35
`
`e 30
`.s
`~
`O'>
`·a;
`:c
`
`25
`20
`15
`10
`5
`0
`40
`35
`30
`25
`20
`15
`10
`5
`0
`
`WB
`
`GOA
`
`+-·-- -..
`PB
`
`··-- ~
`
`--- Fixed light source fit
`-Actual light source
`
`0
`
`0.2
`
`0.6
`0.4
`Slope
`
`0.8
`
`Fig. 7. Perceived height above the checkerboard floor
`of the ball, in the coordinates of the 30 simulated
`world, as a function of the shadow slope. Data are
`shown for three subjects. Each point is the mean of 8
`measurements. Error bars indicate 1 S.E. of the
`mean. As the shadow's trajectory slope goes from
`zero (horizontal) to one (identical to ball), the apparent
`peak height of the ball falls. The solid line shows the
`physically correct setting based on
`the light source
`direction used to render the scene. The dashed lines
`show fits to the data for a model in which each subject
`bases his or her estimate of object motion on an some
`different fixed light source position. In terms of dis(cid:173)
`tance (mm) from the middle of the checkerboard floor
`and slant (deg) with the floor, the light positions used
`to fit the data were: (419 mm, 60.8"); (105 mm, 50.4");
`and, (67 mm, 46.8") for observers WB, GOA, and PB,
`respectively.
`
`used information in the stimulus (e.g. the shading on
`the ball and on the walls of the box) to infer that the
`light source was fixed. A stronger test of the hypothe(cid:173)
`sized constraint would be to test whether the visual
`system can account for a moving light source in its
`interpretation of cast shadow motion when appropri(cid:173)
`ate information about the motion of the light is pro(cid:173)
`vided in a sequence of images.
`
`6
`
`3.2 Demonstrations 4-7: Can the visual sys(cid:173)
`tem account for a moving light source?
`
`In order to answer this question, we made a number of
`animations using a moving light source to generate
`the cast shadows. The animations were designed so
`that observers should see qualitatively different object
`motions if they assume a fixed light source constraint
`than if they accounted for the light source motion. All
`the animations were based on a realistic 30 simula(cid:173)
`tion of a ball oscillating in the front plane of the box.
`The motion of the ball was chosen to give the same
`image trajectory as was used in the previous demon(cid:173)
`strations and experiments (moving diagonally in the
`image plane, with no change in size). Unlike in the
`previous demonstrations and experiments, we gener(cid:173)
`ated shadows for these animations by rendering the
`scene with ray-tracing from the light source; however,
`we simulated a moving light source whose motion
`gave rise to different trajectories for the cast shadows.
`In these animations, the continuously changing shad(cid:173)
`ing on the ball and in the room provided information
`for the motion of the light source. A system which
`could effectively discount this motion should see the
`same 3D motion of the ball in all the animations (the
`"correct" interpretation given the way the animations
`were generated).
`
`Three demonstrations support the hypothesis that the
`visual system relies on a fixed light source constraint
`when interpreting shadow motion. For the first of the
`demonstrations (Demonstration 4), we made two ani(cid:173)
`mations in which the simulated light source motions
`gave rise to cast shadow trajectories mimicking those
`used in Demonstration 1 (one following the ball, the
`other moving horizontally in the image). As in Dem(cid:173)
`onstration 1, all observers reported seeing the ball as
`moving along different 30 trajectories in the two ani(cid:173)
`mations. When asked to compare the perceived object
`motions in these animations with those in the anima(cid:173)
`tions used for Demonstration 1, all observers reported
`that they appeared the same. This suggests that the
`observers were not able to incorporate the information
`for a moving light source into their estimation of
`object motion. The result, however, may have arisen
`either because observers interpreted the changing
`shading of the ball as being due to something other
`than a moving light source or because the changing
`shading on the ball and in the room did not provide
`sufficient information to induce the percept of a mov(cid:173)
`ing light source. In support of the former hypothesis,
`several observers reported that the ball appeared to
`rotate and that the shading on the ball then appeared to
`be from markings on the ball's surface. In order to
`control for this effect, we repeated Demonstration 4
`using an ellipsoidal instead of a spherical ball (Dem(cid:173)
`onstration 5). This led to a correct interpretation of the
`shading pattern (the ellipsoid did not appear to rotate);
`
`Legend3D, Inc. Ex. 2023-0007
`IPR2016-01243
`
`

`

`Discussion
`
`that their percepts of non-linear 30 motion were the
`same for both animations. Taken together, Demonstra(cid:173)
`tions 4 -7 provide strong evidence that the human
`visual system incorporates an assumption of a fixed
`light source in its interpretation of 3D object motion
`from cast shadow motion, and that it ignores even
`strong evidence to the contrary.
`
`3.3 Experiment 3: Is effective light source
`direction determined by prior assump(cid:173)
`tions or image data?
`
`The question remains as to how the human visual sys(cid:173)
`tem incorporates knowledge of light source position
`in generating percepts of 3D object motion from cast
`shadow motion. In a final experiment (Experiment 3),
`we tested whether subjects' implicit light source direc(cid:173)
`tion is determined by the shading information on the
`ball or a prior bias. We ran the same ball-in-a-box
`experiment used for Experiment 1 with three different
`shading conditions for the ball, corresponding to three
`different, fixed light source positions (see figure 6
`caption). If observers used the ball's shading to deter(cid:173)
`mine a light source direction for the estimation of 3D
`object motion from shadow motion, subjects' esti(cid:173)
`mates of the ball's height at the end of its trajectory
`should have varied accordingly. The data (figure 6)
`showed a very small but significant effect consistent
`with observers' usage of shading information to indi(cid:173)
`cate light source direction. The size of the effect, how(cid:173)
`ever, was far from what would be predicted
`theoretically, suggesting that in this experiment, a
`strong prior bias for a default light source position
`determined performance. Stronger image information
`for light source position than that provided by the
`ball's shading may have a greater influence on the sub(cid:173)
`jects' interpretation of cast shadow motion.
`
`4.0 Discussion
`
`Our results raise a number of issues about the compu(cid:173)
`tations involved in the perception of 3D motion from
`shadow motion. Although we have shown that mov(cid:173)
`ing cast shadows are sufficient to produce apparent
`motion in depth, we have not delineated the specific
`properties that shadows must have to perceptually
`"link" them with their casting objects to produce the
`apparent motion. Experiment 1 showed that physi(cid:173)
`cally implausible light shad