United States Patent
`
`[19]
`
`[11]
`
`Patent Number:
`
`5,574,950
`
`Hathorn et al.
`
`[45] Date of Patent:
`
`Nov. 12, 1996
`
`US005574950A
`
`[54]
`
`REMOTE DATA SHADOWING USING A
`MULTHVIODE INTERFACE TO
`DYNAMICALLY RECONFIGURE CONTROL
`LINK-LEVEL AND COMMUNICATION
`LINK-LEVEL
`
`[75]
`
`Inventors: Roger G. Hathorn; Bret W. Holley;
`James L. Iskiyan; William F. Mieka;
`Asim R. Qureshi, all of Tucson, Ariz.
`
`[73]
`
`Assignee:
`
`International Business Machines
`Corporation, Arrnonk, N.Y.
`
`[21]
`
`[22]
`
`[51]
`[521
`
`[53]
`
`[561
`
`Appl. No.: 203,248
`
`Filed:
`
`Mar. 1, 1994
`
`Int. Cl.6 .................................................... .. G06F 15/02
`U.S. Cl.
`............... .. 395/861; 395/200.09; 364/222.2;
`364/242.94; 364/242.5; 364/DIG. 1
`Field of Search ................................... .. 364/952, 200;
`395/650, 145, 600, 575, 200.01, 200.02,
`200.09, 821, 861
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,058,056
`5,089,958
`5,155,845
`
`10/1991 Hammer et al.
`..
`.. 364/900
`
`2/1992 Horton et al.
`.. 395/575
`10/1992 Beal et al.
`............................ .. 395/575
`
`5,404,508
`5,420,988
`5,446,871
`
`........................ .. 395/600
`4/1995 Konrad et a1.
`
`5/1995 Elliott
`...............
`395/275
`...................... .. 395/180
`8/1995 Shomler et al.
`
`FOREIGN PATENT DOCUMENTS
`
`0212425
`0359384
`
`........ G06F 13/10
`3/1987 European Pat. Off.
`........ G06F 11/10
`3/1990 European Pat. Off.
`OTHER PUBLICATIONS
`
`“Escon I/U Interface” SA22—7202—01 Enterprise Systems
`Architecture/390.
`
`Primary Examiner——Thomas C. Lee
`Assistant Examiner—Terance J. Stanton
`Attorney, Agent, or Firm—Floyd E. Anderson; Robert M.
`Sullivan
`
`[57]
`
`ABSTRACT
`
`A remote copy system incorporates dynamically modifiable
`ports on the storage controllers such that those ports can
`operate either as a control unit link-level facility or as a
`channel link~level facility. When configured as a channel
`link-level facility, a primary storage controller can appear as
`a host processor to a secondary storage controller. Using
`dynamic switches coupled between primary and secondary
`sites, fewer ESCON communication links are required since
`the ESCON communication links can function either as a
`channel or as storage controller communication link.
`
`17 Claims, 9 Drawing Sheets
`
`PRIMARY
`
`HOST
`
`SECONDARY
`
`HOST
`
`VMWARE, INC. 1005
`
`1
`
`VMWARE, INC. 1005
`
`

`
`U.S. Patent
`
`Nov. 12, 1996
`
`Sheet 1 of 9
`
`5,574,950
`
`FIG. 1
`
`PRlOR ART
`
`120
`Illllllllllllll CHANNELS
`
`
`
`
`PORTS
`
`PORTS
`
`E — H
`
`A - D
`
`E - H
`
`A - D
`
`CACHE 1_4§
`
`
`
`STORAGE STORAGE
`PATH 0
`PATH 1
`
`NVS
`
`STORAGE STORAGE
`PATH 2
`170
`PATH 3
`T V
`
`
`
` DIRECT ACCESS
`STORAGE DEVICE
`
`2
`
`

`
`U.S. Patent
`
`Nov. 12, 1996
`
`‘Sheet 2 of 9
`
`5,574,950
`
`——-—-—._.—.—_—_.._._—_...——._—_—_..—..———.—.—.——_.———.—__—
`
`PHMAHY
`
`HOST
`
`SECONDARY
`
`HOST
`
`:._.—¢_——._.-:::—.u---—-——-:———-¢--—————¢¢——-p:-——-:a-—::_-u-——-—--———-—-I
`
`.—-—.—.———.——.—-——-.—_-_..—:.————.-—
`
`3
`
`

`
`U.S. Patent
`
`Nov. 12, 1996
`
`Sheet 3 of 9
`
`5,574,950
`
`300
`
`\
`
`FIG. 3
`
`, 360
`
`_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ .. ... J... _ _ _. — —.
`301
`302
`321
`323 :
`'
`:
`DASD
`VOL A I
`VOL 3
`:
`9
`{
`:
`.
`:
`1
`I
`I
`
`»
`
`305
`
`350
`'
`
`
`
`STORAGE
`C CONTROLLER
`I:
`’
`
`.
`
`4 346
`
`E
`
`‘
`#9
`
`CONTROLLER
`
`:
`'
`:
`'
`:
`I
`{
`:
`:
`.
`I
`I
`|
`:
`
`PRIMARY
`
`I
`I 341
`I
`II
`HOST I
`I
`I
`I
`I
`
`343
`
`SECONDARY
`
`I
`I
`I
`I
`HOST I
`I
`I
`I
`I
`I
`
`I l
`
`i
`;
`u
`:
`;
`:
`:
`,
`:
`
`4
`
`

`
`U.S. Patent
`
`Nov. 12, 1996
`
`Sheet 4 of 9
`
`5,574,950
`
`401
`
`402
`
`REMOTE COPY
`PATHS INITIALIZATION
`
`CONFIGURE PATHS
`
`TO SECONDARY
`
`
`
`N0
`
`405
`
`POST ERROR
`MESSAGE
`
`ESTABUSH LOGICAL PATH
`T0 SECONDARY CONTROLLER
`
`403
`
`QUERY STATUS OF
`SECONDARY CONTROLLER
`
`IS
`
`SECONDARY CU
`
`THE CORRECT
` 410
`ONE ?
`
`
`
`
`
`INDICATE
`COMPLETION STATUS
`
`41 1
`
`POST ERROR
`MESSAGE
`
`. 4
`
`403
`
`UNK ADDRESSES
`
`407
`
` YES
`
`LOGICAL
`PATH BEING
`USED ?
`
`
`
`5
`
`

`
`U.S. Patent
`
`Nov. 12, 1996
`
`Sheet 5 of 9
`
`5,574,950
`
`501
`
`502
`
`_
`
`503
`
`ESTABLISH REMOTE
`DUAL COPY
`
`VER|FY PARAMETERS
`AND DISCONNECT
`
`CONNECT TO
`SECONDARY CU
`
`504
`/
`
`NO
`
`YE
`
`S
`
`505
`
`- oBTA|N SECONDARY
`DEVICE INFORMATION
`
`512
`
`_ COPY ALL TRACKS OF DATA »
`FROM PRI DEVICE
`TO SEC DEVICE
`
`’"
`
`5
`
`513
`
`COPY
`COMPLETE
`'7
`-
`
`YES
`
`514
`
`ENTER FULL DUPLEX
`STATE FOR BOTH
`
`PRI AND SEC DEVICES
`
`’
`
`
`
`SET PRIMARY TO
`DUPLEX PENDING
`
`POST ERROR
`STATUS
`
`51 1
`
`PRESENT ENDING
`STATUS
`
`SEND COMMAND
`TO SEC DEVICE ‘GO
`DUPLEX PENDING‘
`
`YES
`
`FIG. 5
`
`510
`
`6
`
`

`
`U.S. Patent
`
`Nov. 12, 1996
`
`Sheet 6 of 9
`
`5,574,950
`
`601
`
`PROCESS WRITE COMMAND
`FROM HOST
`
`602
`
`CHECK ALL
`
`WRITE I/O FLAG
`
`SET
`
`
`
`WRITE DATA TO
`CACHE AND DASD
`
`
`
`
`ENTER SUSPENDED STATE
`ON PRIMARY DEVICE -
`
`RECORD CHANGES IN NVS
`
` ENTER SUSPENDED STATE
`ON PRI AND SEC DEVICES -
`RECORD CHANGES
`
`CRITICAL
`VOLUME
`
`
`
`
`
`
`
`SET FLAG TO UNIT
`CHECK ALL WRITE
`CMDS TO PRI DEVICE
`
`
`
`SET CLEAN
`ENDING STATUS
`
`SET UNIT CHECK
`IN ENDING STATUS
`
`612
`
`PRESENT ENDING
`STATUS TO HOST
`
`7
`
`

`
`U.S. Patent
`
`Nov. 12, 1996
`
`Sheet 7 of 9
`
`5,574,950
`
`PRIMARY
`SENDS EPC FRAME
`EPC RESP. EXPECTED ON
`
`71 1
`
`712
`
`SECONDARY
`PROCESS EPC FRAME
`SENDS ACK FRAME
`
`
`
`YES
`
`SECONDARY
`SEND EPC FRAME WITH
`RESPONSE EPC ACTIVE
`
`PRIMARY
`PROCESS EPC FRAME
`
`71 5
`
`71 6
`
`FIG. 7
`
`701
`
`0
`7 2
`
`704
`
`ERROR
`
`705
`
`70s
`
`PRIMARY
`SENDS ELP FRAME
`
`SECONDARY
`PROCESS ELP FRAME
`
`703
`
` No
`
`YES
`
`SECONDARY
`SEND LPE FRAME
`
`PRIMARY
`PROCESS LPE FRAME
`SEND RID FRAME
`
`SECONDARY
`PROCESS RID FRAME
`SEND IDR FRAME
`
`707
`
`PRIMARY
`PROCESS IDR FRAME
`
`7°°
`
`YES
`
`709
`
`
`
`
`IS
`SERIAL
`NUMBER AND
`
`
`
`
`
`
`?
` YES
`PRIMARY
`SEND ACK FRAME
`
`
`
`
`
` 720
`
`
`SECONDARY
`PROCESS ACK FME
`
`721
`
`IS EPC
`FRAME VALID
`
`
`
`8
`
`

`
`U.S. Patent
`
`Nov. 12, 1996
`
`Sheet 8 of 9
`
`5,574,950
`
`FIG. 8
`
`801
`
`PRIMARY SENDS RQC
`
`FRAME. AS=0
`
`I SECONDARY CU PROCESSES RQC
`
`FRAME. SENDS GRANT FRAME.
`
`
`
`
`
`PRIMARY CU PROCESSES GRANT
`
`FRAME. SENDS COMMAND FRAME.
`
`AS=1.
`
`SECONDARY CU PROCESSES
`
`
`COMMAND FRAME. SENDS CMD
`
`
`
`RESP FRAME.
`
`9
`
`

`
`U.S. Patent
`
`Nov. 12, 1996
`
`Sheet 9 of 9
`
`5,574,950
`
`FIG. 9
`
`MODIFIED READ COMMAND PROTOCOLS
`
`FIRST COMMAND IN A CHAIN
`
`CHAINED COMMAND 91 1
`
`PRIMARY
`
`901
`
`PRIMARY
`
`SEND COMMAND
`
`SEND COMMAND
`
`ESCON
`ARCH.
`
`DOES
`NOT
`ALLOW
`
`FRAME.
`END BIT = o
`
`DATA REO = o
`CCw CNT = 0
`
`FRAME.
`END BIT = o
`
`DATA REO = 0
`CCW CNT = o
`
`ESCON
`ARCI-I.
`
`DOES
`NOT
`ALLOW
`
`902
`
`SECONDARY
`
`SECONDARY
`
`PROCESS COMMAND
`FRAME.
`SEND COMMAND
`RESPONSE FRAME
`
`PROCESS COMMAND
`FRAME.
`SEND COMMAND
`RESPONSE FRAME
`
`PRIMARY
`
`903
`
`PRIMARY
`
`PROCESS COMMAND
`
`RESPONSE FRAME.
`
`SEND DATA
`
`REQUEST FRAME.
`
`END BIT = 0 OR 1
`
`DATA REO = 1
`
`CCw CNT = >o<
`
`SECONDARY
`
`PROCESS DATA
`
`REQUEST FRAME.
`
`SEND DATA FRAME(S).
`
`PROCESS COMMAND
`
`RESPONSE FRAME.
`
`SEND ACCEPT
`
`COMMAND RESPONSE
`
`FRAME.
`
`SEND DATA
`
`REQUEST FRAME.
`
`END BIT = 0 OR 1
`
`DATA REQ = 1
`
`CCw CNT = xx
`
`S ECON DARY
`
`PROCESS ACCEPT
`
`COMMAND RESPONSE
`
`FRAME.
`
`PROCESS DATA
`
`REQUEST FRAME.
`
`SEND DATA FRAME(S).
`
`904
`
`10
`
`10
`
`

`
`5,574,950
`
`1
`REMOTE DATA SHADOWING USING A
`MULTIIVIODE INTERFACE T0
`DYNAMICALLY RECONFIGURE CONTROL
`LINK-LEVEL AND COMMUNICATION
`LINK-LEVEL
`
`FIELD OF THE INVENTION
`
`The present invention relates generally to remote data
`shadowing, and more particularly, to a system for remote
`data shadowing wherein communication links are operable
`in multiple modes.
`
`10
`
`BACKGROUND OF THE INVENTION
`
`Data processing systems, in conjunction with processing
`data, typically are required to store large amounts of data (or
`records), which data can be efliciently accessed, modified,
`and re-stored. Data storage is typically separated into several
`different levels, or hierarchically, in order to provide eflicient
`and cost effective data storage. A first, or highest level of
`data storage involves electronic memory, usually dynamic or
`static random access memory (DRAM or SRAM). Elec-
`tronic memories take the form of semiconductor integrated
`circuits wherein millions of bytes of data can be stored on
`each circuit, with access to such bytes of data measured in
`nano-seconds. The electronic memory provides the fastest
`access to data since access is entirely electronic.
`A second level of data storage usually involves direct
`access
`storage devices
`(DASD). DASD storage,
`for
`example, can comprise magnetic and/or optical disks, which
`store bits of data as micrometer sized magnetically or
`optically altered spots on a disk surface for representing the
`“ones” and “zeros" that make up those bits of the data.
`Magnetic DASD, includes one or more disks that are coated
`with remnant magnetic material. The disks are rotatably
`mounted within a protected environment. Each disk is
`divided into many concentric tracks, or closely spaced
`circles. The data is stored serially, bit by bit, along each
`track. An access mechanism, known as a head disk assembly
`(HDA), typically includes one or more read/write heads, and
`is provided in each DASD for moving across the tracks to
`transfer the data to and from the surface of the disks as the
`disks are rotated past the read/write heads. DASDs can store
`giga bytes of data with the access to such data typically
`measured in rnilli-seconds (orders of magnitudes slower
`than electronic memory). Access to data stored on DASD is
`slower due to the need to physically position the disk and
`HDA to the desired data storage locations.
`A third or lower level of data storage includes tape and/or
`tape and DASD libraries. At this storage level, access to data
`is much slower in a library since a robot or operator is
`necessary to select and load the needed data storage
`medium. The advantage is reduced cost for very large data
`storage capabilities, for example, tera-bytes of data storage.
`Tape storage is often used for back-up purposes, that is, data
`stored at the second level of the hierarchy is reproduced for
`safe keeping on magnetic tape. Access to data stored on tape
`and/or in a library is presently on the order of seconds.
`Having a back-up data copy is mandatory for many
`businesses as data loss could be catastrophic to the business.
`The time required to recover data lost at the primary storage
`level
`is also an important recovery consideration. An
`improvement in speed over tape or library back-up, includes
`dual copy. An example of dual copy involves providing
`additional DASD’s so that data is written to the additional
`DASDs (sometimes referred to as mirroring). Then if the
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`11
`
`2
`
`the secondary DASDs can be
`fail,
`primary DASDs
`depended upon for data. A drawback to this approach is that
`the number of required DASDs is doubled.
`Another data back-up alternative that overcomes the need
`to double the storage devices involves writing data to a
`redundant array of inexpensive devices (RAID) configura-
`tion. In this instance, the data is written such that the data is
`apportioned amongst many DASDs. If a single DASD fails,
`then the lost data can be recovered by using the remaining
`data and error correction procedures. Currently there are
`several different RAID configurations available.
`The aforementioned back-up solutions are generally suf-
`ficient to recover data in the event that a storage device or
`medium fails. These back-up methods are useful only for
`device failures since the secondary data is a mirror of the
`primary data,
`that is,
`the secondary data has the same
`volume serial numbers (VOLSERS) and DASD addresses as
`the primary data. System failure recovery, on the other hand,
`is not available using mirrored secondary data. Hence still
`further protection is required for recovering data if a disaster
`occurs destroying the entire system or even the site, for
`example, earthquakes,
`fires, explosions, hurricanes, etc.
`Disaster recovery requires that the secondary copy of data be
`stored at a location remote from the primary data. A known
`method of providing disaster protection is to back-up data to
`tape, on a daily or weekly basis, etc. The tape is then picked
`up by a vehicle and taken to a secure storage area usually
`some kilometers away from the primary data location. A
`problem is presented in this back-up plan in that it could take
`days to retrieve the back-up data, and meanwhile several
`hours or even days of data could be lost, or worse, the
`storage location could be destroyed by the same disaster. A
`somewhat improved back-up method includes transmitting
`data to a back-up location each night. This allows the data
`to be stored at a more remote location. Again, some data may
`be lost between back-ups since back-up does not occur
`continuously, as in the dual copy solution. Hence, a sub-
`stantial data amount could be lost which may be unaccept-
`able to some users.
`
`A back-up solution providing a greater degree of protec-
`tion is remote dual copy which requires that primary data
`stored on primary DASDs be shadowed at a secondary or
`remote location. The distance separating the primary and
`secondary locations depends upon the level of risk accept-
`able to the user, and for synchronous data communications,
`can vary from just across a fire-wall to several kilometers.
`The secondary or remote location, in addition to providing
`a back-up data copy, must also have enough system infor-
`mation to take over processing for the primary system
`should the primary system become disabled. This is due in
`part because a single storage controller does not write data
`to both primary and secondary DASD strings at the primary
`and secondary sites. Instead, the primary data is stored on a
`primary DASD string attached to a primary storage control-
`ler while the secondary data is stored on a secondary DASD
`string attached to a secondary storage controller.
`Remote dual copy falls into two general categories, syn-
`chronous and asynchronous. Synchronous remote copy
`allows sending primary data to the secondary location and
`confirming the reception of such‘ data before ending a
`primary DASD input/output (I/O) operation (providing a
`channel end (CE)/device end (DE) to the primary host).
`Synchronous remote copy,
`therefore, slows the primary
`DASD I/O response time while waiting for secondary con-
`firmation. Primary I/O response delay is increased propor-
`tionately with the distance between the primary and second-
`ary systems—a factor that limits the remote distance to tens
`
`11
`
`

`
`5,574,950
`
`3
`of kilometers. Synchronous remote copy, however, provides
`sequentially consistent data at the secondary site with rela-
`tively little system overhead.
`Asynchronous remote copy provides better primary appli-
`cation system performance because the primary DASD I/O
`operation is completed (providing a channel end (CE)/
`device end (DE) to the primary host) before data is con-
`firmed at the secondary site. Therefore, the primary DASD
`I/O response time is not dependent upon the distance to the
`secondary site and the secondary site could be thousands of
`kilometers remote from the primary site. A greater amount
`of system overhead is required, however, for ensuring data
`sequence consistency since data received at the secondary
`site will often arrive in an order difl’erent from that written
`on the primary DASDs. A failure at the primary site could
`result in some data being lost that was in transit between the
`primary and secondary location.
`Synchronous real time remote copy for disaster recovery
`requires that copied DASD volumes form a set. Forming
`such a set further requires that a sufiicient amount of system
`information be provided to the secondary site for identifying
`those volumes (VOLSERS) comprising each set and the
`primary site equivalents. Importantly, a volume at the sec-
`ondary site forrns a “duplex pair” with a volume at the
`primary site and the secondary site must recognize when one
`or more volumes are out of sync with the set, that is, “failed
`duplex” has occurred. Connect failures are more visible in
`synchronous remote copy than in asynchronous remote copy
`because the primary DASD 1/0 is delayed while alternate
`paths are retried. The primary site can abort or suspend copy
`to allow the primary site to continue while updates for the
`secondary site are queued. The primary site marks such
`updates to show the secondary site is now out of sync.
`Maintaining a connection between the secondary site and
`the primary site with secondary DASD present and acces-
`sible, however, does not ensure content synchronism. The
`secondary site may lose synchronism with the primary site
`for a number of reasons. The secondary site is initially out
`of sync when the duplex pair is being formed and reaches
`sync when an initial data copy is completed. The primary
`site may break the duplex pair if the primary site is unable
`to write updated data to the secondary site in which case the
`primary site writes updates to the primary DASD under
`suspended duplex pair conditions so that the updating appli-
`cation can continue. The primary site is thus running
`exposed, that is, without current disaster protection copy
`until the duplex pair is restored. Upon restoring the duplex
`pair, the secondary site is not immediately in sync. After
`applying now pending updates, the secondary site returns to
`sync. The primary site can also cause the secondary site to
`lose sync by issuing a suspend command for that volume to
`the primary DASD. The secondary site re-syncs with the
`primary site after the suspend command is ended, duplex
`pair is re—established, and pending updates are copied.
`On-line maintenance can also cause synchronization to be
`lost.
`
`When a secondary volume is out of sync with a primary
`volume, the secondary volume is not useable for secondary
`system recovery and resumption of primary applications. An
`out-of-sync volume at the secondary site must be identified
`as such and secondary site recovery-takeover procedures
`need to identify the out-of-sync volumes for denying appli-
`cation access (forcing the volumes ofi"-line or changing their
`VOLSERS). The secondary site may be called upon to
`recover the primary site at any instant wherein the primary
`site host is inaccessible — thus the secondary site requires all
`pertinent information about a sync state of all volumes.
`
`10
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`12
`
`4
`More recently introduced data disaster recovery solutions
`include remote dual copy wherein data is bacl<ed—up not only
`remotely, but also continuously. In order to conununicate
`duplexed data synchronously from one host processor to
`another host processor, or from one storage controller to
`another storage controller, or some combination thereof,
`expensive communication links are required for connecting
`each host processor and/or storage controller. Such commu-
`nication links,
`include, for example, Enterprise Systems
`Connection (ESCON) fiber optic links providing serial com-
`munication paths extending tens of kilometers.
`In a typical remote dual copy system, there may exist
`multiple primary processors connected, by multiple serial or
`parallel communication links, to multiple primary storage
`controllers, each having strings of primary DASDs attached
`thereto. A similar processing system may exist at a remote
`secondary site. Additionally, many communication links
`may be required to connect primary processors to secondary
`processors and/or secondary storage controllers, and pri-
`mary storage controllers may be connected to secondary
`storage controllers and/or secondary processors. Each com-
`munication link presents a substantial expense in the remote
`dual copy system. This expense is exacerbated by the fact
`that communication between a primary processor and a
`secondary storage subsystem, and between a primary stor-
`age subsystem and a secondary storage subsystem presently
`requires two separately dedicated communication links
`though these links may be inactive for substantial periods of
`time.
`
`Accordingly it is desired to provide a method and appa-
`ratus for providing a real time update of data consistent with
`the data at a primary processing location using shared
`communication links that can dynamically interface either a
`host processor to a storage controller, or can interface one
`storage controller to another storage controller, thus reduc-
`ing a number of communication links required.
`SUMMARY OF THE INVENTION
`
`invention is to provide an
`An object of the present
`improved design and method for communicating between
`host processors and storage controllers or between storage
`controllers.
`
`Another object of the present invention is to provide
`dynamically alterable link-level facilities.
`According to a first embodiment of the present invention,
`a method of communicating between a host processor, a first
`storage subsystem, and a second storage subsystem, the host
`processor, and first and second storage subsystems coupled
`together by at least one communication link and at least one
`dynamic switch. The host processor and each first and
`second storage subsystem having at least one link-level
`facility. The method includes machine effected steps of:
`initializing paths, wherein the host processor link-level
`facility is configured as a channel, and the first and second
`storage subsystem link-level facilities are each configured as
`a control unit link-level facility. Paths between the host
`processor, and first and second storage controllers are then
`configured. The first storage subsystem establishes logical
`paths between the first and second storage subsystems,
`wherein the first storage subsystem link-level facility is
`dynamically re-configured as a channel link-level facility,
`the first storage subsystem acting as a host to the second
`storage subsystem. Establishing pathing control is accom-
`plished wherein the tirst storage subsystem transmits an
`establish pathing control (EPC) frame to the second storage
`Controller for processing.
`
`12
`
`

`
`5,574,950
`
`5
`In mother embodiment of the present invention, a storage
`controller for communicating with a host processor and
`another storage controller over an enterprise system con-
`nection (ESCON) link is described. The host processor has
`a channel link-level facility. The storage controller com-
`prises a serial adapter for connecting to the ESCON link, and
`a storage path for coupling the storage controller with a
`storage device. A link level facility at the storage controller
`is dynamically reconfigurable as a control unit link-level
`facility for communicating with the host processor over the
`ESCON link, and further dynamically re-configurable as a
`channel
`link-level
`facility for communicating with the
`another storage controller over the ESCON link, wherein the
`storage controller acts as host to the another storage con-
`troller.
`
`The foregoing and other objects, features, and advantages
`of the invention will be apparent from the following more
`particular description of a preferred embodiment of the
`invention, as illustrated in the accompanying drawing.
`
`Description of the Figures
`
`FIG. 1 is a block diagram of a prior art data processing
`system including a host processor and storage subsystem.
`FIG. 2 is a block diagram depicting a remote dual copy
`system including primary and secondary sites.
`FIG. 3 is a block diagram of a remote dual copy system
`according to a preferred embodiment of the present inven-
`tion.
`
`10
`
`20
`
`25
`
`30
`
`35
`
`FIG. 4 is a flow diagram describing steps for initiating a
`remote copy path according to a preferred embodiment of
`the present invention.
`FIG. 5 is a flow diagram describing steps for establishing
`a remote device according to a preferred embodiment of the
`present invention.
`FIG. 6 is a flow diagram showing a write command
`intercept process at a primary site according to a preferred
`embodiment of the present invention.
`FIG. 7 is a flow diagram depicting an Establish Pathing 40
`Control sequence according to a preferred embodiment of
`the present invention.
`FIG. 8 is a flow diagram depicting steps for an expanded
`Request Connect according to a preferred embodiment of
`the present invention.
`FIG. 9 is a flow diagram showing modified Read Com-
`mand protocols according to a preferred embodiment of the
`present invention.
`
`45
`
`Detailed Description
`
`Referring to FIG. 1, a typical data processing system is
`shown including a host processor 110, such as an IBM
`Systern/370 or IBM Enterprise Systems/9000 (ES/9000)
`processor for computing and manipulating data, and run-
`ning, for example, data facility storage management sub-
`systern/multiple virtual systems (DFSMS/MVS) software,
`having at least one storage controller 125 attached thereto,
`for example an IBM 3990 storage controller. The storage
`controller 125 is further connected to a direct access storage
`device (DASD) 75, such as an IBM 3380 or 3390 DASD. A
`storage subsystem is formed by the storage controller 125
`and DASD 75. Substantial computing power is provided by
`the host processor 110 while the storage controller 125
`provides the necessary functions to effieiently transfer,
`stage/destage, convert and generally access large databases.
`The storage subsystem is connected to the host processor
`
`50
`
`55
`
`60
`
`65
`
`13
`
`6
`110 via communication links 121, wherein the communica-
`tion links 121 connect to channels 120 of the host processor
`110 and to ports A—D, E—H 130 of the storage controller 125.
`The communication links 121 can be either parallel or serial
`links, for example, enterprise system connections (ESCON)
`serial fiber optic links. The ESCON links 121 are described
`in greater detail in ESCON I/O Interface, IBM publication
`number SA22-7202, which is hereby incorporated by refer-
`ence.
`
`The storage controller 125 includes dual clusters 160 and
`161,
`the dual clusters 160, 161 having separate power
`supplies (not shown) and further including ports A-D, E—F
`130 for providing a communication interface thereto. Both
`non-volatile storage 170 and cache 145 are provided for
`temporary data storage and is accessible to both clusters 160,
`161. Storage paths 0-3 140 provide necessary paths to the
`DASD 75. Vital product data is maintained in VPDs 95 and
`96. A storage controller, similar to the storage controller 125
`is described in U.S. Pat. No. 5,051,887, assigned to the
`assignee of the present invention, and is hereby incorporated
`by reference.
`Referring now to FIG. 2, a remote dual copy system 200
`is shown having a primary site 260 and a secondary site 270,
`wherein the secondary site 270 is located, for example, 20
`kilometers remote from the primary site 260. The primary
`site 260 includes a host or primary processor 201 (herein-
`after referred to as primary host 201), for example, an IBM
`Enterprise Systems/9000 (ES/9000) processor
`running
`DFSMS/MVS operating software, and further having sev-
`eral application programs running thereon. A plurality of
`primary storage controllers 222, 225, for example, IBM
`3990 Model 6 storage controllers, are coupled to the primary
`host 201 via a dynamic switch 205, for example, an IBM
`ESCON Director. As is known in the art, several primary
`hosts 201 can be coupled to the plurality of primary storage
`controllers 222, 225. Primary DASD 223, 226, for example,
`IBM 3390 DASDs, are connected to the plurality of primary
`storage controllers 222, 225. Several primary DASDs 223,
`226 can be connected to the plurality of primary storage
`controllers 222, 225. The plurality of primary storage con-
`trollers 222, 225 and attached primary DASDs 223, 226
`form a primary storage subsystem. Alternatively, each pri-
`mary storage controller 222, 225, and corresponding pri-
`mary DASD 223, 226 could be integrated as single units.
`The primary host 201 connects to ports (not shown) of the
`dynamic switch 205 from primary host channels 202 via a
`communication link 241, for example, a serial communica-
`tion link. Similarly, ports of the dynamic switch 205 connect
`to ports (also known as link-level facilities) 221, 224 of the
`primary storage controllers 222, 225, respectively, via com-
`munication links 249, 250, for example, ESCON links. The
`primary host 201, then, can communicate with each of the
`primary storage controllers 222, 225 using the communica-
`tion links 241, 249, and 250, and the dynamic switch 205.
`The secondary site 270 includes a host or secondary
`processor 211 (hereinafter referred to as secondary host
`211), for example, an IBM ES/9000, running DFSMS/MVS
`operating software. A plurality of secondary storage con-I
`trollers 232, 235, for example, IBM 3990 Model 6 storage
`controllers, are coupled to the secondary host 211 via a
`dynamic switch 215, for example, an IBM ESCON Director.
`As is known in the art, several secondary hosts 211 can be
`coupled to the plurality of secondary storage controllers 232,
`235. Secondary DASDs 233, 236, for example, IBM 3390
`DASDs, are connected to the plurality of secondary storage
`controllers 232, 235. Several secondary DASDs 233, 236
`can be connected to the plurality of secondary storage
`
`13
`
`

`
`5,574,950
`
`7
`controllers 232, 235. The plurality of secondary storage
`controllers 232, 235 and attached secondary DASDs 233,
`236 form a secondary storage subsystem.
`The secondary host 211 connects to ports (not shown) of
`the dynamic switch 205 from secondary host channels 212
`via a communication link 244, for example, a serial com-
`munication link. Similarly, ports of the dynamic switch 215
`connect to ports 231, 234 of the secondary storage control-
`lers 232, 235, respectively, via communication links 245,
`246, for example, ESCON links. The secondary host 211,
`then, can communicate with each of the secondary storage
`controllers 232, 235 using the communication links 244,
`245, and 246, and the dynamic switch 215.
`Communications occur between the primary site 260 and
`the secondary site 270 via two mechanisms. First,
`the
`primary host is connected to the secondary site 270 via a
`charmel link 243 connected between the primary host chan-
`nel 202 and the dynamic switch 215. Similarly, the second-
`ary host is connected to the primary site 260 via a channel
`link 242 connected between the secondary host channel 212
`and the dynamic switch 205. Second, communication links
`247 connect primary storage controller 222 to secondary
`storage controller 235 via respective ports 221 and 234.
`Similarly, communication links 248 connect primary storage
`controller 225 to secondary storage controller 232 via
`respective ports 224 and 231.
`The primary host 201 can thus communicate with any
`secondary storage controller 232, 235, or the secondary host
`211 via the dynamic switch 205 or 215. Likewise,
`the
`secondary host can communicate with any primary storage
`controller 222, 225, or the primary host 201 via the dynamic
`switch 205 or 215. Additionally, primary storage controllers
`222, 225 can communicate with secondary storage control-
`lers 232, 235, respectively. Thus, the primary host 201 could
`send data or records for back—up directly to the secondary
`storage subsystem (however, this may be undesirable due to
`the required primary host resources). More desirably, pri-
`mary storage controllers 222, 225 send data or records for
`back-up directly to secondary storage controllers 232, 235,
`respectively. This communication is quicker since the pri-
`mary host need only wait until the data or records are
`received in secondary storage controllers 232, 235 cache
`(see FIG. 1).
`An area for improvement to the remote dual copy system
`200 revolves around the number of required communication
`links 241-250 and the associated expense. Furthermore,
`primary storage controller ports (A, B) 221 , 224, and
`secondary storage controller ports (C, D) 231, 234 are
`dedicated control unit link-level facilities and cannot com-
`municate as channel link-level facilities. Similarly, primary
`storage controller ports (C, D) 221, 224 are dedicated
`channel link-level facilities (and cannot communicate with
`primary host channels 202), and secondary storage control-
`ler ports (A, B) 231, 234 are dedicated control unit link-level
`facilities.
`
`FIG. 3 depicts a remote dual copy system 300 that is
`similar to but improves upon the remote dual copy system
`200. The communication links 247 and 248 are reduced to
`and replaced by communication links 351. The communi-
`cation links 351 are connected between dynamic switches
`305 and 315. Not only are the number of communication
`links reduced, but additional ports on the primary and
`secondary storage controllers are opened for other commu-
`nications, for example, a communication link 352 (serial or
`parallel) connecting a secondary host channel 312 to sec-
`ondary storage controller 335 is now available. Reducing a
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`number of communication links is enabled by modification
`of the storage controller ports or link-level facilities into
`dual function link-level facilities. For example, the primary
`and secondary storage controller ports 321, 324, 331, and
`334 can be dynamically set to communicate either as a
`channel or control unit link-level facility. Hence, primary
`storage controller 322, via port A 321, can communicate
`with primary host 301 by communication links 350,
`dynamic switch 305 and communication link 341, wherein
`port A 321 is a control unit link-level facility. Alternately,
`primary storage controller 322, via the same port A 321, can
`communicate w