Mass Storage System Reference Model:
`Version 4 (May, 1990)
`
`Developed by the IEEE Technical Committee on Mass
`Storage Systems and Technology
`
`Edited by:
`
`Sam Coleman
`Lawrence Livermore National Laboratory
`
`Steve Miller
`SRI International
`
`All rights reserved by the Technical Committee on Mass Storage Systems and Technology,
`Institute of Electrical and Electronics Engineers, Inc.
`
`This is an unapproved draft subject to change and cannot be presumed to reflect the position of the
`Institute of Electrical and Electronics Engineers, Inc.
`
`Oracle Exhibit 1002, page 1
`
`

`

`i i
`
`Mass Storage System Reference Model:
`
`Mass Storage System Reference Model:
`Version 4 (May, 1990)
`
`Contents
`
`Page
`
`1 .
`
`2 .
`
`3 .
`
`Preface ....................................................................................................................... 1
`1 . 1 Transparencies............................................................................................... 2
`1 . 2 Requirements ................................................................................................. 3
`Introduction................................................................................................................ 5
`2 . 1 Background..................................................................................................... 5
`2 . 2 Motivation ...................................................................................................... 9
`2 . 3 Reference Model Architecture......................................................................1 0
`2.3.1 Abstract Objects............................................................................1 0
`2.3.2 Client/Server Properties.............................................................1 0
`2.3.3 Reference Model Modules ..............................................................1 1
`Detailed Description of the Reference Model ...........................................................1 5
`3 . 1 Bitfile Client ................................................................................................1 5
`3 . 2 Name Server.................................................................................................1 6
`3 . 3 Bitfile Server...............................................................................................1 7
`3.3.1 Bitfile Server Commands..............................................................1 8
`3.3.2 Bitfile Request Processor .............................................................1 9
`3.3.3 Bitfile Descriptor Manager and Descriptor Table........................2 0
`3.3.4 Bitfile ID Authenticator ................................................................2 1
`3.3.5 Migration Manager........................................................................2 1
`3.3.6 Space Limit Manager.....................................................................2 2
`3 . 4 Storage Server .............................................................................................2 2
`3.4.1 Physical Storage System...............................................................2 2
`3.4.2 Physical Device Manager ..............................................................2 3
`3.4.3 Logical-to-Physical Volume Manager ..........................................2 4
`3 . 5 Physical Volume Repository ........................................................................2 5
`3 . 6 Communication Service................................................................................2 8
`3 . 7 Site Manager.................................................................................................2 9
`3.7.1 Storage Management......................................................................3 0
`3.7.1.1 Requirements .................................................................3 0
`3.7.1.2 Tools Needed....................................................................3 0
`3.7.2 Operations .....................................................................................3 1
`3.7.2.1 Requirements .................................................................3 1
`3.7.2.2 Tools Needed....................................................................3 1
`3.7.3 Systems Maintenance ....................................................................3 1
`3.7.3.1 Requirements .................................................................3 1
`3.7.3.2 Tools Needed....................................................................3 1
`3.7.4 Software Support ..........................................................................3 2
`3.7.4.1 Requirements .................................................................3 2
`3.7.4.2 Tools Needed....................................................................3 2
`3.7.5 Hardware Support.........................................................................3 2
`3.7.5.1 Requirements .................................................................3 2
`3.7.5.2 Tools Needed....................................................................3 2
`3.7.6 Administrative Control .................................................................3 3
`3.7.6.1 Requirements .................................................................3 3
`3.7.6.2 Tools Needed....................................................................3 3
`4. References.....................................................................................................................3 4
`5. Glossary ........................................................................................................................3 7
`
`Oracle Exhibit 1002, page 2
`
`

`

`Version 4 (May, 1990)
`
`1
`
`
`
`                                                            
`
`                                                       
`
`1 . Pr eface
`                                                            
`
`
`                                                       
`
`The purpose of this reference model is to
`identify
`the high-level abstractions
`that
`underlie modern storage systems. The i n -
`formation to generate the model was c o l -
`lected from major practitioners who have
`built and operated large storage facilities,
`and represents a distillation of the wisdom
`they have acquired over
`the years. The
`model provides a common terminology and
`set of concepts to allow existing systems to
`be examined and new systems to be discussed
`and built. It is intended that the model and
`the interfaces identified from it will allow
`and encourage vendors to develop mutually-
`compatible storage components that can be
`combined to form integrated storage systems
`and services.
`
`The reference model presents an abstract
`view of the concepts and organization of
`storage systems. From this abstraction w i l l
`come the identification of the interfaces and
`modules that will be used in IEEE storage
`system standards. The model is not yet
`suitable as a standard; it does not contain
`implementation decisions, such as how a b -
`stract objects should be broken up into
`software modules or how software modules
`should be mapped to hosts; it does not give
`policy specifications, such as when files
`should be migrated; does not describe how
`the abstract objects should be used o r
`connected; and does not refer
`to specific
`hardware components. In particular, it does
`not fully specify the interfaces.
`
`A storage system is the portion of a com-
`puting facility
`responsible
`for
`the long-
`term storage of large amounts of informa-
`tion. It is usually viewed as a shared facility
`and has traditionally been organized around
`specialized hardware devices.
`It usually
`contains a variety of storage media that
`offer a range of tradeoffs among cost, p e r -
`formance, reliability, density, and power
`requirements. The storage system includes
`the hardware devices for storing informa-
`tion, the communication media for t r a n s -
`ferring
`information, and
`the software
`
`modules for controlling
`managing the storage.
`
`the hardware and
`
`The size and complexity of this software i s
`often overlooked, and
`its
`importance
`i s
`growing as computing systems become
`larger and more complex. Large storage
`facilities tend to grow over a period of years
`and, as a result, must accommodate a c o l -
`lection of heterogeneous equipment from a
`variety of vendors. Modern computing f a -
`cilities are putting increasing demands on
`their storage facilities. Often, large n u m -
`bers of workstations as well as specialized
`computing machines such as mainframes,
`mini-supercomputers, and supercomputers
`are attached to the storage system by a
`communication network. These computing
`facilities are able to generate both large
`numbers of files and large files, and the
`requirements for transferring
`information
`to and
`from
`the storage system often
`overwhelms the networks.
`
`The type of environment described above i s
`the one that places the greatest strain on a
`storage system design, and the one that most
`needs a storage system. The abstractions i n
`the reference model were selected to a c -
`commodate this type of environment. While
`they are also suitable for simpler e n v i -
`ronments, their desirability is perhaps best
`appreciated when viewed from
`the p e r -
`spective of
`the most complicated e n v i -
`ronment.
`
`There is a spectrum of system architec-
`tures, from storage services being supplied
`as single nodes specializing
`in
`long-term
`storage to what is referred
`to as “ f u l l y
`distributed systems”. The steps
`in
`this
`spectrum are most easily distinguished by
`the transparencies that they provide, where
`they are provided in the site configuration,
`and whether they are provided by a site
`administrator or by system management
`software. The trend toward distributed s y s -
`tems is appealing because it allows a l l
`storage to be viewed in the same way, as
`part of a single, large, transparent storage
`
`Oracle Exhibit 1002, page 3
`
`

`

`2
`
`Mass Storage System Reference Model:
`
`space that can be globally optimized. This i s
`especially important as systems grow more
`complex and better use of storage is r e -
`quired to achieve satisfactory performance
`levels. Distributed systems also tend
`to
`break the dependence on single, powerful
`storage processors and may
`increase
`availability by reducing reliance on single
`nodes.
`
`1 . 1 Transparencies
`
`Many aspects of a distributed system are
`irrelevant
`to a user of the system. As a
`result,
`it is often desirable to hide these
`details from the user and provide a higher-
`level abstraction of the system. Hiding de-
`tails of system operation or behavior from
`users is known as providing transparency
`for
`those details. Providing
`transparency
`has the effect of reducing the complexity of
`interacting with
`the system and thereby
`improving
`the dependability, maintain-
`ability,
`and usability
`of applications.
`Transparency also makes
`it possible
`to
`change the underlying system because the
`hidden details will not be embedded in a p -
`plication programs or operating practices.
`
`The disadvantage of using transparency i s
`that some efficiency can be lost in resource
`usage or performance. This occurs because
`the mechanism that provides
`the t r a n s -
`parency masks semantic
`information and
`causes the system to be used conservatively.
`High-performance data base systems, f o r
`example, may need to organize disk storage
`directly and schedule disk operations to gain
`performance, rather than depend on l o w e r -
`level file systems with their own structure,
`scheduling, and policies
`for caching and
`migration.
`
`that can be
`There is a range of support
`provided
`for distributed systems
`in a
`computer network. A system with
`few
`transparencies is often called a networked
`system. The simplest kind of networked
`system provides utilities to allow a program
`to be started on a specified host and i n f o r -
`mation to be transferred between specified
`storage devices. Examples include TELNET
`and FTP, respectively. This type of system
`rarely provides support for heterogeneity.
`At the other end of the spectrum are f u l l y
`distributed systems
`that provide many
`
`transparencies. An example is LOCUS. I n
`distributed systems, a goal is for worksta-
`tions to appear to have unlimited storage and
`processing capacities.
`
`System and application designers must think
`carefully about what transparencies will be
`provided and whether they will be manda-
`tory.
`It
`is possible
`for applications
`to
`provide certain
`transparencies and not
`others. Fundamental transparencies can be
`implemented by the system, saving each
`user
`from
`re-implementing
`them. A
`common implementation will also improve
`the likelihood that the transparency will be
`implemented efficiently.
`
`The common transparencies are:
`
`Access
`Clients do not know if objects or services
`are local or remote.
`
`Concurrency
`Clients are not aware that other clients
`are using services concurrently.
`
`Data representation
`Clients are not aware that different data
`representations are used
`in different
`parts of the system.
`
`Execution
`Programs can execute in any location
`without being changed.
`
`F a u l t
`Clients are not aware that certain faults
`have occurred.
`
`I d e n t i t y
`Services do not make use of the identity of
`their clients.
`
`Location
`Clients do not know where objects o r
`services are located.
`
`M i g r a t i o n
`Clients are not aware that services have
`moved.
`
`Naming
`Objects have globally unique names which
`are independent of resource and accessor
`location.
`
`Oracle Exhibit 1002, page 4
`
`

`

`Version 4 (May, 1990)
`
`3
`
`Performance
`Clients see the same performance r e -
`gardless of the location of objects and
`services (this
`is not always achievable
`unless the user is willing
`to slow down
`local performance).
`
`Replication
`Clients do not know if objects or services
`are replicated, and services do not know
`if clients are replicated.
`
`Semantic
`The behavior of operations is independent
`of the location of operands and the type of
`failures that occur.
`
`Syntactic
`Clients use the same operations and p a -
`rameters to access local and remote o b -
`jects and services.
`
`Some of the transparencies overlap or i n -
`clude others.
`
`With this in mind, it is incumbent upon the
`Storage System Standards Working Group to
`identify
`interfaces and modules that are
`invariant from single storage nodes to f u l l y
`distributed systems. Many sites are not
`likely to embrace fully distributed systems
`in a single step. Rather, they are likely to
`evolve gradually as growing system size and
`complexity dictate and as vendors make
`available products supporting
`fully d i s -
`tributed systems.
`
`1 . 2 Requirements
`
`Modern computing facilities are large and
`complex. They contain a diverse collection of
`hardware
`connected by
`communication
`networks, and are used by a wide variety of
`users with a spectrum of often-conflicting
`requirements. The hardware
`includes a
`range of processors from personal com-
`puters and workstations to mainframes and
`supercomputers, and many types of storage
`devices such as magnetic disks, optical
`disks, and magnetic tapes. This equipment i s
`typically supplied by a variety of vendors
`and, as a result, is usually heterogeneous.
`Both the hardware characteristics and the
`user requirements make this type of facility
`extremely complicated.
`
`To insure that the reference model applies
`to many computer environments, the IEEE
`Technical Committee on Mass Storage
`Systems and Technology identified the f o l -
`lowing requirements:
`
`• The model should support both cen-
`tralized and distributed hierarchical,
`multi-media file systems.
`
`• The model should support the simplest
`randomly addressable file abstraction
`out of which higher level file s t r u c -
`tures can be created (e.g., a segment of
`bits or bytes and a header of a t -
`tributes).
`
`• Where the defined services are a p -
`propriate,
`the model should use n a -
`tional or international standard p r o -
`tocols and
`interfaces,
`or
`subsets
`thereof.
`
`• The model should be modular such that
`it meets the following needs:
`
`- The modules should make sense to
`produce commercially.
`
`-
`
`It should be reasonable to integrate
`modules from two or more vendors.
`
`- The modules should integrate with
`each other and existing operating
`systems
`(centralized
`and
`d i s -
`tributed), singly or together.
`
`-
`
`It should be possible to build h i e r -
`archical centralized or distributed
`systems from the standard modules.
`The hierarchy might
`include,
`f o r
`example, solid state disks, rotating
`disks (local and remote), an on-line
`library of archival
`tape cartridges
`or optical disks, and an off-line,
`manually-operated archival vault.
`
`- Module interfaces should remain the
`same even though implementations
`may be replaced and upgraded over
`time.
`
`- Modules should have standardized
`interfaces hiding
`implementation
`details. Access
`to module objects
`should only be through these i n t e r -
`faces. Interfaces should be specified
`
`Oracle Exhibit 1002, page 5
`
`

`

`4
`
`Mass Storage System Reference Model:
`
`by the abstract object data s t r u c -
`tures visible at those interfaces.
`
`- Module interfaces should be media
`independent.
`
`• File operations and parameters should
`meet the following requirements:
`
`- Access to local and remote resources
`should use the same operations and
`parameters.
`
`- Behavior of an operation should be
`independent of operand location.
`
`- Performance should be as indepen-
`dent of location as possible.
`
`-
`
`It should be possible to read and
`write both whole files and a r b i -
`trary-sized,
`randomly-accessible
`pieces of files.
`
`- The model should separate policy and
`mechanism such that
`it supports
`standard as well as vendor- or s i t e -
`specific policy submodules and i n -
`terfaces
`for access control, a c -
`counting, allocation, site manage-
`ment, security, and migration.
`
`- The model should provide for de-
`bugging, diagnostics, and mainte-
`nance.
`
`- The model should support a r e -
`quest/reply
`(transaction) oriented
`communication model.
`
`- Request and data communication
`associations should be separated to
`support high speed direct source to
`destination data channels.
`
`(e.g.
`services
`- Transformation
`translation, check summing, e n -
`cryption) should be supported.
`
`• The model should meet the following
`naming requirements:
`
`- Objects should have globally unique,
`machine-oriented names which are
`independent of resource and access
`location.
`
`- Each operating system or site e n -
`vironment may have a different
`human-oriented
`naming
`system,
`therefore human- and machine-
`oriented naming should be clearly
`separated.
`
`distributively
`unique,
`- Globally
`file
`identifiers
`generated, opaque
`the c l i e n t - t o -
`should be used at
`storage-system interface.
`
`• The model should support the following
`protection mechanism requirements:
`
`- System security mechanisms should
`assume mutual suspicion between
`nodes and networks.
`
`- Mechanism should exist to establish
`access rights independent of location.
`
`- Access list, capability or other site,
`vendor, or operating system specific
`access control should be supportable.
`
`- Security or privacy
`exist for all objects.
`
`labels should
`
`• The model should support appropriate
`lock types for concurrent file access.
`
`• Lock mechanisms for automatic m i -
`gration and caching
`(i.e., multiple
`copies of the same data or files) should
`be provided.
`
`• The model should provide mechanisms
`to aid recovery from network, client,
`server crashes and protection against
`network or interface errors.
`In p a r -
`ticular, except for file locks, the f i l e
`server should be stateless (e.g., no
`state maintained between “open” and
`“close” calls).
`
`• The model should support the concept of
`fixed and removable logical volumes as
`separate abstractions
`from physical
`volumes.
`
`•
`
`It should be possible to store one o r
`many logical volumes on a physical
`volume, and one logical volume should
`be able to span multiple physical v o l -
`umes.
`
`Oracle Exhibit 1002, page 6
`
`

`

`Version 4 (May, 1990)
`
`5
`
`
`
`                                                                                                  
`
`                 
`
`Introduction
`2 .
`                                                                                                        
`
`
`           
`
`2 . 1 Background
`
`days of computers,
`the early
`From
`“storage” has been used to refer
`to the
`levels of storage outside the central p r o -
`cessor. If “memory”
`is differentiated to be
`inside the central processor and “storage”
`to be outside, (i.e.,
`requiring an i n p u t -
`output channel to access), the first level of
`storage
`is
`called
`“primary
`storage”
`(Grossman 89). The predominant
`tech-
`nology for this level of storage has been
`magnetic disk, or solid-state memory con-
`figured to emulate magnetic disks, and w i l l
`remain so for
`the foreseeable future
`i n
`virtually every size of computer system
`from personal computers
`to supercom-
`puters. Magnetic disks connected directly to
`I/O channels are often called “local” disks
`while magnetic disks accessed through a
`network are referred
`to as “remote” o r
`“central” disks. Sometimes a solid-state
`cache
`is
`interposed between
`the main
`memory and primary
`storage. Because
`networks have altered the access to p r i m a r y
`storage we will use the terms “local s t o r -
`age” and “remote storage” to differentiate
`the different roles of disks.
`
`The next level of data storage is often a
`magnetic tape library. Magnetic tape has
`also played several roles:
`
`• On-line archive known as “long term
`storage” (e.g., less active storage than
`magnetic disk),
`
`• off-line archival storage (possibly off-
`site),
`
`• backup for critical files, and
`
`• as an I/O medium (transfer to and from
`other systems).
`
`Magnetic tape has been used in these roles
`because it has enjoyed the lowest cost-per-
`bit of any of the widely used technologies. As
`an I/O medium, magnetic tape must conform
`to standards such that
`the tape can be
`
`written on one system and read on another.
`This is not necessarily true for archival o r
`backup storage roles, where nonstandard
`tape sizes and formats can be used, even
`though there are potential disadvantages i f
`standards are not used even for these p u r -
`poses.
`
`In the early 1970s nearly every major
`computer vendor, a number of new com-
`panies, and vendors not otherwise
`in the
`computer business, developed some type of
`large peripheral storage device. Burroughs
`and Bryant experimented with spindles of
`4-ft diameter magnetic disks. Control Data
`experimented with 12
`in. wide magnetic
`tape wrapped nearly all the way around a
`drum with a head per track. The tape was
`moved to an indexed location and stopped
`while
`the drum
`rotated
`for operation.
`(Davis 82 presents an interesting com-
`parison of devices that actually got to the
`marketplace.)
`
`Examples of early storage systems are the
`Ampex Terabit Memory (TBM) (Wildmann
`75), IBM 1360 Photostore (Kuehler 6 6 ) ,
`Braegan Automated Tape Library, IBM 3850
`Mass Storage System (Harris 75, Johnson
`75), Fujitsu M861, and the Control Data
`38500. One of the earliest systems to e m -
`ploy these devices was the Department of
`Defense Tablon system (Gentile 71), which
`made use of both the Ampex TBM and the
`IBM Photostore. Much was learned about the
`software requirements from this installa-
`tion.
`
`first delivered in the late
`The IBM 1360,
`1960s,
`used write-once,
`read-many
`(WORM) chips of photographic film. Each
`chip measured 1.4 x 2.8 in. and stored 5
`megabits of data. “File modules” were
`formed of either 2250 or 4500 cells of 3 2
`chips each. The entire process of writing a
`chip, photographically developing it,
`i n -
`serting the chip in a cell, and a cell in a f i l e
`module, storing and retrieving
`for
`read,
`etc., was managed by a control processor
`similar
`to an IBM 1800. The complex
`
`Oracle Exhibit 1002, page 7
`
`

`

`6
`
`Mass Storage System Reference Model:
`
`chemical and mechanical processing r e -
`quired considerable maintenance expertise
`and, while the Photostore almost never lost
`data, the maintenance cost was largely r e -
`sponsible for its retirement. A terabit s y s -
`tem could retrieve a file in under 10 sec-
`onds.
`
`The TBM, first delivered in 1971, was a
`magnetic tape drive that used 2-inch-wide
`magnetic tape in large 25,000-foot
`reels.
`Each reel of tape had a capacity of 44 giga-
`bits and a file could be retrieved, on the
`average, in under 17 seconds. With
`two
`drives per module, a ten module (plus two
`control modules) system provided a terabit
`of storage. The drive was a digital
`r e -
`engineering of broadcast video technology.
`The drive connected to a channel through a
`controller, and cataloging was the respon-
`sibility of the host system.
`
`The Braegan Automated Tape Library was
`first delivered in the mid 1970s and con-
`sisted of special shelf storage housing
`several thousand half-inch magnetic tape
`reels, a robotic mechanism for moving reels
`between shelf storage and self-threading
`tape drives, and a control processor. This
`conceptually simple system was originally
`developed by Xytecs, sold to Calcomp, and
`then to Braegan. In late 1986, the produc-
`tion rights were acquired by Digital Storage
`Systems, Longmont, Colorado. Sizes vary,
`but up to 8,000
`tape reels (9.6 terabits)
`and 12 tape drives per cabinet are f a i r l y
`common.
`
`The IBM 3850 (Johnson 75, Harris 7 5 )
`used a cartridge with a 2.7-inch-wide,
`770-in.
`long magnetic tape. A robotic c a r -
`tridge handler moved cartridges between
`their physical storage location (sometimes
`called the honeycomb wall) and read/write
`devices. Data accessed by the host was staged
`to magnetic disk for host access. De-staging
`the changed pages (about 2 megabits) o c -
`curred when those pages became the least
`recently used pages on the staging disks.
`Staging disks consisted of a few real disk
`devices, which served as buffers to the e n -
`tire tape cartridge library. The real disks
`were divided into pages and used to make up
`many virtual disk devices that could appear
`to be on-line at any given time.
`
`Manufactured by Fujitsu and marketed i n
`this country by MASSTOR and Control Data,
`the M861 storage module uses the same data
`cartridge as the IBM 3850; however, it i s
`formatted to hold 175 megabytes per c a r -
`tridge. The M861 holds up to 316 c a r -
`tridges and provides unit capacity of 0 . 4 4
`terabits. The physical cartridges are stored
`on the periphery of a cylinder, where a
`robotic mechanism picks them for the read-
`write station. The unit achieves about 1 2 -
`second
`access
`time
`and
`5 0 0
`mounts/dismounts per hour.
`
`A spectrum of interconnection mechanisms
`was described (Howie 75) that included:
`
`• The host being entirely responsible f o r
`special hardware characteristics of the
`storage system device,
`
`•
`
`•
`
`being
`characteristics
`device
`the
`translated (by emulation in the storage
`system) to a device known by the host
`operating system, and
`
`the storage system and host software
`being combined to create a general s y s -
`tem.
`
`This has sometimes been termed moving
`from tightly coupled to loosely coupled s y s -
`tems. Loosely coupled systems use message
`passing between autonomous elements.
`
`The evolution of the architectural view of
`what constitutes a large storage system has
`been shaped by the growth in shear size of
`systems, more rapid growth of interactive
`rather than batch processing, the growth of
`networks, distributed computing, and the
`growth of personal computers, worksta-
`tions, and file servers.
`
`tracked
`Many commercial systems have
`growth rates of 60-100% per year over
`many years. As systems grow, a number of
`things change just because of size. It b e -
`comes difficult for large numbers of people
`to handle tape reels, so automating
`the
`fetching and returning and the mounting and
`dismounting of reels becomes important. As
`size increases, it also becomes more d i f f i -
`cult for humans to decide which devices to
`use for load balancing.
`
`Oracle Exhibit 1002, page 8
`
`

`

`Version 4 (May, 1990)
`
`7
`
`Because of this growth, early users of
`storage systems were forced to do much of
`the systems integration in their own site
`environments. Large portions (software and
`hardware)
`of many existing
`systems
`(Gentile 71, Penny 73, Fletcher 7 5 ,
`Collins 82, Coleman 84) were developed by
`user organizations that were faced with the
`problem
`of
`storing,
`retrieving,
`and
`managing trillions of bits and cataloging
`millions of files. The sheer size of such
`storage problems meant that only organi-
`zations such as government
`laboratories,
`which possessed sufficient systems engi-
`neering resources and talent to complete the
`integration, initially
`took on the develop-
`ment task. These individualized develop-
`ments and integrations resulted in storage
`systems that were heavily intertwined with
`other elements of each unique site.
`
`i n
`These systems initiated an evolution
`storage products in which three stages are
`readily recognizable today. During the f i r s t
`stage, a storage system was viewed as a very
`large peripheral device serving a single
`system attached to an I/O channel on a
`central processor in the same manner as
`other peripheral devices. Tasks to catalog
`the files and free space of the device, manage
`the flow of data to and from it, take care of
`backup and recovery, and the many other
`file management tasks, were added as a p -
`plication programs within
`the systems
`environment. Many decisions, such as when
`to migrate a file, were left to the user or to
`a manual operator. If data was moved from
`the storage system to local disk, two host
`channels (one for each device) were r e -
`quired plus a significant amount of main
`memory space and central processing c a -
`pability (Davis 82).
`
`i n
`During this stage, the primary effort
`to
`design was machine-room automation
`reduce the need to manually mount and
`dismount magnetic tapes.
`
`to present)
`The second stage (late 1970s
`has been characterized by centralized
`shared service that takes advantage of the
`economies of scale and provides file server
`nodes to serve several, perhaps heteroge-
`neous, systems (Svobodova 84).
`
`This stage of the storage system evolution i s
`the one that is most prevalent today. The
`
`storage system node entails using a control
`processor to perform
`the functions of the
`reference model in a storage hierarchy
`(O'Lear 82). The cost of the storage system
`makes it desirable to share these centralized
`facilities among several computational s y s -
`tems rather than provide a storage system
`for each computational system. This
`i s
`especially true when supercomputers b e -
`come a part of the site configuration.
`
`to providing storage has
`This approach
`several advantages:
`
`• The number of processors that have
`access to a file is larger than that which
`can share a peripheral device. (This
`type of access is not the same as sharing
`a file, which implies concurrent a c -
`cess.)
`
`• Multiple read-only copies of data can be
`provided, circumventing the need for a
`large number of processors having
`access to common storage devices.
`
`• Processors of different architectures
`can have access to common storage, and
`therefore to common data, if they are
`attached to a network and use a common
`protocol for bit-stream transfer.
`
`• The independence between the levels of
`storage allows
`the inclusion of new
`storage devices as they become com-
`mercially available.
`
`Some of the earliest systems in this shared
`service stage were at the Lawrence Berkeley
`National Laboratory
`(Penny
`70)
`and
`Lawrence Livermore National Laboratory
`(LLNL) (Fletcher 75, Watson 80).
`
`The Los Alamos Common File System
`(Collins 82, McLarty 84) and the system at
`the National Center
`for Atmospheric
`Research (Nelson 87, O'Lear 82) are more
`recent examples of shared, centralized
`storage system nodes.
`
`The third stage is the emerging distributed
`system. An essential feature of a distributed
`system (Enslow 78, Watson 81a, 84, 8 8 )
`is that the network nodes are autonomous,
`employing cooperative communication with
`other nodes on the network. The control
`
`Oracle Exhibit 1002, page 9
`
`

`

`8
`
`Mass Storage System Reference Model:
`
`processors of storage systems developed
`during this stage provide this capability.
`
`The view of a storage system as a distributed
`storage hierarchy is neither a device nor a
`single service node, but is the integration of
`distributed computing systems and storage
`system architectures with the elements that
`provide
`the storage service distributed
`throughout
`the system. The distributed
`computing community has been very
`i n -
`terested in the problems of providing f i l e
`mana