versian
`5 .2
`
`Marshall Kirk McKusick
`George V. Nevill -Neil
`
`CSCO-1011
`Page 1 of 43
`
`

`

`LIBRARY OF CONGRESS
`
`Already widely used for Internet services and firewalls, high-availability
`servers, and general timesharing systems, the lean quality of
`FreeBSD also suits the growing area of embedded systems. Unlike
`Linux, FreeBSD does not require users to publicize any changes they
`make to the source code.
`
`11l11l11i11i11il
`
`111111111111111111111111
`9 780201 702453
`ISBN 0-201-70245-2
`$59.99 us
`$86. 99
`CANADA
`
`Page 2 of 43
`
`

`

`9333...min...
`
`Q :
`
`
`
`5=83:.5.—51753.533=3$33:
`
`Page 3 of 43
`
`Page 3 of 43
`
`

`

`
`
`
`
`Page 4 of 43
`
`Page 4 of 43
`
`

`

`ownlvnalmx...
`
`2%:
`
`Page 5 of 43
`
`Page 5 of 43
`
`
`

`

`
`
`_.
`
`[a
`
`Page 6 of 43
`
`Page 6 of 43
`
`

`

`The Design and Implementation of the
`Free BSD
`Operating System
`
`Marshall Kirk McKusick
`1,
`
`George V. Neville-Neil
`
`.,,,6T Addison-Wesley
`
`Boston
`
`Capetown
`
`• Toronto
`• New York
`• San Francisco
`London
`• Munich
`• Paris
`• Madrid
`• Sydney
`• Tokyo
`• Singapore
`• Mexico City
`
`• Montreal
`
`Page 7 of 43
`
`

`

`UNIX is a registered trademark of X/Open in the United States and other countries. Many
`of the designations used by manufacturer and sellers to distinguish their products are
`claimed as trademarks. Where those designations appear in this book, and Addison-Wesley
`was aware of a trademark claim, the designations have been printed with initial capital let-
`ters or in all capitals.
`
`The authors and publisher have taken care in the preparation of this book, but make no
`expressed or implied warranty of any kind and assume no responsibility for errors or omis(cid:173)
`sions. No liability is assumed for incidental or consequential damages in connection with
`or arising out o
`-of-the information or programs contained herein.
`
`McKusick, Mars all Kirk.
`The design and implementation of the FreeBSD operating system I Marshall
`Kirk McKusick, George V. Neville-Neil.
`p. cm.
`Includes bibliographical references and index.
`ISBN 0-201-70245-2 (he: alk. paper)
`1. FreeBSD. 2. Free computer software. 3. Operating systems
`(Computers) I. Neville-Neil, George V. II. Title.
`
`QA 76.76.063M398745 2004
`005.3--dc22
`
`2004010590
`
`Copyright © 2005 by Pearson Education, Inc.
`
`All rights reserved. No part of this publication may be reproduced, stored in a retrieval sys(cid:173)
`tem, or transmitted, in any form or by any means, electronic, mechanical, photocopying,
`recording, or otherwise, without the prior consent of the publisher. Printed in the United
`States of America. Published simultaneously in Canada.
`
`For information on obtaining permission for use of material from this work, please submit a
`written request to:
`Pearson Education, Inc.
`Rights and Contracts Department
`75 Arlington Street, Suite 300
`Boston, MA 02116
`Fax: (617) 848-7047
`
`Text printed on recycled and acid-free paper.
`ISBN 0-201-70245-2
`123 4 5 6 7 8 9 10-CRW-0807060504
`First Printing, July 2004
`
`Page 8 of 43
`
`

`

`.
`
`I
`
`-
`
`Dedication
`
`This book is dedicated to the BSD community.
`Without the contributions of that community's members,
`there would be nothing about which to write.
`
`Page 9 of 43
`
`

`

`
`
`Page 10 of 43
`
`Page 10 of 43
`
`

`

`Preface
`
`This book follows the earlier authoritative and full-length descriptions of the
`design and implementation of the 4.3BSD and 4.4BSD versions of the UNIX sys(cid:173)
`tem developed at the University of California at Berkeley. Since the final
`Berkeley release in 1994, several groups have continued development of BSD.
`This book details FreeBSD, the system with the largest set of developers and the
`most widely distributed releases. Although the FreeBSD distribution includes
`nearly 1000 utility programs in its base system and nearly 10,000 optional utilities
`in its ports collection, this book concentrates almost exclusively on the kernel.
`
`UNIX-like Systems
`
`UNIX-like systems include the traditional vendor systems such as Solaris and
`HP-UX; the Linux-based distributions such as Red Hat, Debian, Suse, and
`Slackware; and the BSD-based distributions such as FreeBSD, NetBSD, OpenBSD,
`and Darwin. They run on computers ranging from laptops to the largest super(cid:173)
`computers. They are the operating system of choice for most multiprocessor,
`graphics, and vector-processing systems, and are widely used for the original pur(cid:173)
`pose of timesharing. The most common platform for providing network services
`(from FTP to WWW) on the Internet, they are collectively the most portable oper(cid:173)
`ating system ever developed. This portability is due partly to their implementation
`language, C [Kernighan & Ritchie, 1989] (which is itself a widely ported lan(cid:173)
`guage), and partly to the elegant design of the system.
`Since its inception in 1969 [Ritchie & Thompson, 1978], the UNIX system
`has developed in several divergent and rejoining streams. The original developers
`continued to advance the state of the art with their Ninth and Tenth Edition UNIX
`inside AT&T Bell Laboratories, and then their Plan 9 successor to UNIX. Mean(cid:173)
`while, AT&T licensed UNIX System V as a product before selling it to Novell.
`Novell passed the UNIX trademark to X/OPEN and sold the source code and distri(cid:173)
`bution rights to Santa Cruz Operation (SCO). Both System V and Ninth Edition
`
`xix
`
`Page 11 of 43
`
`

`

`xx
`
`Preface
`
`UNIX were strongly influenced by the Berkeley Software Distributions produced
`by the Computer Systems Research Group (CSRG) of the University of California
`at Berkeley. The Linux operating system, although developed independently of
`the other UNIX vruiants, implements the UNIX interface. Thus, applications
`developed to run on other UNIX-ha ed platforms can be easily ported to run on
`Linux.
`
`Berkeley Software Distributions
`
`The di tributions from Berkeley were the first UNIX-based systems to introduce
`many important features including the fclloWifig:
`
`• Demand-paged virtual-memory support
`•Automatic configuration of the hardware and 1/0 system
`• A fast and recoverable filesystem
`•The socket-based interprocess-communication (IPC) primitives
`•The reference implementation of TCP/IP
`
`The Berkeley release found their way into the UNIX systems of many vendors
`and were used internally by the development groups of many other vendors. The
`implementation of the TCP/IP networking protocol suite in 4.2BSD and 4.3BSD,
`and the availability of those systems, played a key role in making the TCP/IP net(cid:173)
`working protocol suite a world standard. Even the non-UNIX vendors such as
`Microsoft have adopted the Berkeley socket design in their Winsock IPC interface.
`The BSD releases have also been a strong influence on the POSIX (IEEE Std
`1003. l) operating-system interface standard, and on related standards. Several
`features-such as reliable signals, job control, multiple access groups per process,
`and the routines for directory operations-have been adapted from BSD for
`POSIX.
`Early BSD releases contained licensed UNIX code, thus requiring recipients to
`have an AT&T source license to be able to obtain and use BSD. In 1988, Berkeley
`separated its distribution into AT&T licen ed and freely redistributable code. The
`freely redistributable code was licensed sepru·ately and could be obtained, used,
`and redistributed by anyone. The final freely redistributable 4.4BSD-Lite2 release
`from Berkeley in 1994 contained nearly the entire kernel and all the important
`libraries and utilities.
`Two groups, NetBSD and FreeBSD, sprang up in 1993 to begin supporting and
`distributing systems built from the freely redistributable releases being done by
`Berkeley. The NetBSD group emphasized portability and the minimalist approach,
`porting the systems to nearly forty platforms and pushing to keep the system lean
`to aid embedded applications. The FreeBSD group emphasized maximal support
`for the PC architecture and pushed to ease installation for, and market their system
`to, as wide an audience as possible. In I 995, the OpenBSD group split from the
`NetBSD group to develop a distribution that emphasized security. Over the years
`there bas been a healthy competition among the BSD distributions, with many
`ideas and much code flowing between them.
`
`Page 12 of 43
`
`

`

`Preface
`
`xxi
`
`Material Covered in this Book
`
`This book is about the internal structure of the FreeBSD 5.2 kernel and about the
`concepts, data structures, and algorithms used in implementing FreeBSD's system
`facilities. Its level of detail is similar to that of Bach's book about UNIX System v
`[Bach, 1986]; however, this text focuses on the facilities, data structures, and algo(cid:173)
`rithms used in the FreeBSD variant of the UNIX operating system. The book cov(cid:173)
`ers FreeBSD from the system-call level down-from the interface to the kernel to
`the hardware itself. The kernel includes system facilities, such as process man(cid:173)
`agement, virtual memory, the I/O system, filesystems, the socket IPC mechanism,
`and network protocol implementations. Material above the system-call level(cid:173)
`such as libraries, shells, commands, programming languages, and other user inter(cid:173)
`faces-is excluded, except for some material related to the terminal interface and
`to system startup. Following the organization first established by Organick's book
`about Multics [Organick, 1975], this book is an in-depth study of a contemporary
`operating system.
`Where particular hardware is relevant, the book refers to the Intel Personal
`Computer (PC) architecture. Because FreeBSD has emphasized development on
`the PC, that is the architecture with the most complete support, so it provides a
`convenient point of reference.
`
`Use by Computer Professionals
`
`FreeBSD is widely used to support the core infrastructure of many companies
`worldwide. Because it can be built with a small footprint, it is also seeing
`increased use in embedded applications. The licensing terms of FreeBSD do not
`require the distribution of changes and enhancements to the system. The licensing
`terms of Linux require that all changes and enhancements to the kernel be made
`available in source form at minimal cost. Thus, companies that need to control the
`distribution of their intellectual property build their products using FreeBSD.
`This book is of direct use to the professionals who work with FreeBSD sys(cid:173)
`tems. Individuals involved in technical and sales support can learn the capabilitie
`and limitations of the system; applications developers can learn how to effectively
`and efficiently interface to the system; system administrators without direct expe(cid:173)
`rience with the FreeBSD kernel can learn how to maintain, tune, and configure the
`system; and systems programmers can learn how to extend, enhance, and interface
`to the system.
`Readers who will benefit from this book include operating-system implemen(cid:173)
`tors, system programmers, UNIX application developers, administrators, and curi(cid:173)
`ous users. The book can be read as a companion to the source code of the system,
`falling as it does between the manual pages and the code in detail of treatment.
`But this book is neither exclusively a UNIX programming manual nor a user tuto(cid:173)
`rial (for a tutorial, see Libes & Ressler [1988]). Familiarity with the use of some
`version of the UNIX system (see, for example, Stevens [1992]) and with the C pro(cid:173)
`gramming language (see, for example, Kernighan & Ritchie [1989]) would be
`extremely useful.
`
`Page 13 of 43
`
`

`

`Page 14 of 43
`
`Page 14 of 43
`
`

`

`CHAPTER
`
`History and Goals
`
`1.1 History of the UNIX System
`
`The UNIX system has been in wide use for over 30 years and has helped to define
`many areas of computing. Although numerous individuals and organizations have
`contributed (and still contribute) to the development of the UNIX system, this book
`primarily concentrates on the BSD thread of development.
`
`• Bell Laboratories, which invented UNIX
`
`•The Computer Systems Research Group (CSRG) at the University of California
`at Berkeley, which gave UNIX virtual memory and the reference implementation
`of TCP/IP
`
`• The FreeBSD project, the NetBSD project, and the OpenBSD project, which con(cid:173)
`tinue the work started by the CSRG
`
`•The Darwin operating system at the core of Apple's OS X. Darwin is based on
`FreeBSD.
`
`Origins
`
`The first version of the UNIX system was developed at Bell Laboratories in 1969
`by Ken Thompson as a private research project to use an otherwise idle PDP-7.
`Thompson was joined shortly thereafter by Dennis Ritchie, who not only con(cid:173)
`tributed to the design and implementation of the system, but also invented the C
`programming language. The system was completely rewritten into C, leaving
`almost no assembly language. The original elegant design of the system [Ritchie,
`1978] and developments of the first 15 years [Ritchie, 1984a; Compton, 1985]
`have made the UNIX system an important and powerful operating system [Ritchie,
`1987].
`
`Page 15 of 43
`
`

`

`4
`
`Chapter l
`
`History and Goals
`
`Ritchie, Thompson, and other early UNIX developers at Bell Laboratories had
`worked previously on the Multics project [Peirce, 1985; Organick, 1975], which
`had a strong influence on the newer operating system. Even the name UNIX is
`merely a pun on Multics; in areas where Multics attempted to do many tasks,
`UNIX tried to do only one task but do it well. The basic organization of the UNIX
`filesystem, the idea of using a user process for the command interpreter, the gen(cid:173)
`eral organization of the filesystem interface, and many other system characteristics
`come directly from Multics.
`Ideas from various other operating systems, such as the Massachusetts Institute
`of Technology's (MIT's) CTSS, also have been incorporated. The fork operation to
`create new processes comes from Berkeley's GENIE (SDS-940, later XDS-940)
`operating system. Allowing a user to create processes inexpensively led to using
`one process per command rather than commands being run as procedure calls, as
`is done in Multics.
`
`Research UNIX - - - - -
`The first major editions of UNIX were the Research systems from Bell Laboratories.
`In addition to the earliest versions of the system, these systems include the UNIX
`Time-Sharing System, Sixth Edition, commonly known as V6, which in 1976 was
`the first version widely available outside of Bell Laboratories. Systems are identi(cid:173)
`fied by the edition numbers of the UNIX Programmer's Manual that were current
`when the distributions were made.
`The UNIX system was distinguished from other operating systems in three
`important ways.
`
`1. It was written in a high-level language.
`
`2. It was distributed in source form.
`
`3. It provided powerful primitives normally found in only those operating sys(cid:173)
`tems that ran on much more expensive hardware.
`
`Most of the system source code was written in C rather than in assembly lan(cid:173)
`guage. The prevailing belief at the time was that an operating system had to be
`written in assembly language to provide reasonable efficiency and to get access to
`the hardware. The C language itself was at a sufficiently high level to allow it to
`be compiled easily for a wide range of computer hardware, without its being so
`complex or restrictive that systems programmers had to revert to assembly lan(cid:173)
`guage to get reasonable efficiency or functionality. Access to the hardware was
`provided through assembly-language stubs for the 3 percent of the operating(cid:173)
`system functions-such as context switching-that needed them. Although the
`success of UNIX does not stem solely from its being written in a high-level lan(cid:173)
`guage, the use of C was a critical first step [Kernighan & Ritchie, 1978; Kernighan
`& Ritchie, 1989; Ritchie et al., 1978]. Ritchie's C language is descended [Rosler,
`1984] from Thompson's B language, which was itself descended from BCPL
`[Richards & Whitby-Strevens, 1980]. C continues to evolve [Tuthill, 1985; ISO,
`1999].
`
`Page 16 of 43
`
`

`

`Section 1.1
`
`History of the UNIX System
`
`The second important distinction of UNIX was its early releasf' from Bell
`Laboratories to other research environments in source form. By providing source,
`the system's founders ensured that other organizations would be able not only to
`use the system, but also to tinker with its inner workings. The ease with which
`new ideas could be adopted into the system always has been key to the changes
`that have been made to it. Whenever a new system that tried to upstage UNIX
`came along, somebody would dissect the newcomer and clone its central ideas
`into UNIX. The unique ability to use a small, comprehensible system, written in a
`high-level language, in an environment swimming in new ideas led to a UNIX sys(cid:173)
`tem that evolved far beyond its humble beginnings. Though recipients of the
`source code had to be licensed, campuswide licenses were cheaply available to
`universities. Thus, many people became versed in the way that UNIX worked, set(cid:173)
`ting the stage for the open-source world that would follow.
`The third important distinction of UNIX was that it provided individual users
`with the ability to run multiple processes concurrently and to connect these pro(cid:173)
`cesses into pipelines of commands. At the time, only operating systems running
`on large and expensive machines had the ability to run multiple processes, and the
`number of concurrent processes usually was controlled tightly by a system admin(cid:173)
`istrator.
`Most early UNIX systems ran on the PDP-11, which was inexpensive and
`powerful for its time. Nonetheless, there was at least one early port of Sixth
`Edition UNIX to a machine with a different architecture: the Interdata 7 /32
`[Miller, 1978]. The PDP-11 also had an inconveniently small address space. The
`introduction of machines with 32-bit address spaces, especially the VAX-111780,
`provided an opportunity for UNIX to expand its services to include virtual memory
`and networking. Earlier experiments by the Research group in providing UNIX(cid:173)
`like facilities on different hardware had led to the conclusion that it was as easy to
`move the entire operating system as it was to duplicate UNIX's services under
`another operating system. The first UNIX system with portability as a specific
`goal was UNIX Time-Sharing System, Seventh Edition (V7), which ran on the
`PDP-11 and the Interdata 8/32 and had a VAX variety called UNIX/32V Time(cid:173)
`Sharing, System Version 1.0 (32V). The Research group at Bell Laboratories has
`also developed UNIX Time-Sharing System, Eighth Edition (V8); UNIX Time(cid:173)
`Sharing System, Ninth Edition (V9); and UNIX Time-Sharing System, Tenth
`Edition (VlO). Their 1996 system is Plan 9.
`
`AT&T UNIX System III and System V
`
`After the distribution of Seventh Edition in 1978, the Research group turned over
`external distributions to the UNIX Support Group (USG). USG had previously di -
`tributed internally such systems as the UNIX Programmer's Work Bench (PWB),
`and had sometimes distributed them externally as well [Mohr, 1985].
`USG's first external distribution after Seventh Edition was UNIX System ill
`(System ill) in 1982, which incorporated feature of Seventh Edition, of 32V, and
`also of several UNIX systems developed by groups other than the Research group.
`Features of UNIX/RT (a real-time UNIX system) were included, as were many
`features from PWB. USG released UNIX System V (System V) in 1983; that
`
`Page 17 of 43
`
`

`

`Chapter 1
`
`History and Goals
`
`system is largely derived from System ill. The court-ordered divestiture of the
`Bell Operating Companies from AT&T permitted AT&T to market System V
`aggressively [Bach, 1986; Wilson, 1985].
`USG metamorphosed
`into
`the UNIX System Development Laboratory
`(USDL), which released UNIX System V, Release 2
`in 1984. System V,
`Release 2, Version 4 introduced paging [Jung, 1985; Miller, 1984], including
`copy-on-write and shared memory, to System V. The System V implementation
`was not based on the Berkeley paging system. USDL was succeeded by AT&T
`Information Systems (ATIIS), which distributed UNIX System V, Release 3, in
`1987. That system included STREAMS, an IPC mechanism adopted from V8
`[Presotto & Ritchie, 1985]. ATIIS was succeeded by UNIX System Laboratory
`(USL), which was sold to Novell in 1993. Novell passed the UNIX trademark to
`the X/OPEN consortium, giving the latter sole rights to set up certification stan(cid:173)
`dards for using the UNIX name on products. Two years later, Novell sold UNIX
`to The Santa Cruz Operation (SCO).
`
`Berkeley Software Distributions
`
`The most influential of the non-Bell Laboratories and non-AT&T UNIX develop(cid:173)
`ment groups was the University of California at Berkeley [DiBona et al., 1999].
`Software from Berkeley was released
`in Berkeley Software Distributions
`(BSD)-for example, as 4.4BSD. Berkeley was the source of the BSD name, and
`their distributions were the first distinct identity for the BSD operating system.
`The first Berkeley VAX UNIX work was the addition to 32V of virtual memory,
`demand paging, and page replacement in 1979 by William Joy and Ozalp
`Babaoglu, to produce 3BSD [Babaoglu & Joy, 1981]. The reason for the large vir(cid:173)
`tual-memory space of 3BSD was the development of what at the time were large
`programs, such as Berkeley's Franz LISP. This memory-management work con(cid:173)
`vinced the Defense Advanced Research Projects Agency (DARPA) to fund the
`Berkeley team for the later development of a standard system (4BSD) for DARPA's
`contractors to use.
`A goal of the 4BSD project was to provide support for the DARPA Internet
`networking protocols, TCP/IP
`[Comer, 2000]. The networking implementation
`was general enough to communicate among diverse network facilities, ranging
`from local networks, such as Ethernets and token rings, to long-haul networks,
`such as DARPA's ARPANET.
`We refer to all the Berkeley VAX UNIX systems following 3BSD as 4BSD,
`although there were really several releases: 4.0BSD, 4. lBSD, 4.2BSD, 4.3BSD,
`4.3BSD Tahoe, and 4.3BSD Reno. 4BSD was the UNIX operating system of choice
`for VAXes from the time that the VAX first became available in 1977 until the
`release of System V in 1983. Most organizations would purchase a 32V license
`but would order 4BSD from Berkeley. Many installations inside the Bell System
`ran 4.lBSD (and replaced it with 4.3BSD when the latter became available). A
`new virtual-memory system was released with 4.4BSD. The VAX was reaching
`the end of its useful lifetime, so 4.4BSD was not ported to that machine. Instead,
`4.4BSD ran on the newer 68000, SPARC, MIPS, and Intel PC architectures.
`
`Page 18 of 43
`
`

`

`Section 1.2
`
`BSD and Other Systems
`
`The 4BSD work for DARPA was guided by a steering committee that included
`many notable people from both commercial and academic institutions. The culmi(cid:173)
`nation of the original Berkeley DARPA UNIX project was the release of 4.2BSD in
`1983; further research at Berkeley produced 4.3BSD in mid-1986. The next
`releases included the 4.3BSD Tahoe release of June 1988 and the 4.3BSD Reno
`release of June 1990. These releases were primarily ports to the Computer
`Consoles Incorporated hardware platform. Interleaved with these releases were
`two unencumbered networking releases: the 4.3BSD Netl release of March 1989
`and the 4.3BSD Net2 release of June 1991. These releases extracted nonpropri(cid:173)
`etary code from 4.3BSD; they could be redistributed freely in source and binary
`form to companies that and individuals who were not covered by a UNIX source
`license. The final CSRG release requiring an AT&T source license was 4.4BSD in
`June 1993. Following a year oflitigation (see Section 1.3), the free-redistributable
`4.4BSD-Lite was released in April 1994. The final CSRG release was 4.4BSD-Lite
`Release 2 in June 1995.
`
`UNIX in the World
`
`The UNIX system is also a fertile field for academic endeavor. Thompson and
`Ritchie were given the Association for Computing Machinery Turing award for
`the design of the system [Ritchie, 1984b]. The UNIX system and related, specially
`designed teaching systems-such as Tunis [Ewens et al., 1985; Holt, 1983], XINU
`[Comer, 1984), and MINIX [Tanenbaum, 1987)-are widely used in courses on
`operating systems. Linus Torvalds reimplemented the UNIX interface in his freely
`redistributable Linux operating system. The UNIX system is ubiquitous in univer(cid:173)
`sities and research facilities throughout the world, and is ever more widely used in
`industry and commerce.
`
`1.2 BSD _and Other Systems
`
`The CSRG incorporated features from not only UNIX systems but from other oper(cid:173)
`ating systems. Many of the features of the 4BSD terminal drivers are from
`TENEX!fOPS-20. Job control (in concept-not in implementation) is derived from
`TOPS-20 and from the MIT Incompatible Timesharing System (ITS). The virtual(cid:173)
`memory interface first proposed for 4.2BSD, and finally implemented in 4.4BSD,
`was based on the file-mapping and page-level interfaces that first appeared in
`TENExtrOPS-20. The current FreeBSD virtual-memory system (see Chapter 5)
`was adapted from Mach, which was itself an offshoot of 4.3BSD. Multics has
`often been a reference point in the design of new facilities.
`The quest for efficiency was a major factor in much of the CSRG's work.
`Some efficiency improvements were made because of comparisons with the pro(cid:173)
`prietary operating system for the VAX, VMS [Joy, 1980; Kashtan, 1980).
`Other UNIX variants have adopted many 4BSD features. AT&T UNIX System
`V [AT&T, 1987), the IEEE POSIX.l standard [Pl003.l, 1988), and the related
`
`Page 19 of 43
`
`

`

`National Bureau of Standards (NBS) Federal Information Processing Standard
`(FIPS) have adopted the following.
`
`Chapter I
`
`History and Goals
`
`• Job control (Chapter 2)
`
`•Reliable signals (Chapter 4)
`
`• Multiple file-access permission groups (Chapter 6)
`
`• Filesystem interfaces (Chapter 8)
`
`The X/OPEN Group (originally con isting of only European vendors but now
`including most U.S. UNIX vendors) produced the XIOPEN Portability Guide
`[X/OPEN, 1987] and, more recently, the Spec 1170 Guide. These documents
`any of the utility programs available to
`specify both. the kernel interface and
`UNIX system users. When Novell purchased UNIX from AT&T in 1993, it trans(cid:173)
`ferred exclusive ownership of the UNIX name to X/OPEN. Thus, all systems that
`want to brand themselves as UNIX must meet the X/OPEN inte1face specifica(cid:173)
`tions. To date, no BSD system has ever been put through the X/OPEN interface(cid:173)
`specification tests, so none of them can be called UNIX. The X/OPEN guides
`have adopted many of the POSIX facilities. The POSIX. l standard is also an ISO
`International Standard, named SC22 WG15. Thus, the POSIX facilities have been
`accepted in most UNIX-like systems worldwide.
`The 4BSD socket interprocess-communication mechanism (see Chapter 11)
`was designed for portability and was immediately ported to AT&T System ill,
`although it was never distributed with that system. The 4BSD implementation of
`the TCP/IP networking protocol suite (see Chapter 13) is widely used as the basis
`for further implementations on systems ranging from AT&T 3B machines running
`System V to VMS to embedded operating systems such as VxWorks.
`The CSRG cooperated closely with vendors whose systems are based on
`4.2BSD and 4.3BSD. This simultaneous development contributed to the ease of
`further ports of 4.3BSD and to ongoing development of the system.
`
`The Influence of the User Community
`
`Much of the Berkeley development work was done in response to the user com(cid:173)
`munity.
`Ideas and expectations came not only from DARPA, the principal
`direct-funding organization, but also from users of the system at companies and
`universities worldwide.
`The Berkeley researchers accepted not only ideas from the user community
`but also actual software. Contributions to 4BSD came from universities and
`other organizations in Australia, Canada, Europe, Japan, and the United States.
`These contributions included major features, such as autoconfiguration and disk
`quotas. A few ideas, such as the fcntl system call, were taken from System V,
`although licensing and pricing considerations prevented the use of any code from
`System ill or System V in 4BSD. In addition to contributions that were included
`in the distributions proper, the CSRG also distributed a set of user-contributed
`software.
`
`Page 20 of 43
`
`

`

`'
`
`-
`
`_I _ _
`
`_
`
`_
`
`Section 1.3
`
`The Transition of BSD to Open Source
`
`An example of a community-developed facility is the public-domain time(cid:173)
`zone-handling package that was adopted with the 4.3BSD Tahoe release. It was
`designed and implemented by an international group, including Arthur Olson,
`Robert Elz, and Guy Harris, partly because of discussions in the USENET news(cid:173)
`group comp.std.unix. This package takes time-zone-conversion rules completely
`out of the C library, putting them in files that require no system-code changes to
`change time-zone rules; this change is especially useful with binary-only distribu(cid:173)
`tions of UNIX. The method also allows individual processes to choose rules rather
`than keeping one ruleset specification systemwide. The distribution includes a
`large database of rules used in many areas throughout the world, from China to
`Australia to Europe. Distributions are thus simplified because it is not necessary
`to have the software set up differently for different destinations, as long as the
`whole database is included. The adoption of the time-zone package into BSD
`brought the technology to the attention of commercial vendors, such as Sun
`Microsystems, causing them to incorporate it into their systems.
`
`1.3 The Transition of BSD to Open Source
`
`Up through the release of 4.3BSD Tahoe, all recipients of BSD had to first get an
`AT&T source license. That was because the BSD systems were never released by
`Berkeley in a binary-only format; the distributions always contained the complete
`source to every part of the system. The history of the UNIX system, and the BSD
`system in particular, had shown the power of making the source available to the
`users. Instead of passively using the system, they actively worked to fix bugs,
`improve performance and functionabty, and even add completely new features.
`With the increasing cost of the AT&T source licenses, vendors that wanted to
`build stand-alone TCP/IP-based networking products for the PC market using the
`BSD code found the per-binary costs prohibitive. So they requested that Berkeley
`break out the networking code and utilities and provide them under licensing
`terms that did not require an AT&T source license. The TCP/IP networking code
`clearly did not exist in 32N and thus had been developed entirely by Berkeley and
`its contributors. The BSD-originated networking , code and supporting utilities
`were released in June 1989 as Networking Release 1, the first freely redis(cid:173)
`tributable code from Berkeley.
`The licensing terms were liberal. A licensee could release the code modified or
`unmodified in source or binary form with no accounting or royalties to Berkeley.
`The only requirements were that the copyright notices in the source file be left
`intact and that products that incorporated the code include in their documentation
`that the product contained code from the University of California and its contribu(cid:173)
`tors. Although Berkeley charged a $1000 fee to get a tape, anyone was free to get
`a copy from somebody who already had it. Indeed, several large sites put it up for
`anonymous FTP shortly after it was released. Though the code was freely avail(cid:173)
`able, several hundred organizations purchased tapes, which helped to fund the
`CSRG and encouraged further development.
`
`Page 21 of 43
`
`

`

`10
`
`Networking Release 2
`
`Chapter 1
`
`History and Goals
`
`With the success of the first open-source release, the CSRG decided to see how
`much more of BSD they could spring free. Keith Bostic led the charge by solicit(cid:173)
`ing people to rewrite the UNIX utilities from scratch based solely on their pub(cid:173)
`lished descriptions. Their only compensation would be to have their name listed
`among the Berkeley contributors next to the name of the utility that they rewrote.
`The contributions started slowly and were mostly for the trivial utilities. But as
`the list of completed utilities grew, and Bostic continued to hold forth for contribu(cid:173)
`tions at public events such as Usenix, the rate of cont