`USOOSS79476A
`
`United States Patent
`
`[19]
`
`[11] Patent Number:
`
`5,579,476
`
`[45] Date of Patent: Nov. 26, 1996
`Cheng et al.
`
`
`
`[54] AUTOMATIC TEST ENVIRONMENT FOR
`COMMUNICATIONS PROTOCOL
`SOFTWARE
`
`Beizer, Software System Testing and Quality Assurance,
`1984, pp. 91—140.
`
`[75]
`
`Inventors: Hsu-Feng Cheng, Hsinchu; Jung-Ming
`Fang, Taipei Hsizn, both of Taiwan
`
`Primary Examiner—Robert W. Beausoliel, Jr.
`Assistant Examiner—Norman M. Wright
`Attorney, Agent, or Finn—Meltzer, Lippe Goldstein et a1.
`
`[73] Assignee:
`
`Industrial Technology Research
`Institute, Hsinchu, Taiwan
`
`[57]
`
`ABSTRACT
`
`[21]
`
`[22]
`
`[63]
`
`[51]
`[52]
`[58]
`
`[56]
`
`Appl. No.: 501,456
`
`Filed:
`
`Jul. 12, 1995
`
`Related U.S. Application Data
`
`Continuation of Ser. No. 139,303, Oct. 19, 1993, abandoned.
`
`Int. Cl.6 ...................................................... G06F 11/00
`U.S. Cl.
`...............
`.. 395/183.08; 395/183.15
`
`Field of Search
`............. 395/18308, 183.01,
`395/185.02, 183.1, 183.11,183.14, 183.15,
`183.13
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`5,233,611
`
`8/1993 Triantafyllos et a1.
`OTHER PUBLICATIONS
`
`371/161
`
`Automator Developer Guide with Examples, Jul. 1991,
`Version QANDG 1.0 pp. 1—4 to 1-8; l—12 to l—15;4—21 to
`4—24; and 5—24 to 5—34.
`Automator Programmer Reference Manual, Jul. 1991, Ver-
`sion QANPR 1.0 pp. 1—7; 3—47; 4-24 to 4—25.
`Automator User Guide, Jul. 1001, Version QANUG 1.0 pp.
`2—2; 3—8 to 3—9; and 5—1 to 5—6.
`
`A test system and method are disclosed for testing a com—
`munications protocol software implementation or combina—
`tion of communications protocol software implementations.
`The test system has a memory for storing a communications
`protocol software implementation of a particular layer in a
`hierarchy of protocols. The test system has a processor for
`catching data outputted from the communications protocol
`software implementation to upper and lower layer commu-
`nications protocol software. The processor is also for inter-
`preting each frame contained in a test case file. In response
`to interpreting an input data frame, the processor feeds input
`data contained in the input data frame to the communica-
`tions protocol software implementation. In response to inter-
`preting an output data frame,
`the processor compares
`received data outputted from the communications protocol
`software implementation with expected output data inter-
`preted from the output data frame, and generates a message
`depending on this comparison. Furthermore, the processor
`may simultaneously monitor the execution of the commu—
`nications protocol software implementation and generate a
`wakeeup message if the communications protocol software
`implementation properly or unexpectedly ceases execution.
`In response to receiving a wake-up message, the processor
`writes an appropriate message in a test report file.
`
`15 Claims, 5 Drawing Sheets
`
`[NOHHATIY
`COMPLETED]
`HAKEIPEH
`
`MINE
`TEST CAFE
`FILE?
`
`
`HAIT FOR OUTPUT
`(CATCHERI
`
`
`
`SEMI
`'HAKEIP'
`MESSAGE
`
`AUTfl-VAHIABLE TD TESTETT
`
`SOTO
`SPECIFIED
`FRME N0.
`
`SET VALVE [1F
`
`CONFIGIT 1022
`
`CONFIGIT 1022
`
`1
`
`
`
`US. Patent
`
`Nov. 26, 1996
`
`Sheet 1 of 5
`
`5,579,476
`
`22222352;:
`0:\
`
`add
`
`.myHum
`
`$5”:$33
`
`dmfl
`
`EBBEma: E;ESE“H.
`
`2
`
`
`
`US. Patent
`
`Nov. 26, 1996
`
`Sheet 2 of 5
`
`5,579,476
`
`omm
`
`
`
`mm><4mmzoq
`
`monH<quzzzoo
`
`Jouopoma
`
`mm<zhmom
`
`mmm
`
`
`
`¢m><4mmmmz
`
`monH<Uszzzou
`
`Acuopomm
`
`mm<zhmom
`
`6mm
`
`____.__________—_.______—______...__._._..._____—_________—___—.
`
`mam
`
`
`
`mamImzamH
`
`ovm
`
`EQHEQHm.mHk
`AH¢<
`
`ammHmMH
`
`[om
`
`maHazoo
`
`3
`
`
`
`
`
`
`
`U.S. Patent
`
`Nov. 26, 1996
`
`Sheet 3 of 5
`
`_
`
`5,579,476
`
`330
`
`I
`
`FIG. 3
`(PRIOR ART)
`
`340
`
`""""""""""""" '1
`
`I“
`'
`A
`.
`I
`
`!
`l
`:
`I
`
`I— """""""""
`
`TEST RESPONDEH
`(UPPER TESTER)
`3m
`
`EXCEPTION
`
`HANDLER
`
`!
`A
`LowER
`
`I
`I
`TESTER
`
`L _______________________ __I
`L _________________ .1
`
`’1
`I
`,
`.
`!
`
`!
`l
`:
`!
`
`FIG. 4
`
`I" """""""""""""""""""" ‘1
`
`
`
`DISPLAY
`
`.
`
`MANUAL
`INPUT
`DEVICE
`
`DEVICE
`3Q
`
`
`fl
`
`
`
`
`IQ
`
`l_ _____________________________ __I
`
`4
`
`
`
`US. Patent
`
`Nov. 26, 1996
`
`Sheet 4 of 5
`
`5,579,476
`
`550
`
`540
`
`£5
`
`I‘UIE?.
`
`570
`
`' FEEDER
`______________________________
`
`_-_—o—_--._~—..__.._.n-—-___——--——.-
`
`
` PSEUDD
`
`PSEUDOLOWER
`UPPER
`
`
`
`TESTER
`
`500-——""
`
`WAKEUPER
`
`520
`
`5
`
`
`
`US. Patent
`
`Nov. 26, 1996
`
`Sheet 5 of 5
`
`_
`
`5,579,476
`
`MAKEUPER
`
`SEND
`'MAKEUP'
`MESSAGE
`TO TESTER
`
`;
`
`; !
`
`FKIE;.
`
`£3
`
`TESTER
`
`(NORMALLY
`COMPLETED)
`
`MAKEUPER
`
`MORE
`TEST CASE
`FILE?
`
`604
`
`MORE
`FRAME?
`
`MAIT FOR OUTPUT
`(CATCHER)
`
`.
`
`.
`
`540/550
`
`UPPER/LOWER
`PSEUDO
`
`GOTU
`SPECIFIED
`FRAME NO.
`
`TO BE
`INPUTTED?
`
`wATT FOR OUTPUT
`(CATCHER)
`
`SET VALVE OF
`AUTO-VARIABLE
`
`WAKEUPER
`
`FEED INTO IUT
`BY FEEDER
`
`MATCHED
`MITH
`EXPECTED?
`
`
`
`REPORT
`ERROR
`
`6
`
`
`
`1
`AUTOMATIC TEST ENVIRONIVIENT FOR
`COMMUNICATIONS PROTOCOL
`SOFTWARE
`
`This is a continuation of application Ser. No. 08/139,303,
`filed Oct. 19, 1993, now abandoned, for AUTOMATIC
`TEST ENVIRONMENT FOR COMMUNICATIONS PRO-
`TOCOL SOFTWARE.
`
`FIELD OF THE INVENTION
`
`The present invention is directed to a system for testing
`communications software. In particular, the present inven-
`tion provides a test system for feeding input test data to, and
`receiving output data from, a communications protocol
`software implementation or combination of communications
`protocol software implementations.
`
`BACKGROUND OF THE INVENTION
`
`A communications system typically includes two devices
`which can communicate with each other either directly or
`via one or more intermediary communications devices of the
`communications system. Some examples of communica-
`tions devices include: private branch exchanges, data termi-
`nal equipments, central office switches, ISDN customer
`premises equipment, etc. These devices communicate with
`one another according to one or more communications
`protocols or sets of semantic and syntactic rules for com-
`municating. Some examples of communications protocols
`include: LAPD (Q.921), Q.931, X.25PLP, LAPB, T70,
`T62, DTAM, and V.120.
`
`Illustratively, devices of the communications system con—
`tain protocol software for achieving communication accord-
`ing to the different communications protocols. Such protocol
`software may be hierarchically organized into levels called
`layers according to the Open Systems Interconnection (081)
`model. According to 081, communications protocol soft—
`ware for communicating on the highest layer may be used
`for requesting high level or abstract communications goals.
`Communications protocol software of each lower layer may
`be used for requesting more specific sub-goals of higher
`layers. Finally, communications protocol software of the
`lowest layer may be used for requesting low level or actual
`physical tasks (i.e., the actual transmission or reception of
`data signals). Typically, a communicating dcvice communi—
`cates by first generating a request to perform some relatively
`abstract goal using high layer communications protocol
`software. This in turn may cause the device to generate one
`or more requests to perform more specific sub-goals of the
`abstract goal using lower layer communications protocol
`software, etc., until one or more requests are generated using
`communications protocol software of the lowest layer.
`US. Pat. Nos. 3,869,603, 4,055,801, 4,108,358, 4,713,
`815, 4,759,019, 4,821,266, 4,916,641, and 5,111,402 dis-
`close automated test systems for testing a variety of types of
`hardware. Advantageously, implementations of communica-
`tions protocol software for operating devices of a commu-
`nications system are tested in a simulated environment
`before they are deployed in the equipment of the commu-
`nications system. To that end, the communications protocol
`software implementation under test (IUT) is tested by emu—
`lating the data and control message inputs which may be
`received from higher and lower layer communications pro-
`tocol software. If in response to these inputs, the IUT outputs
`the correct data and control messages (as per the protocol of
`the higher or lower layer), the IUT passes the test.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`4o
`
`45
`
`50
`
`55
`
`60
`
`65
`
`5,579,476
`
`2
`
`Traditionally, IUT’s have been tested in a test environ-
`ment 100 such as depicted in FIG. 1. An upper tester 110 is
`provided for simulating communications protocol software
`of a higher layer than an IUT 120 from which the IUT 120
`may receive messages, or to which the IUT 120 may
`transmit messages. For example, the IUT 120 may be an
`X.25PLP software implementation (layer 3 communications
`protocol software) and the upper tester may contain a T.70
`software implementation (layer 4 communications protocol
`software). The upper tester 110 transmits to, and receives
`from, the IUT 120 different kinds of simulated inputted data
`and control messages according to a higher layer protocol.
`Likewise, a lower tester 130 is provided for simulating
`communications protocol software on a lower layer than the
`IUT 120 to which the IUT 120 may transmit messages, or
`from which the IUT 120 may receive messages. In the above
`example, the lower tester 130 may contain a LAPB software
`implementation (layer 2 communications protocol
`soft—
`ware). The lower tester 130 transmits different kinds of
`simulated input data and control messages according to a
`lower layer protocol.
`The simulated input data and control messages of the
`upper and lower testers may be obtained automatically from
`a test script file 111 or 131, respectively. Alternatively, the
`input data and control messages may be inputted by a user
`in an interactive session. In response to receiving these
`inputted messages,
`the IUT may output messages to the
`upper tester 110 and/or the lower tester 130. As shown by the
`line 140, the test environment 100 must provide intricate
`synchronization procedures so that the upper tester 110 and
`lower tester 130 simulate, as close as possible, the actual
`environment in which the IUT 120 is to be deployed.
`FIG. 2 shows a first conventional communications pro-
`tocol software test system 200 which provides a test envi-
`ronment according to the general environment 100 (FIG. 1).
`The test system 200 has a testbed 210 which illustratively
`may be a computer such as a Micro VaxTM, manufactured by
`Digital Equipment Corporation. The IUT 220 is maintained
`in a separate device 230, e.g., the actual hardware in which
`the IUT 220 is to be deployed. The testbed 210 communi-
`cates with the device 230 via an ISDN S-interface bus 212.
`Furthermore, a user 240 may input data and control mes-
`sages to, and receive data and control messages from the
`device 230. The device 230 in which the IUT 220 resides has
`
`lower layer communication protocol software 251 and upper
`layer communication protocol software 252 which serve to
`temporarily store information transmitted to the IUT 220 or
`received from the user 240 0r testbed 210.
`
`In operation, the testbed 210 serves as an automatic lower
`tester. The testbed 210 automatically feeds simulation input
`data and control message data from a test case file 260 to the
`IUT 220. The user 240 serves as an upper tester. The user
`manually inputs data and control messages to, and receives
`data and control messages from, the IUT 220 during an
`interactive session. For example, if the testbed 210 is a
`telephone, the user 240 might lift the handset from the cradle
`(creating an olI-hook signal) and press keys on the telephone
`keypad. The user receives data from the lower tester 210 in
`the form of a dial tone, etc.
`FIG. 3 depicts a second conventional test system 300
`according to the conventional architecture 100 (FIG. 1). An
`upper tester 350 is provided which is also called the test
`responder. Both the test responder 350 and the IUT 320 are
`stored in the device 340 which can be a Sun Microsystems’
`SUN 3““, SUN 4““, or SPARCTM computers.
`The test system 300 has lower tester 330 in the form of a
`device such as a Sun Microsystems’ SUN 3““, SUN 4““, or
`
`7
`
`
`
`5,579,476
`
`3
`SPARCTM computers. The lower tester device 330 is a
`separate device from the device 340. The two devices 330
`and 340 can communicate with each other, e.g., via a Sun
`Microsystems, SUNLINK OSITM interface 355.
`The lower tester 330 contains a test driver program 310,
`a referencer 360 and an exception handler 370. The refer-
`encer 360 is a certified communications protocol software
`implementation of the same communications protocol(s) as
`the IUT 320. The referencer 360 may thus be used as a
`reference as to how the IUT 320 should respond.
`The exception handler 370 operates under the control of
`the test driver program 310. The exception handler 370 is
`provided for purposes of altering the messages communi—
`cated between the referencer 360 and the SUNLINK OSITM
`
`interface 355. The exception handler 370 can thus create an
`abnormality in the messages outputted from the referencer
`360,
`
`In operation, the user compiles a test case file which
`includes the data to be inputted to, and outputted from, the
`upper tester 350 and the lower tester 330. The test driver 310
`then transfers instructions, using Test Driver Responder
`Protocol (TDRP) language, regarding the operation of the
`upper tester 350 to the test responder 350 via the referencer
`360. SUNLINK OSITM interface 355 and the IUT 320. The
`device 340 and the device 330 then execute appropriate
`steps to automatically feed input messages to the IUT 320
`and to receive output messages therefrom.
`The prior art test systems 200 (FIG. 2) and 300 (FIG. 3)
`have several disadvantages. In the test system 200, the user
`is required to enter appropriate input data in an interactive
`session. Such an interactive session is tedious and prone to
`errors. Moreover, because of the limited speed of human
`data entry, the test requires a long time to complete. The time
`factor is important when considering that the same test may
`be applied to several difierent IUT’ s or reapplied to the same
`IUT in the course of developing the IUT.
`In using the test system 300, the IUT is compiled together
`with a test script file which contains the inputted messages
`of the upper and lower tester. The tests carried out by each
`test script file may be planned and separately stored ahead of
`time. These test script files may then be compiled with the
`IUT to produce an executable object (machine language)
`file. However, any time a test or the IUT is modified, the files
`must be recompiled. Furthermore, an object file (containing
`the executable machine language code) is produced each
`time the IUT is compiled with a test script file. Thus, a great
`deal of effort must be expended in managing the files, for
`example, to ensure that only current object files are stored
`and only unneeded object files are discarded.
`In both test systems 200 and 300, the data outputted from
`the IUT is transmitted through the upper and lower testers.
`The upper and lower testers may have certain error conten-
`tion capabilities in which the upper and lower testers con-
`tinue to perform normal operations even in the event erro—
`neous data and control messages are outputted from the IUT.
`Thus, the upper and lower testers may filter out erroneous
`data and control messages outputted from the IUT. This
`reduces the ability to fully test the IUT.
`It is thus an object of the present invention to overcome
`the disadvantages of the prior art.
`
`SUMMARY OF THE INVENTION
`
`The present invention achieves these objects by providing
`an automatic interpretive test system with a single tester, i.e.,
`the lower and upper tester are combined into a single tester.
`
`IO
`
`15
`
`20
`
`25
`
`30
`
`35
`
`4o
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`Furthermore, the test system automatically feeds input data
`simulating both the upper and lower tester, and checks
`output data received from the IUT, by sequentially inter—
`preting instructions from a test case file. The test case file
`may be generated and stored prior to performing the test.
`According to one embodiment, a test system is provided
`with a processor and a memory. Prior to conducting the test,
`the IUT is stored in the memory. Furthermore, a test case file
`is prepared containing a sequence of instructions called
`frames. The frames may be input data frames which contain
`data to be inputted to the IUT or output data frames which
`contain expected data to be outputted from the IUT. Illus»
`tratively, the input and output data frames are sequentially
`ordered in the test case file to simulate a test session of
`requests and responses between the IUT and upper and
`lower layer communications protocol software. Optionally,
`the test system may be provided with a disk memory, such
`as a hard disk drive, for storing test case files. Thus, test files
`can be planned and stored prior to performing the tests.
`During a test, the processor executes a process called the
`tester. The tester process has four sub-processes: a catcher
`subprocess, an interpreter subprocess, a feeder subprocess
`and a comparator subprocess. The catcher subprocess is for
`receiving data outputted from the IUT to upper and lower
`layer communications protocol software. The interpreter
`subprocess is for interpreting each frame of the sequence of
`frames in the test case file. In response to interpreting an
`input data frame, the processor executes the feeder subpro—
`cess to feed input data contained in the input data frame
`directly to the IUT. In response to interpreting an output data
`frame, the processor executes the comparator subprocess to
`compare output data contained in the output data frame with
`corresponding output data received by the catcher subpro—
`cess. In response to the comparison, the processor illustra-
`tively generates messages indicating whether or not the data
`actually outputted from the IUT matched the expected data
`contained in the output data frames. Illustratively these
`generated messages are stored in a test report file.
`Illustratively,
`the processor also contemporaneously
`executes a second process called the wake—upper. The wake»
`upper monitors the IUT to determine if the IUT properly
`completes its execution or if it unexpectedly stops. The
`wake—upper process generates a wake-up message depend—
`ing on whether or not the IUT executes properly. Illustra—
`tively, this is achieved by assigning a priority to the wake—
`upper process which is lower than the tester and IUT
`processes. This ensures that
`the wake-upper does not
`execute unless both the tester process and IUT are idle.
`The wake-up message generated by the wake-upper pro-
`cess is received by the processor while executing the catcher
`subprocess and analyzed by the processor while executing
`the comparator subprocess. Depending on the particular
`wake-up message, the processor generates an error message
`or successful execution message. Illustratively, in response
`to receiving a wake-up message, the processor writes an
`appropriate message in the above-mentioned test report file.
`In short, a flexible test system is provided for testing
`implementations of communications protocol software or
`communications software incorporating a combination of
`protocols. The tester system according to the present inven»
`tion provides the following advantages:
`(1) tedious manual data entry is avoided
`(2) human error is avoided
`(3) tests require less time to complete
`(4) test cases may be predetermined and stored prior to
`performing the test
`
`8
`
`
`
`5,579,476
`
`5
`(5) because the test system is an interpreter, the test cases
`need not be re-compiled with the IUT each time the IUT
`or test case is modified
`(6) file maintenance is minimized
`(7) because the test system interacts directly with the IUT
`rather than via upper or lower layer communication
`protocol software, no erroneous results produced by an
`IUT with a bug are filtered out, and
`(8) because a single tester is provided which sequentially
`interprets frames from a test case file, intricate synchro-
`nization between upper and lower testers is not needed.
`
`BRIEF DESCRIPTION OF THE DRAWING
`
`FIG. 1 depicts a conventional communications protocol
`software test system architecture.
`FIG. 2 depicts a functional diagram of a first conventional
`communications protocol software test system.
`FIG. 3 depicts a functional diagram of a seCOnd conven—
`tional communications protocol software test system.
`FIG. 4 depicts a communications protocol software test
`system according to one embodiment of the present inven—
`tion.
`
`FIG. 5 depicts a functional diagram of the test system of
`FIG 4.
`
`FIG. 6 is a flowchart which schematically illustrates the
`operation of the test system depicted in FIGS. 4—5.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`FIG. 4 depicts a test system 10 according to the present
`invention. Illustratively, the test system 10 is a multi—tasking
`computer system. The test system 10 has one or more
`processors 12 or CPU’s for executing instructions of pro-
`cesses. The test system 10 also has a main memory 14 and
`a disk memory 16 such as a hard disk drive. Typically, a
`process executed by the processor 12 is stored, at least in
`part, in the main memory 14. The disk memory 16 is used
`for storing files. In addition, the test system 10 has a manual
`input device 18, such as a keyboard, and a display device 20,
`such as a CRT or LCD display. A data bus 22 is also provided
`for transferring data between different devices of the test
`system 10.
`Illustratively, the processor 12 of the test system is a
`multi—tasking processor. In other words, the processor 12 is
`capable of contemporaneously (or simultaneously, if the test
`system 10 has more than one processor 12) executing more
`than one process. This is usually achieved by alternately
`executing one or more instructions of each non-idle process
`in a round robin fashion. Due to the speed of the processor
`12, a user monitoring the input and output of a process via
`the display device 18 has the impression that all processes
`are executing simultaneously and continually. Each contem-
`poraneously executing process may be assigned a priority
`which controls how often that process is executed by the
`processor 12 in relation to other processes. Processes with
`higher priorities are allotted more execution time in the
`processor 12 than processes with lower priorities.
`FIG. 5 shows a functional block diagram of a test system
`500 according to one embodiment of the present invention.
`As shown, the test system 500 executes a tester process 510
`and a wake—upper process 520. The tester process 510 can
`feed messages to, and receive messages from, the IUT 530
`via a pseudo lower tester 540 and a pseudo upper tester 550.
`Illustratively, the tester process 510 performs all aspects of
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`' 6
`the test. Thus, the pseudo upper tester 550 and pseudo lower
`tester 540 may be mere buflers.
`The tester process 510 includes four sub—processes: a
`feeder sub-process 512, a catcher sub-process 514, an inter-
`preter sub—process 516 and a comparator sub-process 518.
`The interpreter sub—process 516 sequentially reads and inter—
`prets each frame or instruction of the test case file. These
`frames are discussed in greater detail below. The feeder
`sub—process 512 inputs data received from the interpreter
`sub-process 516 to the IUT 530. The catcher sub—process
`514 receives data outputted from the IUT 530. The com-
`parator sub—process 518 compares expected output data in an
`output data frame interpretted by the interpreter sub-process
`516 with output data received by the catcher sub-process
`514.
`
`Appendix A depicts two illustrative test case files. A test
`case file may include a file header (e.g.,
`lines 1—8 or
`lll—l18), a summary (e.g., lines 10—29 or 120—133) and a
`sequence of one or more frames (e.g.,
`lines 31—110 or
`135—171). Each frame begins with a frame designator (e.g.,
`@_DX23). Frames may be input data frames, output data
`frames, SET frames or GOTO frames. Input and output data
`frames have the following syntax:
`
`[<frame r10.>] <eomment>
`HEADER h1 h2 ha h4 hs h6 h7 he 1'19 Ulesl Tx/Rx Ulevel Info
`{<no. data items>} <data items>
`
`to 65535
`The frame number is a unique label from 1
`which distinguishes the frame from other frames. Comments
`may be provided following the frame number. HEADER
`identifies the beginning of the header. The nine following
`bytes h1 h2 h3 h4 h5 h6 h7 h8 h9 are reserved for later
`expansion. A value Um, follows which value serves as the
`process identifier of the IUT. The next bit Tx/Rx indicates
`whether the data contained in the frame is input data
`(Tl/R50) to be fed into the IUT 530 or whether the data is
`expected output data (TJRX=1) to be received from the IUT
`530. The next bit Ulml is the process identifier of the pseudo
`level via which data contained in the frame is to be received
`
`or transmitted (UzeszO indicates pseudo upper level and
`Ulmrtl
`indicates pseudo lower level). A series of four
`“Info” digits follows indicating the kind of information
`represented by the data frame.
`The number in the braces indicates the number of data
`items to be inputted to, or expected to be outputted from, the
`IUT 530 which input data items or expected output data
`items follow the number in the braces. Data items can take
`the following form:
`(1) A 0-ff hexadecimal value representing one byte.
`(2) ? indicating a byte of unspecified value. The value of
`such a data item is irrelevant as this data item is to be
`ignored.
`(3) Uzz indicating an auto-variable. Illustratively, 256
`byte sized auto-variables are provided in the test system
`500. Each auto variable functions as an ordinary vari-
`able. Thus, the expression Uf8 returns the value stored
`in the auto variable Uf8. Operations may be performed
`on auto variables using the following syntax: Uzz opl
`zl op2 22 op3 z3, etc., where each operation op, can be
`“+”, “—”,“*”,“/”, “&” (logical AND), “I” (logical OR),
`“"” (logical XOR), ‘5” (right bit-shift), and “<” (left
`bit-shift).
`(4) 322 indicating that a value is stored in an auto—
`variable. This data item only occurs in an output data
`frame for causing data outputted from the IUT 530 to
`be stored in a particular auto-variable.
`
`9
`
`
`
`5,579,476
`
`7
`A SET frame is indicated by the word “SET” following
`the frame designator @WDX23. The SET frame is used for
`storing values in auto-variables. For example, in lines 36—37
`of Appendix A, 16 is stored in the auto-variable Ue8 and 40
`is stored in the auto-variable Uf8.
`A GOTO frame is indicated by the word “GOTO” fol—
`lowing the frame designator @_DX23. The GOTO frame
`causes the processor 12 (FIG. 4) to branch to the frame
`indicated by the label contained in the GOTO frame. For
`example, after interpreting the GOTO frame in line 171 of
`Appendix A, the processor 12 (FIG. 4) resumes interpreting
`with frame 4 on line 145. GOTO frames are useful for
`causing the processor 12 to repeatedly interpret a particular
`sub-sequence of frames of the test case file.
`FIG. 6 shows a flowchart illustrating the execution of the
`test system 500. Referring to FIGS. 4, 5, and 6, the operation
`of the test system 500 is now illustrated. Initially,
`the
`processor 12 loads the IUT 530 into the memory 14 from,
`e.g., the storage device (disk memory) 16 for execution. The
`processor 12 then executes the interpreter sub-process 516
`by proceeding to step 602. In step 602, the processor 12
`determines whether there are any test case files left to
`interpret. Illustratively, the test system 500 can interpret one
`or more sequences of frames in one or more test case files.
`If all test case files have been interpreted, the processor 12
`ceases execution. Otherwise,
`the processor executes step
`604 in which the processor 12 determines if there are any
`more frames left in the currently interpreted test case file. If
`not,
`the processor 12 switches to executing the catcher
`sub-process 514 by proceeding to step 606. If there are
`frames left, the processor 12 continues executing the inter—
`preter sub-process 516 by interpreting the next frame in step
`610.
`In step 610, the processor 12 determines whether the
`currently interpreted frame is an input/output data frame, a
`SET frame or a GOTO frame. In the event that the frame is
`a GOTO frame, the processor 12 executes step 612. In step
`612, the processor 12 causes interpretation to branch to the
`frame indicated by the label in the GOTO frame. That is,
`when the processor 12 executes a step requiring the inter-
`pretation of the next frame (step 610), the processor 12 will
`resume interpretation at the frame indicated by the label in
`the GOTO frame. The processor 12 then returns to step 604.
`In the event the frame is a SET frame, the processor 12
`executes step 614. Therein,
`the processor 12 stores an
`appropriate value in the auto-variable indicated by the SET
`frame. The processor then returns to step 604.
`In the event the frame is a data frame (i.e., input or
`output), the processor 12 proceeds to step 616. Therein, the
`processor 12 determines whether the data frame is an input
`data frame or an output data frame. In the event the data
`frame is an input data frame, the processor 12 executes the
`feeder sub-process 512 by proceeding to step 620. Therein,
`the processor 12 feeds the data contained in the input data
`frame directly to the IUT 530. The processor 12 then returns
`to step 604.
`In the event the data frame is an output data frame, the
`processor 12 first switches to executing the catcher sub-
`process 514 by proceeding to step 622. In step 622, the
`processor 12 waits until either output data is received from
`the IUT 530 via the pseudo upper tester 540 or pseudo lower
`tester 550 or until a wake-up message is received from the
`wake—upper process 520 (as discussed in greater detail
`below).
`When a wake-up message or output data is received, the
`processor executes the comparator sub-process 518 by pro-
`ceeding to step 624. Therein, the processor 12 first deter—
`
`10
`
`15
`
`20
`
`25
`
`3O
`
`35
`
`4O
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`mines whether a wake-up message or output data was
`received. If a wake-up message was received, the processor
`12 proceeds to step 626, wherein the processor 12 reports an
`error in a test report file and ceases execution. Otherwise, the
`processor 12 proceeds to step 628 in which the processor 12
`compares the actually received output data with the output
`data contained in the interpreted output data frame. If they
`are not the same, the processor proceeds to step 626, wherein
`the processor reports an error in the test report file and ceases
`execution. Otherwise, the processor 12 returns to step 604.
`In addition, the processor 12 may also illustratively indicate
`in the test report file that data was properly received if the
`received output data matches the expected output data. For
`example, see Appendix B which shows two sample test
`report files generated for the two test case files of Appendix
`A.
`
`the processor 12 waits to receive data
`In step 606,
`outputted from the IUT 530 via the pseudo upper or pseudo
`lower tester 540, 550 or a wake—up message from the
`wake-upper 520 (in a similar fashion to step 622). Upon
`receiving either output data or a wake-up message,
`the
`processor 12 switches to executing the comparator subi
`process 518 by proceeding to step 608. In step 608,
`the
`processor 12 determines whether a wake-up message or
`output data from the IUT 530 was received. If output data
`was received, then an error has occurred. Illustratively, all
`output data must be anticipated by an output data frame.
`Because, there are no more data frames to be interpreted
`(step 604) there are no output data frames anticipating
`output data. In such a case, the processor 12 proceeds to step
`626 and an error is written in the test report file.
`On the other hand, while executing the wake-upper pro—
`cess 520, the processor 12 outputs a message when the IUT
`530 completes its execution. If a wake-up message was
`received, then the wake-upper 520 has simply indicated that
`the IUT 530 properly completed the test. Thus, the processor
`returns to step 602 and the next test case file is interpreted.
`Contemporaneously while the processor 12 executes the
`tester process 530,
`the processor 12 also executes the
`wake—upper process 520. Illustratively, the wake-upper pro—
`cess 520 executes at a lower priority than the tester process.
`In such a case, the wake-upper process 520 does not execute
`unless and until the tester process 510 and IUT 530 are both
`idle.
`The sole purpose of the wake-upper process is to notify
`the tester process 510 if the IUT 530 properly ceases
`execution when expected or if the IUT 530 unexpectedly
`ceases execution. If either condition occurs, the processor 12
`(while executive the wake-upper process 520) generates an
`appropriate wake-up message. The wake—up message is
`received while the processor 12 executes the catcher sub—
`process 514 of the tester process 510.
`The test system 500 utilizes message passing as the only
`channel for communicating between different executing
`processes such as the IUT 530, the tester process 510 and the
`wake-upper process 520