US005579476A
`United States Patent
`5,579,476
`[11] Patent Number:
`
`[45] Date of Patent: Nov. 26, 1996
`Chengetal.
`
`UNO OA UY ATA
`
`[54] AUTOMATIC TEST ENVIRONMENT FOR
`COMMUNICATIONS PROTOCOL
`SOFTWARE
`
`Beizer, Software System Testing and Quality Assurance,
`1984, pp. 91-140.
`
`[75]
`
`Inventors: Hsu-Feng Cheng, Hsinchu; Jung-Ming
`Fang, Taipei Hsizn, both of Taiwan
`
`Primary Examiner—Robert W. Beausoliel, Jr.
`Assistant Examiner—Norman M. Wright
`Attorney, Agent, or Firm—Meltzer, Lippe Goldstein et al.
`
`[73] Assignee:
`
`Industrial Technology Research
`Institute, Hsinchu, Taiwan
`
`(57]
`
`ABSTRACT
`
`[21]
`
`[22]
`
`Appl. No.: 501,456
`
`Filed:
`
`Jul. 12, 1995
`
`Related U.S. Application Data
`
`[63]
`
`Continuation of Ser. No. 139,303, Oct. 19, 1993, abandoned.
`
`A test system and method are disclosed for testing a com-
`munications protocol software implementation or combina-
`tion of communications protocol software implementations.
`Thetest system has a memory for storing a communications
`protocol software implementation of a particular layer in a
`hierarchy of protocols. The test system has a processor for
`catching data outputted from the communications protocol
`software implementation to upper and lower layer commu-
`Tint. C1. oie ceccsssssssesesssssssesesssecsssssnseneeeen GO06F 11/00
`nications protocol software. The processor is also for inter-
`[51]
`
`US. Ch. cece
`.. 395/183.08; 395/183.15
`preting each frame contained inatestcasefile. In response
`[52]
`to interpreting an input data frame, the processorfeeds input
`Field of Search
` .............cc000 395/183.08, 183.01,
`[58]
`data contained in the input data frame to the communica-
`395/185.02, 183.1, 183.11, 183.14, 183.15,
`183.13
`tions protocol software implementation. In responseto inter-
`preting an output data frame,
`the processor compares
`received data outputted from the communications protocol
`software implementation with expected output data inter-
`preted from the output data frame, and generates a message
`depending on this comparison. Furthermore, the processor
`may simultaneously monitor the execution of the commu-
`nications protocol software implementation and generate a
`wake-up message if the communications protocol software
`implementation properly or unexpectedly ceases execution.
`In response to receiving a wake-up message, the processor
`writes an appropriate message in a test report file.
`
`[56]
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`5,233,611
`
`.............. 371/16.1
`8/1993 Triantafyllos et al.
`OTHER PUBLICATIONS
`
`Automator Developer Guide with Examples, Jul. 1991,
`Version QANDG1.0 pp. 1+ to 1-8; 1-12 to 1-15; 4-21 to
`4-24; and 5-24 to 5-34.
`Automator Programmer Reference Manual, Jul. 1991, Ver-
`sion QANPR 1.0 pp. 1-7; 3-47; 4-24 to 4-25.
`Automator User Guide, Jul. 1001, Version QANUG 1.0 pp.
`2-2; 3-8 to 3-9; and 5-1 to 5-6.
`
`15 Claims, 5 Drawing Sheets
`
` (NOAMALLY
`
`COMPLETED)
`WAKEUPER
`
`
`
`
`
` WAIT FOR QUTPUT
`
`
`
`(CATCHER)
`
`
`SEND
`
`“WAKEUP"
`HESSAGE
`TO TESTER
`
`
`
`SET VALVE OF
`AUTO-VARTABLE
`
`1
`
`CONFIGIT 1022
`
`CONFIGIT 1022
`
`1
`
`

`

`U.S. Patent
`
`Nov. 26, 1996
`
`Sheet 1 of 5
`
`5,579,476
`
`NOILYZINOUHONAS--
`)
`
`Ov}
`
`x
`+
`Pecomwascesseewwaussnennwecont
`
`HALS31
`
`OV
`
`
`
`HALSALYANO)
`
`eT
`
`Yaddh (LHYYOTHd)bOIG
`
`2
`
`

`

`U.S. Patent
`
`Nov. 26, 1996
`
`Sheet 2 of 5
`
`5,579,476
`
`0E¢
`
`
`
`HAV]YANO]
`
`SNOTLVOINNWWOD
`
`109010Ud
`
`SVM40S
`
`cS¢
`
`USAV1Usd!
`
`SNOTLVOINAWNO)
`
`100010Hd
`
`SUV405
`
`1S¢
`
`14!!4t!1(I||44'''‘‘'‘J‘!I'‘i}t(i4‘‘114!4itr)‘i‘iii'1i41{!t|tI‘t''‘
`
`che
`
`SNd-SNST
`
`Ove
`
`ec‘9I4
`
`(LHYHOTHd)
`
`dSaLSal
`
`Oke
`
`JTdN0)
`
`3
`
`
`
`
`
`

`

`U.S. Patent
`
`Noy. 26, 1996
`
`Sheet 30f5
`
`5,579,476
`
`FIG.
`3
`340
`(PRIORART]
`+0)
`330
`™
`i eeSD _ od
`
`
`||EXCEPTION |
`
`
`'|HANDLER !
`310
`|
`|
`
`LOWER
`|
`|
`
`TESTER
`|
`|
`
`Loee =
`
`
`
`4
`
`

`

`U.S. Patent
`
`Nov. 26, 1996
`
`Sheet 4 of 5
`
`5,579,476
`
`990
`
`940
`
`FEEDER
`ee epee eee eo ee ee ee
`
`ewe ee eee ee eee eee eee eee ee
`
`5/0
`
`5 F
`
`IG.
`
`
` LOWER PSEUDO
`PSEUDO
`UPPER
`
`
`
`TESTER
`
`500—~”
`
`WAKEUPER
`
`920
`
`5
`
`

`

`U.S. Patent
`
`Nov. 26, 1996
`
`Sheet 5of5
`
`5,579,476
`
`TESTER)—_(NORMALLY
`COMPLETED)
`WAKEUPER
`
`FIG. 6
`
`WAIT FOR QUTPUT
`(CATCHER)
`
`SEND
`“WAKEUP"
`MESSAGE
`TO TESTER
`
`GOTO
`SPECIFIED
`FRAME NO.
`
`MATCHED
`WITH
`EXPECTED?
`
`
`
`REPORT
`ERROR
`
`6
`
`

`

`5,579,476
`
`1
`AUTOMATIC TEST ENVIRONMENT FOR
`COMMUNICATIONS PROTOCOL
`SOFTWARE
`
`This is a continuation of application Ser. No. 08/139,303,
`filed Oct. 19, 1993, now abandoned, for AUTOMATIC
`TEST ENVIRONMENT FOR COMMUNICATIONSPRO-
`TOCOL SOFTWARE.
`
`FIELD OF THE INVENTION
`
`The present invention is directed to a system for testing
`communications software. In particular, the present inven-
`tion providesa test system for feeding input test data to, and
`receiving output data from, a communications protocol
`software implementation or combination of communications
`protocol software implementations.
`
`BACKGROUNDOF THE INVENTION
`
`10
`
`20
`
`2
`Traditionally, IUT’s have been tested in a test environ-
`ment 100 such as depicted in FIG. 1. An uppertester 110 is
`provided for simulating communications protocol software
`of a higher layer than an JUT 120 from which the TUT 120
`may receive messages, or to which the IUT 120 may
`transmit messages. For example, the JUT 120 may be an
`X.25PLP software implementation (layer 3 communications
`protocol software) and the upper tester may contain a T.70
`software implementation (layer 4 communications protocol
`software). The upper tester 110 transmits to, and receives
`from, the IUT 120 different kinds of simulated inputted data
`and control messages according to a higher layer protocol.
`Likewise, a lower tester 130 is provided for simulating
`communications protocol software on a lowerlayer than the
`TUT 120 to which the TUT 120 may transmit messages, or
`from which the [UT 120 may receive messages. In the above
`example, the lower tester 130 may contain a LAPB software
`implementation (layer 2 communications protocol soft-
`ware). The lower tester 130 transmits different kinds of
`simulated input data and control messages according to a
`lower layer protocol.
`The simulated input data and control messages of the
`upper and lower testers may be obtained automatically from
`a test script file 111 or 131, respectively. Alternatively, the
`input data and control messages may be inputted by a user
`in an interactive session. In response to receiving these
`inputted messages,
`the [UT may output messages to the
`uppertester 110 and/or the lowertester 130. As shown by the
`line 140, the test environment 100 must provide intricate
`synchronization procedures so that the upper tester 110 and
`lower tester 130 simulate, as close as possible, the actual
`environment in which the IUT 120 is to be deployed.
`FIG. 2 showsa first conventional communications pro-
`tocol software test system 200 which provides a test envi-
`ronment according to the general environment 100 (FIG. 1).
`The test system 200 has a testbed 210 which illustratively
`may be a computer such as a Micro Vax™, manufactured by
`Digital Equipment Corporation. The [UT 220 is maintained
`in a separate device 230, e.g., the actual hardware in which
`the IUT 220 is to be deployed. The testbed 210 communi-
`cates with the device 230 via an ISDN S-interface bus 212.
`Furthermore, a user 240 may input data and control mes-
`sages to, and receive data and control messages from the
`device 230. The device 230 in which the IUT 220resides has
`lower layer communication protocol software 251 and upper
`layer communication protocol software 252 which serve to
`temporarily store information transmitted to the [UT 220 or
`received from the user 240 or testbed 210.
`
`A communications system typically includes two devices
`which can communicate with each other either directly or
`via one or more intermediary communications devicesof the
`communications system. Some examples of communica-
`tions devices include: private branch exchanges, data termi-
`nal equipments, central office switches, ISDN customer
`premises equipment, etc. These devices communicate with
`one another according to one or more communications
`protocols or sets of semantic and syntactic rules for com-
`municating. Some examples of communications protocols
`include: LAPD (Q.921), Q.931, X.25PLP, LAPB, T.70,
`T.62, DTAM, and V.120.
`Illustratively, devices of the communications system con-
`tain protocol software for achieving communication accord-
`ing to the different communications protocols. Such protocol
`software may be hierarchically organized into levels called
`layers according to the Open SystemsInterconnection (OSI)
`model. According to OSI, communications protocol soft-
`ware for communicating on the highest layer may be used
`for requesting high level or abstract communications goals.
`Communications protocol software of each lower layer may
`be used for requesting more specific sub-goals of higher
`layers. Finally, communications protocol software of the
`lowest layer may be used for requesting low level or actual
`physical tasks (i.e., the actual transmission or reception of
`data signals). Typically, a communicating devicc communi-
`cates by first generating a request to perform somerelatively
`abstract goal using high layer communications protocol
`Jn operation, the testhed 210 serves as an automatic lower
`software. This in turn may cause the device to generate one
`tester. The testbed 210 automatically feeds simulation input
`350
`or more requests to perform more specific sub-goals of the
`data and control message data fromatest casefile 260 to the
`abstract goal using lower layer communications protoco]
`IUT 220. The user 240 serves as an upper tester. The user
`software, etc., until one or more requests are generated using
`manually inputs data and control messages to, and receives
`communications protocol software of the lowest layer.
`data and control messages from, the JUT 220 during an
`U.S. Pat. Nos. 3,869,603, 4,055,801, 4,108,358, 4,713,
`interactive session. For example, if the testbed 210 is a
`815, 4,759,019, 4,821,266, 4,916,641, and 5,111,402 dis-
`telephone, the user 240 mightlift the handset from the cradle
`close automated test systemsfor testing a variety of types of
`(creating an off-hook signal) and press keys on the telephone
`hardware. Advantageously, implementations of communica-
`keypad. The user receives data from the lowertester 210 in
`tions protocol software for operating devices of a commu-
`the form of a dial tone,etc.
`nications system are tested in a simulated environment
`FIG. 3 depicts a second conventional test system 300
`before they are deployed in the equipment of the commu-
`according to the conventional architecture 100 (FIG. 1). An
`nications system. To that end, the communications protocol
`upper tester 350 is provided which is also called the test
`software implementation under test (UT)is tested by emu-
`responder. Both the test responder 350 and the IUT 320 are
`lating the data and control message inputs which may be
`stored in the device 340 which can be a Sun Microsystems’
`received from higher and lower layer communications pro-
`SUN 3™, SUN 4™, or SPARC™computers.
`tocol software. If in response to these inputs, the [UT outputs
`Thetest system 300 has lower tester 330 in the form of a
`the correct data and control messages(as per the protocol of
`device such as a Sun Microsystems’ SUN 3™, SUN 4",or
`the higher or lower layer), the IUT passesthetest.
`
`40
`
`45
`
`55
`
`60
`
`65
`
`7
`
`

`

`5,579,476
`
`3
`SPARC™ computers. The lower tester device 330 is a
`separate device from the device 340. The two devices 330
`and 340 can communicate with each other, e.g., via a Sun
`Microsystems, SUNLINK OSI™interface 355.
`The lowertester 330 contains a test driver program 310,
`a referencer 360 and an exception handler 370. The refer-
`encer 360 is a certified communications protocol software
`implementation of the same communications protocol(s) as
`the IUT 320. The referenccr 360 may thus be used as a
`teference as to how the [UT 320 should respond.
`The exception handler 370 operates under the control of
`the test driver program 310. The exception handler 370 is
`provided for purposes of altering the messages communi-
`cated between the referencer 360 and the SUNLINK OSI
`interface 355. The exception handler 370 can thus create an
`abnormality in the messages outputted from the referencer
`360.
`
`In operation, the user compiles a test case file which
`includes the data to be inputted to, and outputted from, the
`upperLester 350 and the lower tester 330. Thetest driver 310
`then transfers instructions, using Test Driver Responder
`Protocol (TDRP) language, regarding the operation of the
`uppertester 350 to the test responder 350 via the referencer
`360. SUNLINK OSI"interface 355 and the IUT 320. The
`device 340 and the device 330 then execute appropriate
`sleps to automatically feed inpul messages to the TUT 320
`and to receive output messages therefrom.
`The prior art test systems 200 (FIG. 2) and 300 (FIG.3)
`have several disadvantages. In the test system 200, the user
`is required to enter appropriate input data in an interactive
`session. Such an interactive session is tedious and prone to
`errors. Moreover, because of the limited speed of human
`data entry, the test requires a long time to complete. The time
`factor is important when considering that the same test may
`be applied to several different IUT’s or reapplied to the same
`JUT in the course of developing the IUT.
`In using the test system 300, the IUT is compiled together
`with a test script file which contains the inputted messages
`of the upper and lowertester. The tests carried out by each
`test script file may be planned and separately stored ahead of
`time. These test script files may then be compiled with the
`IUT to produce an executable object (machine language)
`file. However, any time a test or the IUT is modified,thefiles
`must be recompiled. Furthermore, an object file (containing
`the executable machine language code) is produced each
`time the IUT is compiled with a test scriptfile. Thus, a great
`deal of effort must be expended in managingthefiles, for
`example, to ensure that only current objectfiles are stored
`and only unneeded object files are discarded.
`In both test systems 200 and 300, the data outputted from
`the IUT is transmitted through the upper and lowertesters.
`The upper and lowertesters may have certain error conten-
`tion capabilities in which the upper and lowertesters con-
`tinue to perform normal operations even in the event erro-
`neous data and control messages are outputted from the IUT.
`Thus, the upper and lowertesters may filter out erroneous
`data and control messages outputted from the JUT. This
`reduces the ability to fully test the IUT.
`It is thus an object of the present invention to overcome
`the disadvantages of the priorart.
`
`SUMMARYOF THE INVENTION
`
`The present invention achieves these objects by providing
`an automatic interpretive test system with a singletester,i-e.,
`the lower and uppertester are combined into a single tester.
`
`10
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`Furthermore, the test system automatically feeds input data
`simulating both the upper and lower tester, and checks
`output data received from the IUT, by sequentially inter-
`preting instructions from a test case file. The test casefile
`may be generated and stored prior to performing thetest.
`According to one embodiment, a test system is provided
`with a processor and a memory. Prior to conductingthetest,
`the IUT is stored in the memory. Furthermore,a test casefile
`is prepared containing a sequence of instructions called
`frames. The frames may be input data frames which contain
`data to be inputted to the [UT or output data frames which
`contain expected data to be outputted from the TUT. Tlus-
`tratively, the input and output data frames are sequentially
`ordered in the test case file to simulate a test session of
`requests and responses between the TUT and upper and
`lower layer communications protocol software. Optionally,
`the test system may be provided with a disk memory, such
`as ahard disk drive, for storing test case files. Thus, testfiles
`can be planned and stored prior to performing the tests.
`Duringa test, the processor executes a process called the
`tester. The tester process has four sub-processes: a catcher
`subprocess, an interpreter subprocess, a feeder subprocess
`and a comparator subprocess. The catcher subprocess is for
`receiving data outputted from the IUT to upper and lower
`layer communications protocol software. The interpreter
`subprocessis for interpreting each frame of the sequence of
`frames in the test case file. In response lo interpreting an
`input data frame, the processor executes the feeder subpro-
`cess to feed input data contained in the input data frame
`directly to the IUT. In response lo interpreting an output data
`frame, the processor executes the comparator subprocess to
`compare output data contained in the output data frame with
`corresponding output data received by the catcher subpro-
`cess. In response to the comparison, the processor illustra-
`tively generates messages indicating whetheror notthe data
`actually outputted from the IUT matched the expected data
`contained in the output data frames. [llustratively these
`generated messages are stored in a test report file.
`Illustratively,
`the processor also contemporaneously
`executes a second processcalled the wake-upper. The wake-
`upper monitors the IUT to determine if the [UT properly
`completes its execution or if it unexpectedly stops. The
`wake-upper process generates a wake-up message depend-
`ing on whether or not the IUT executes properly. Ilustra-
`tively, this is achieved by assigning a priority to the wake-
`upper process which is lower than the tester and IUT
`processes. This ensures that
`the wake-upper does not
`execute unless both the tester process and IUTareidle.
`The wake-up message generated by the wake-upper pro-
`cess is received by the processor while executing the catcher
`subprocess and analyzed by the processor while executing
`the comparator subprocess. Depending on the particular
`wake-up message, the processor generates an error message
`or successful execution message.Illustratively, in response
`to receiving a wake-up message, the processor writes an
`appropriate message in the above-mentioned test report file.
`In short, a flexible test system is provided for testing
`implementations of communications protocol software or
`communications software incorporating a combination of
`protocols. The tester system according to the present inven-
`tion provides the following advantages:
`(1) tedious manual data entry is avoided
`(2) human error is avoided
`(3) tests require less time to complete
`(4) test cascs may be predetermined and stored prior to
`performing thetest
`
`8
`
`

`

`5,579,476
`
`5
`(5) because the test system is an interpreter, the test cases
`need not be re-compiled with the IUT each time the IUT
`or test case is modified
`(6) file maintenance is minimized
`(7) because the test system interacts directly with the IUT
`rather than via upper or lower layer communication
`protocol software, no erroneous results produced by an
`IUT with a bugarefiltered out, and
`(8) because a single tester is provided which sequentially
`interprets frames from a test case file, intricate synchro-
`nization between upper and lowertesters is not needed.
`
`BRIEF DESCRIPTION OF THE DRAWING
`
`FIG. 1 depicts a conventional communications protocol
`software test system architecture.
`FIG.2 depicts a functional diagram ofa first conventional
`communications protocol software test system.
`FIG.3 depicts a functional diagram of a second conven-
`tional communications protocol software test system.
`FIG. 4 depicts a communications protocol software test
`system according to one embodimentof the present inven-
`tion.
`
`15
`
`20
`
`FIG. 5 depicts a functional diagram of the test system of
`FIG 4.
`
`25
`
`FIG.6 is a flowchart which schematicallyillustrates the
`operation of the test system depicted in FIGS. 4-5.
`
`6
`the test. Thus, the pseudo upper tester 550 and pseudo lower
`tester 540 may be mere buffers.
`The tester process 510 includes four sub-processes: a
`feeder sub-process 512, a catcher sub-process 514, an inter-
`preter sub-process 516 and a comparator sub-process 518.
`Theinterpreter sub-process 516 sequentially reads and inter-
`prets each frame or instruction of the test case file. These
`frames are discussed in greater detail below. The feeder
`sub-process 512 inputs data received from the interpreter
`sub-process 516 to the JUT 530. The catcher sub-process
`514 receives data outputted from the IUT 530. The com-
`parator sub-process 518 compares expected output data in an
`output data frameinterpretted by the interpreter sub-process
`516 with output data received by the catcher sub-process
`514.
`
`Appendix A depicts two illustrative test case files. A test
`case file may include a file header (e.g.,
`lines 1-8 or
`111-118), a summary (e.g., lines 10-29 or 120-133) and a
`sequence of one or more frames (e.g.,
`lines 31-110 or
`135-171). Each frame begins with a frame designator(e.g.,
`@_DX23), Frames may be input data frames, output data
`frames, SET frames or GOTO frames. Input and output data
`frames have the following syntax:
`
`[<frame no.>] <comment>
`HEADERhyhy hy hy bs bg Hy Bg by Ure, TR, Usever Info
`{<no. data items>} <data items>
`
`The frame number is a unique label from 1 to 65535
`which distinguishes the frame from other frames. Comments
`may be provided following the frame number. HEADER
`identifies the beginning of the header. The nine following
`bytes h, h, h, h, h; hg h, hg hy are reserved for later
`FIG. 4 depicts a test system 10 according to the present
`expansion. A value U,,,, follows which value serves as the
`invention.Ilustratively, the test system 10 is a multi-tasking
`process identifier of the IUT. The next bit T/R,, indicates
`computer system. The test system 10 has one or more
`whether the data contained in the frame is input data
`processors 12 or CPU’s for executing instructions of pro-
`(T,/R,=0) to be fed into the [UT 530 or whether the data is
`cesses. The test system 10 also has a main memory 14 and
`expected output data (T./R,=1) to be received from the IUT
`a disk memory 16 such as a hard disk drive. Typically, a
`530. The nextbit U,,,,.,is the process identifier of the pseudo
`process executed by the processor 12 is stored, at least in
`level via which data contained in the frameis to be received
`part, in the main memory 14. The disk memory 16 is used
`or transmitted (U,,,.=0 indicates pseudo upper level and
`for storing files. In addition,the test system 10 has a manual
`U,ver=!
`indicates pseudo lower level). A series of four
`input device 18, such as a keyboard, and a display device 20,
`“Info” digits follows indicating the kind of information
`such as a CRT or LCDdisplay. A data bus22 is also provided
`represented by the data frame.
`for transferring data between different devices of the test
`The number in the braces indicates the number of data
`system 10.
`itemsto be inputted to, or expected to be outputted from,the
`Tllustratively, the processor 12 of the test system is a
`TUT 530 which input data items or expected output data
`multi-tasking processor. In other words, the processor 12is
`items follow the numberin the braces. Data items can take
`capable of contemporaneously (or simultaneously,if the test
`the following form:
`system 10 has more than one processor 12) executing more
`(1) A 0-ff hexadecimal value representing one byte.
`than one process. This is usually achieved by alternately
`(2) ? indicating a byte of unspecified value. The value of
`executing one or moreinstructions of each non-idle process
`such a data item is irrelevant as this data item is to be
`in a round robin fashion. Due to the speed of the processor
`ignored.
`12, a user monitoring the input and output of a process via
`(3) Uzz indicating an auto-variable. Illustratively, 256
`the display device 18 has the impression that all processes
`355
`byte sized auto-variables are providedin the test system
`are executing simultaneously and continually. Each contem-
`500. Each auto variable functions as an ordinary vari-
`poraneously executing process may be assignedapriority
`able. Thus, the expression Uf8 returnsthe value stored
`which controls how often that process is executed by the
`in the auto variable Uf8. Operations may be performed
`processor 12 in relation to other processes. Processes with
`on auto variables using the following syntax: Uzz op1
`higher priorities are allotted more execution time in the
`zl op2 z2 op3 23, etc., where each operation op, can be
`processor 12 than processes with lower priorities.
`eeg77 ee? sehr ep? “8” (logical AND), “!”” Cogical OR),
`FIG. 5 showsa functional block diagram of a test system
`““” (logical XOR), “>” (right bit-shift), and “<” (left
`500 according to one embodimentof the present invention.
`bit-shift).
`As shown,the test system 500 executes a tester process 510
`(4) Szz indicating that a value is stored in an auto-
`and a wake-upper process 520. The tester process 510 can
`variable. This data item only occurs in an output data
`feed messages to, and receive messages from, the [UT 530
`frame for causing data outputted from the IUT 530 to
`via a pseudo lower tester 540 and a pseudo uppertester 550.
`be stored in a particular auto-variable.
`Illustratively, the tester process 510 performsall aspects of
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`30
`
`35
`
`45
`
`50
`
`60
`
`65
`
`9
`
`

`

`5,579,476
`
`7
`A SETframeis indicated by the word “SET” following
`the frame designator @__DX23. The SET frameis used for
`storing values in auto-variables. For example, in lines 36-37
`of Appendix A,16 is stored in the auto-variable Ue8 and 40
`is stored in the auto-variable Uf8.
`A GOTOframeis indicated by the word “GOTO” fol-
`lowing the frame designator @_DX23. The GOTO frame
`causes the processor 12 (FIG. 4) to branch to the frame
`indicated by the label contained in the GOTO frame. For
`example, after interpreting the GOTO framein line 171 of
`‘Appendix A, the processor 12 (FIG. 4) resumes interpreting
`with frame 4 on line 145. GOTO frames are useful for
`causing the processor 12 to repeatedly interpret a particular
`sub-sequence of frames of the test case file.
`FIG. 6 showsa flowchart illustrating the execution of the
`test system 500. Referring to FIGS. 4,5, and 6, the operation
`of the test system 500 is now illustrated. Initially,
`the
`processor 12 loads the TUT 530 into the memory 14 from,
`e.g., the storage device (disk memory) 16 for execution. The
`processor 12 then executes the interpreter sub-process 516
`by proceeding to step 602. In step 602, the processor 12
`determines whether there arc any test case files left to
`interpret. IIustratively, the test system 500 can interpret one
`or more sequences of frames in one or moretest casefiles.
`Tf all test case files have been interpreted, the processor 12
`ceases execution. Otherwise,
`the processor executes step
`604 in which the processor 12 determines if there are any
`more frameslcft in the currently interpretedtest casefile. If
`not,
`the processor 12 switches to executing the catcher
`sub-process 514 by proceeding to step 606. If there are
`framcsleft, the processor 12 continues executing the inter-
`preter sub-process 516 by interpreting the next framein step
`610.
`In step 610, the processor 12 determines whether the
`currently interpreted frame is an input/output data frame, a
`SET frame or a GOTO frame. In the event that the frame is
`a GOTOframe, the processor 12 executes step 612. In step
`612, the processor 12 causes interpretation to branch to the
`frame indicated by the label in the GOTO frame. That is,
`when the processor 12 executes a step requiring the inter-
`pretation of the next frame (step 610), the processor 12 will
`resume interpretation at the frame indicated by the label in
`the GOTOframe. The processor 12 then returns to step 604.
`In the event the frame is a SET frame, the processor 12
`executes step 614. Therein,
`the processor 12 stores an
`appropriate value in the auto-variable indicated by the SET
`frame. The processor then returns to step 604.
`In the event the frame is a data frame (i.e., input or
`output), the processor 12 proceedsto step 616. Therein,the
`processor 12 determines whether the data frame is an input
`data frame or an output data frame. In the event the data
`frame is an input data frame, the processor 12 executes the
`feeder sub-process 512 by proceeding to step 620. Therein,
`the processor 12 feeds the data contained in the input data
`framedirectly to the IUT 530. The processor 12 then returns
`to step 604.
`In the event the data frame is an output data frame, the
`processor 12 first switches to executing the catcher sub-
`process 514 by proceeding to step 622. In step 622, the
`processor 12 waits until cithcr output data is received from
`the IUT 530 via the pseudo upper tester 540 or pseudo lower
`tester 550 or until a wake-up message is received from the
`wake-upper process 520 (as discussed in greater detail
`below).
`When a wake-up message or output data is received, the
`processor executes the comparator sub-process 518 by pro-
`ceeding to step 624. Therein, the processor 12 first deter-
`
`20
`
`25
`
`40
`
`45
`
`50
`
`55
`
`65
`
`3
`mines whether a wake-up message or output data was
`received. lf a wake-up message was received, the processor
`12 proceeds to step 626, wherein the processor 12 reports an
`error in a test report file and ceases execution. Otherwise, the
`processor 12 proceedsto step 628 in which the processor 12
`compares the actually received output data with the output
`data contained in the interpreted output data frame. If they
`are notthe same,the processor proceedsto step 626, wherein
`the processor reports an error in the test report file and ceases
`execution. Otherwisc, the processor 12 returns to step 604.
`In addition, the processor 12 mayalso illustratively indicate
`in the test report file that data was properly received if the
`received output data matches the expected output data. For
`example, see Appendix B which shows two sample test
`report files generated for the twotest case files of Appendix
`A.
`
`the processor 12 waits to receive data
`In step 606,
`outputted from the IUT 530 via the pseudo upper or pseudo
`lower tester 540, 550 or a wake-up message from the
`wake-upper 520 (in a similar fashion to siep 622). Upon
`receiving either output data or a wake-up message,
`the
`processor 12 switches to executing the comparator sub-
`process 518 by proceeding to step 608. In step 608,
`the
`processor 12 determines whether a wake-up message or
`output data from the IUT 530 was received. If output data
`was received, then an error has occurred. Illustratively, all
`output data must be anticipated by an output data frame.
`Because, there are no more data frames to be interpreted
`(step 604) there are no output data frames anticipating
`outputdata. In such a case,the processor 12 proceeds to step
`626 and an error is written in the test report file.
`On the other hand, while executing the wake-upper pro-
`cess 520, the processor 12 outputs a message when the IUT
`530 completes its execution. If a wake-up message was
`received, then the wake-upper 520 has simply indicated that
`the IUT 530 properly completed the test. Thus, the processor
`returns to step 602 and the nexttest case file is interpreted.
`Contemporaneously while the processor 12 executes the
`tester process 530,
`the processor 12 also executes the
`wake-upper process 520.Illustratively, the wake-upper pro-
`cess 520 executes at a lower priority than the tester process.
`In such a case, the wake-upper process 520 does not execute
`unless and until the tester process 510 and IUT 530 are both
`idle.
`The sole purpose of the wake-upper process is to notify
`the tester process 510 if the JUT 530 properly ceases
`execution when expected or if the IUT 530 unexpectedly
`ceases execulion.If either condition occurs, the processor 12
`(while executive the wake-upper process 520) generates an
`appropriate wake-up message. The wake-up message is
`received while the processor 12 executes the catcher sub-
`process 514 of the tester process 510.
`Thetest system 500 utilizes message passing as the only
`channel for communicating between different executing
`processes such as the IUT 530,the tester process 510 and the
`wake-upper process 520. Almostall protocol software can
`be implemented as a process which communicates in such a
`fashion. Thus, the test system 500 can be used to test the
`majority of communications protocol software. The test
`system 500 can be used with the following protocols: LAPD
`(Q.921), Q.931, X.25PLP, LAPB, T.70, T.62, DTAM, and
`V.120.
`The interpretive test system provides significant advan-
`tages over the prior art. For example, the testing of X.25PLP
`using the conventional system 200 of FIG. 2 would require
`over four hours for applying all 90 test cases of a particular
`test plan. In contrast, the test system 500 can apply the same
`
`10
`
`10
`
`

`

`5,579,476
`
`9
`tests in less than four minutes and automatically generates a
`test report. Moreover, the test coverage is