Multimedia Client Implementation on
`Personal Digital Assistants
`
`Markus Lauff and Hans-Werner Gellersen
`TecO - Telecooperation Office - University of Karlsruhe
`Vincenz-Prießnitz-Str. 1 - 76131 Karlsruhe - GERMANY
`Phone +49 (721) 6902-69, Fax +49 (721) 6902-16
`e-mail: {markus, hwg}@teco.uni-karlsruhe.de
`
`Abstract. Their small size and ease of use makes Personal Digital Assistants
`(PDAs) very attractive for inclusion as clients in mobile multimedia
`applications. Alas,
`the
`cost
`for
`integration of PDA in distributed
`multimedia systems is high, as the resource limitations on PDA require
`client adaptation, and as software development on PDA is comparatively
`expensive. We present PocketWeb, a tool facilitating PDA integration in
`multimedia applications based on the web protocols while freeing
`application developers from PDA specific software development. We have
`applied PocketWeb for integration of PDA as multimedia clients in
`MILLION, a distributed multimedia system developed in an ESPRIT
`project.
`
`Keywords. Multimedia Client, Personal Digital Assistant (PDA), Mobile
`Computing, Mobile Multimedia Applications, Multimedia Software
`Development.
`
`Introduction
`1
`The new generation of small computing devices such as the Apple Newton
`MessagePad, the US Robotics Pilot or the Psion Series 3c are currently primarily
`used in personal stand-alone applications, for example as electronic filofaxes. Now,
`their small size and ease of use also renders them attractive and potentially useful as
`clients in networked applications in mobile environments.
`In fact, a range of
`applications have already been reported, demonstrating the useful integration of
`Personal Digital Assistants (PDA) in networked applications, for instance in the
`health care domain or in field engineering [1,2,3]. The integration of PDA in
`networked applications, though, remains difficult and expensive for three reasons:
`(cid:129) Resource Limitations. PDAs impose limitations on client functionality because of
`comparatively limited user interface, memory and computing resources. These
`limitations imply a high adaptation cost for adapting a client to the available
`resources, for example for development of user interface components that require
`less screen real estate.
`(cid:129) Network-level integration. The facilities for connecting to networks vary with
`available PDAs, but in general are rather low-level, thus implying additional cost
`for realizing the connectivity required for a PDA multimedia client.
`
`Motorola PX 1012_1
`
`

`

`• Expensive PDA Software Development. PDA
`is
`software development
`comparatively expensive for a number of reasons. PDAs in general come with their
`own development environments [e.g. 4,5,6]. These environments have not yet
`matured to the efficiency and reliability available on standard platforms. Further,
`there is a lack of experience in PDA software development in general, and PDA
`software developers are hard to find. Last not least, software components developed
`for standard platforms are difficult to reuse or even to port to PDA, and reusable
`PDA software components are not yet available to any noticeable degree.
`
`In this paper, we present the tool PocketWeb as a generic solution for integration of
`PDAs as multimedia clients in mobile multimedia applications. Pocket Web
`facilitates integration of PDA in applications based on the World-Wide Web
`protocols. From an end user perspective, PocketWeb is a web browser on Apple
`Newton MessagePad. From an application developers perspective,
`though,
`PocketWeb enables PDA integration in networked applications without requiring
`software development on PDA, thus reducing the cost of PDA integration effectively.
`Regarding client adaptation to limited resources on PDA, PocketWeb embodies a
`number of built-in adaptation strategies based on a thorough analysis of the design
`space for a multimedia client on PDA.
`
`In the following section we will discuss resource limitations on PDA, and the options
`for handling them. This discussion leads to consideration of pre-processing of
`multimedia information for a PDA client. The discussion of the design space for a
`PDA multimedia client extends to consideration of network-level integration in
`section 3. In section 4, we will describe the PocketWeb tool and briefly compare it to
`related work. Finally, in section 5 we will describe the application of PocketWeb to
`integration of PDA in MILLION, a large distributed multimedia system for the
`tourism sector currently being developed in an ESPRIT co-funded project [7]. The
`paper concludes with a brief summary and outlook on further development.
`
`2 Resource Limitations and Multimedia Preprocessing
`
`Today's information system clients such as WWW browsers have lots of built-in
`modules to display the different types of multimedia information. Apart from the weak
`computing power, discussed later in this document, PDAs have several other
`restrictions that makes it difficult to present multimedia information.
`
`These are:
`•
` small display
`•
` limited colour depth
` small memory
`•
` weak computing power
`•
` poor digital audio interface
`•
`
`2.1 Restrictions concerning the display of PDAs
`
`The usual size of displays known from Desktop computers is between 14" to 21" and
`the size of the display from Laptop computers is about 12". Compared to theses
`values the size of PDA displays, from about 3" to 6", is very small.
`
`Motorola PX 1012_2
`
`

`

`Most of the information available on information systems are thought to be displayed
`on a display with at most 80 columns. It is no problem to reformat plain text to a
`smaller display, for example to 50 columns but this is not as easy for tables or
`multimedia information as pictures or video.
`
`2.1.1 Tables:
`
`Tables need a lot of space because they use additional characters to separate the single
`columns. This leads in almost every case to a larger width than the available 50
`columns on a PDA.
`
`There are several strategies to display a table on the small display:
`
`1. two scrollbars
`Usually the user has the possibility to navigate up and down through the document.
`In addition to this vertical scrolling a horizontal scrolling is used to view the table.
`
`This approach is very inconvenient for the user because it is necessary to scroll for
`every row from the left to the right margin to view the table. Also it is difficult to
`overlook the whole table and to remember the position within the table. If the thirst
`column can be locked, that means the first column is not scrolled and always shown,
`it is easier to remember the position within the document. The disadvantage is that
`the space is always used for the first column.
`
`2. reformat the single columns
`This approach is used in common HTML viewers. The available space is divided
`into the number of used columns and the text is reformatted to fit into the column.
`
`This above table is divided in 5 columns of almost the same width. The single
`columns are separated with one character as delimiter. So the average size of a single
`
`Motorola PX 1012_3
`
`

`

`column is about 9 characters. This means that usually only a maximum of two words
`fit on a single line within the column. A lot of space is lost if only one word fits on
`the line because the second word is too long. In this case we can force a word break to
`use the remaining characters. This is shown in the fifth column. Without a forced
`word break, that makes the document harder to read, the length of the document
`growth very fast.
`
`3. Don't use a table. Reformat the table to multiple sections.
`The idea that leads to this approach is, that a table contains in every row the
`information that belongs under a given topic to a given point. This is not always
`applicable but will do for the most cases.
`
`The following example shows the original table
`
`and the decomposition to multiple sections
`
`This approach is useful for tables where it is not necessary to compare the different
`rows.
`
`Motorola PX 1012_4
`
`

`

`4. Use floating columns
`This approach uses the basic characteristics of a table. A table is a formation with a
`separation into columns and rows. It is not essential that the columns always start at
`the same horizontal position. It only must be recognisable for the user to which
`column the current text belongs.
`If the display supports colours a better separation of the columns can be reached by the
`use of different background colours.
`This approach results in a very uncommon structure and is therefore only acceptable
`on very small displays.
`
`5. Ignore the table
`At first sight this approach may look mad or trivial but on a closer look it is at least
`reasonable to ignore the table construct in some cases. This approach can only be
`used in addition to another approach to display tables where it is not possible to
`ignore the table construct.
`
`This leads to the question which table constructs can be ignored? In many cases
`tables are used to have a nicer presentation of the information. But on a small display
`this brings the contrary result. Deciding where it is possible to ignore the table or not
`can only be done by an heuristic algorithm.
`
`Motorola PX 1012_5
`
`

`

`This example shows a typical case where tables are not necessary. The table is only
`used to separate headers in a pretty way.
`
`In addition to these approaches it is always possible to reduce the size of the font to
`increase the number of characters per line but this isn't very useful on a PDA because
`PDAs usually still use small fonts.
`
`2.1.2 Pictures:
`
`The problem with the pictures is, that the difference between the PDA and a PC is
`even more worse than with the before mentioned tables.
`The common resolution of a PC display is 800 x 600 pixel or more with at least 256
`colours.
`For example the Apple Newton MessagePad only has a monochrome display with a
`resolution of 320 x 240 pixel.
`
`Nevertheless it is necessary to be able to display as many multimedia information as
`possible in any way because they may contain inalienable information.
`On the other hand there are a lot of useless pictures in documents for example pictures
`displaying bullets or delimiters.
`
`Displaying images can fail because of several limitations (computing power, memory,
`size of the display) this sections describes how to solve the problems with the small
`display the other problems are described later in this document.
`
`In every case where the picture has a higher colour depth than the display the image
`must be dithered before it can displayed.
`
`Similar to the procedure with the tables there are two main strategies to display
`images.
`
`1. Use two scrollbars
`The easiest way to display a large picture is to use a horizontal and a vertical scrollbar
`to view the picture.
`There are two problems occurring with this solution:
`•
` it can be difficult to imagine the complete picture
`•
` the PDA must have enough memory to store the whole picture and enough
`computing power to scroll through the picture
`
`2. Scaling the image
`An obvious solution is to scale the image to fit on the display of the PDA. But
`depending on the original size and the content of the image the scaled image can be
`useless and for that reason a fallback to the previous approach should be possible.
`
`3. Ignore the image
`This approach is only applicable if the image contains no information, for example
`bullets or delimiters. A special algorithm has to decide if an image is worth to be
`displayed or not.
`
`Motorola PX 1012_6
`
`

`

`2.2 Restrictions because of the small memory and the low computing power
`
`The common PDA does not have a secondary memory for the storage of programs and
`data. Therefore all programs and data must be stored in the primary memory. With a
`usual size of about 2 - 4 MByte the memory of a PDA is a very restricting resource.
`
`2.2.1 Processing and displaying images
`
`For image processing on the PDA there are three things to do:
`1. load the image
`2. decode the image
`3. convert the image
`
`1. load the image
`As a matter of course we need the memory to store the image if we want to load it.
`But this is not always possible. Large images or truecolour images may need to much
`memory so they can not be stored in the original format. If this happens it is
`necessary to process the image on-the-fly.
`Processing on-the-fly means that the small parts of the image are decoded and
`converted immediately when they arrive on the PDA and stored in a new format.
`
`2. / 3. decode and convert the image
`Decoding a picture from the GIF or the JPG format to a raw pixel-format is a hard job
`for a PDA. Particularly decoding a JPG image is currently not done in a few seconds.
`In addition to the required computing power the image grows in size while it is
`decoded and more memory is necessary to store the decoded picture.
`Therefore the image is dithered (reduce the colourdepth) and scaled before it is stored.
`
`Conclusion loading, decoding and converting images
`In many cases it is not possible for the PDA to load, decode or convert a image
`because of the limited resources. Why should not another machine process the image
`for the PDA?
`The PDA is connected to a network with many powerful machines, why he should
`not use the computing power and memory from another machine to preprocess the
`information?
`The following section describes several possibilities of preprocessing information for
`the PDA.
`
`2.3 Preprocess information to eliminate limitations
`
`A PDA is one of the smallest machines in the world of information. It is very
`comfortable to use a PDA to view as many information as possible. But that is the
`problem: „as many information as possible„. If the PDA is responsible for decoding,
`formatting and processing the requested information the limits are reached very soon.
`Now it is conceivable that the PDA is able to give some jobs to another machine on
`the existing information network.
`We can now build several models depending on what is done by the PDA and what
`is preprocessed from another machine. The machine that is responsible for
`preprocessing is viewed as a proxy.
`
`Motorola PX 1012_7
`
`

`

`2.3.1 The PDA only displays prebuild screens
`
`The PDA send the request to a special PDA-Proxy. In addition to the request the
`PDA sends some information describing his possibilities, for example
`•
` size of the display (320 x 240 pixel)
`•
` resolution (50 dpi)
` number of colours (monochrome)
`•
` digital audio capabilities (no)
`•
` memory available (50 kByte)
`•
`The proxy now computes the page, depending on the possibilities of the PDA and
`sends the result as a bitmap to the PDA.
`If the page contains large elements (tables, pictures) the proxy generates a preview and
`integrates this preview into the page. The user than is able to request the complete
`objects as an independent page.
`
`Advantages:
`•
` There are no more restrictions concerning the original format of the information.
`The limits are defined by the memory, the type of the display and the digital
`audio possibilities of the PDA. The computing power of the PDA is not the
`restricting factor.
`
`Disadvantages:
`•
` The PDA is not able to request new pages without a proxy
`•
` A page is stored as bitmap and may need a lot of memory depending on the
`resolution and number of colours from the PDA
`
`Operational area:
`•
` PDA with low computing power
`•
` low resolution and a small number of colours, because the size of the computed
`bitmap
`
`2.3.2 The PDA can submit supported features with the request
`
`In addition to the request the browser sends his capabilities concerning the different
`HTML tags to the proxy. The proxy examines the HTML document and converts
`from the browser unsupported elements to supported elements.
`Example 1:
`If the browser is not able to format tables the proxy can restructure the table to a
`preformatted textarea using monospaced fonts.
`Example 2:
`If the browser can not display inline graphics the proxy can replace pictures that are
`used as bullets in front of a list with a usual bullet symbol.
`
`Advantages:
`•
` The proxy can replace some elements that are not supported by the browser with
`an at least readable alternative.
`
`Disadvantages:
`•
` The proxy must parse and modify the HTML document
`•
` If the browser uses heuristical algorithm to remove unsupported elements the
`information can be corrupted
`
`Motorola PX 1012_8
`
`

`

`Operational area:
`•
` The browser on the PDA is able to format standard items and the proxy is
`especially written to enhance the functionality of the browser
`
`2.3.3 The PDA can submit supported formats with the request
`
`With the request the PDA sends the supported formats or desired properties (size,
`number of colours) to the proxy. The proxy examines the requested data and tries to
`convert the information to one of the supported formats.
`If the proxy is able to recognise the identification of the browser sent within the header
`of the request, there is modification of the HTTP necessary. In this case the PDA
`needs not to sent the desired properties or formats, because the proxy knows these
`facts from the identification of the browser.
`
`For example:
`If the PDA only supports GIF images at a given maximum size and requests a page
`containing a bigger JPG-coded image, the proxy converts the JPG image to the GIF
`format and scales the image to the maximum size.
`
`•
`
`Advantages:
`•
` The PDA is also able to work if the proxy is not available
`•
` The proxy can use standard tools for image conversions the HTML document is
`not modified
` No modification of HTTP is necessary
`Disadvantages:
`•
` The proxy only enhances the functionality of the browser that are originally
`limited by the number of supported formats
` There is no support for preview items as described above
`Operational area:
`•
` The browser on the PDA is able to format standard items and the functionality
`should be transparent enlarged.
`
`•
`
`3 Network Connectivity for PDA
`
`To retrieve any information it is necessary that the client is connected to a source
`where the information is stored, usually this is an information server. In many cases
`the information is spread over a lot of servers that are connected through a network.
`
`There are several possibilities to connect a PDA to a network. Almost every PDA has
`other facilities to connect to a network.
`For example the Apple Newton MessagePad can connect directly to an AppleTalk
`network. To use the Newton with an TCP/IP network a serial line to a PPP1-server is
`necessary.
`A US Robotics Pilot has no possibility to connect to a network, the Pilot only has a
`serial connector that can be used to connect to another machine.
`
`1 Point to Point Protocol
`
`Motorola PX 1012_9
`
`

`

`A Psion Serie 3c also has only a serial connector with an overlying TCP/IP protocol
`to connect to a PPP-server or an IrDA infrared channel to connect to another Psion
`Serie 3c
`At the moment there is no PDA that is able to support a direct TCP/IP connection to
`a network and so in almost every case a serial line is used to connect to an network
`provider.
`With a PDA the usual mobile environment uses a modem to connect through the
`telephone line to a network provider.
`If we use a mobile phone we can speak from a real mobile environment because there
`are no restrictions we have to make against the environment. Using a mobile phone is
`surely the most convenient solution but it is also the most expensive one.
`A connection is either online or offline.
`Online means that we have a physical connection and so we can retrieve new
`information from our information provider. The opposite to this is offline meaning
`that there is no possibility to retrieve new information.
`A online connection always means maintaining a connection to an service provider
`and therefore producing costs.
`But an access to an information system is only meaningful if the costs are
`proportionate to the outcome.
`Because of this we have developed another interesting connection type besides online
`and offline that tries to use a few new elements to optimise the ratio between costs and
`outcome.
`We name this approach „semi-online connection„ meaning that we try to present the
`user the information like having a real online connection.
`
`3.1 „Semi-Online„ Connection
`
`If the standard request is used every information is requested with an own request
`statement. If the user wants to load the following information he has to open a new
`connection to do the request.
`In many cases it is possible to guess the information the user wants to have and to
`request the information in advance within the current connection.
`
`Whenever it is possible to modify the original information, it is suitable to add extra
`information to group the information to blocks which can loaded altogether to reduce
`the number of necessary connections. For this we have introduced a new HTML tag
`„REQUIRE“. The value of the tag is a page/picture that is required to view/follow
`the current page. With the help of this tag we can build a tree of information. The
`spanning tree is called a HTML package and is requested with a single request
`statement.
`
`Motorola PX 1012_10
`
`

`

`4
`
`PocketWeb: A Generic Multimedia Client
`
`As a result of our analysis we have developed „PocketWeb„ as an generic multimedia
`client on an Apple Newton MessagePad.
`PocketWeb is based on HTML/HTTP and
`FTP and extensions developed through our
`before mentioned studies.
`The current version 2.1 is available from the
`TecO.
`
`Features of PocketWeb:
`•
` support of most HTML 2.0 elements
`•
` pictures
` forms
`•
` flexible cache management
`•
` supports download of complete HTML-
`•
`packages
`
`for
`PocketWeb uses optimized methods
`requesting,
`displaying
`and
`storing
`multimedia
`objects
`on
`the Apple
`MessagePad.
`Some examples on how objects are requested
`or displayed have been explained in section
`2,3. Now we will show how the cache is
`optimized for performance and an optimum
`
`memory usage.
`
`Optimized cache architecture
`Instead of storing the information as they arrive the information is preprocessed and
`than stored in an object oriented cache architecture. This means, that the information
`within the HTML page is bind to the procedure that is responsible to display it.
`This cache architecture allows the information to be displayed very fast without
`wasting unnecessary memory.
`A comparison of the memory usage of cached pages between PocketWeb and the
`commercial product Nethopper from Allpen has shown, that Allpen uses 2-7 times
`more memory for caching pages than PocketWeb depending on the number of used
`HTML elements within the page.
`
`Optimized requests
`In the current version PocketWeb works with a special subclient. This subclient is
`responsible for translating serial requests to TCP/IP requests and for preprocessing
`information as explained in section 2.
`At the moment we are working on a new version of the subclient that acts as an usual
`proxy on the net and is able to preprocess the information as requested from the client.
`In addition to the explained optimizations we are thinking about adding a quality of
`service module that has been developed at the TecO for mobile clients to access the
`net.
`
`Motorola PX 1012_11
`
`

`

`5 Application of PocketWeb
`
`5.1 Use of PocketWeb Clients in MILLION
`
`We have refined and applied the PocketWeb tool in the ESPRIT project MILLION.
`The objective of MILLION is to develop a coherent set of tools and services for
`municipal tourism, including tools for commercial transactions. In the cities of
`Venice, Bologna and Khania/Crete MILLION system pilots are being installed
`currently. In these pilots, PDA are integrated for mobile multimedia information
`access. In the pilot evaluation phase, PDAs will be handed out by tourism agencies to
`tourists for the duration of their stay. Tourists can download multimedia information
`according to their personal preferences and the use the PDA in an offline mode as
`travel guide. This part of the MILLION project is called VIP and covers all areas of
`mobile access and transactions within MILLION using PDAs.
`With the help of PocketWeb and the ideas we describe in this document we have
`reached the goal to use a PDA as a multimedia client without adapting and
`developing new specialized software for every new area.
`
`5.2 Other Parts of MILLION
`
`MILLION BOOK
`MILLION BOOK is the fundamental part of MILLION and covers the integration and
`development of multimedia databases. A main goal of MILLION is the integration of
`existing databases of any type in a for the user transparent fashion.
`
`MILLION GOLD
`The Dutch company Digicash integrate their transaction and electronic cash payment
`system ecash in the MILLION project to allow secure transactions and payments on
`the net.
`
`MILLION VOICE
`This part of the projects studies the possibilities to add voice services to the
`MILLION system. For example one area for voice integration in MILLION is the
`access to the MILLION system through usual telephone using text to speech and
`voice recognition.
`
`5.3 Further Applications of PocketWeb
`
`Through the universal architecture of PocketWeb it can be reused in many areas to
`access multimedia information without developing new expensive software for PDAs.
`A very early version of PocketWeb [8] has been successful tested within the
`ViaPerfecta project to access a route optimisation system through a PDA.
`
`6 Conclusion
`
`PDAs are in principal very attractive as clients in mobile multimedia applications
`with their major advantages being small size, ease of use, and comparatively low cost.
`Integration of PDAs in distributed multimedia systems, though, implies a range of
`problems: handling of multimedia in the context of limited resources, connecting
`
`Motorola PX 1012_12
`
`

`

`PDA at network-level, and, in order to solve these issues, expensive software
`development on PDA. We have presented a generic solution to these problems based
`on the tool PocketWeb. While PocketWeb is primarily perceived as web browser for
`end users, it actually facilitates integration of PDA in intranet/internet applications
`based on the web protocols. PocketWeb embodies strategies for handling multimedia
`on PDA, thus minimising if not eliminating the cost for adaptation of user interfaces
`to PDA resources. Further PocketWeb uses the http and html protocols, abstracting
`from low-level communication and granting wide-spread acceptance and applicability.
`PocketWeb is applied within the ESPRIT project MILLION for integration of PDAs
`as clients in a multimedia system for the tourism sector. As part of the MILLION
`system, the PocketWeb-based multimedia clients will be evaluated in pilot
`installations in Venice, Bologna and Khania/Crete.
`
`Acknowledgements
`
`This work was partially supported by the European Union’s ESPRIT programme in
`Project 20772 MILLION, whose partners are Omega Generation, Gruppo Formula,
`Digicash, Venis, Intracom, MUSIC and the municipalities of Venice, Bologna and
`Khania.
`
`References
`
`1. Brian Grimm, Education Research Laboratories Inc., The Williams Pocket
`Clinical Consultant, Resident's Medical Reference Library, Contact:
`health.wave@quantum.net
`
`2. PDA for Field Engineers in Railway, Information Systems Group, Railway
`Technical Research Institute, 2-8-38, Hikari-cho, Kokubunji-shi, Tokyo 185,
`Japan.
`
`3. FieldWorker, Cindy Park, 551 Millwood, Toronto, Canada,
`http://www.fieldworker.com/
`
`4. Newton Developer Kit, Newton Script Programming Environment, Apple
`Computer Inc., http://www.newton.apple.com/
`
`5. Psion EPOC/16, SIBO hardware and software architecture, Organiser
`Programming Language,
`http://www.psion.com/developerreference/develbackindex.html
`
`6. U.S. Robotics Palm OS Software Development Kits,
`http://www.usr.com/palm/5034.html
`
`7. Multimedia Interactive Leading Life-giving Initiative On Net, Esprit Project
`MILLION, Omega Generation, http://www.omega.it/million/
`
`8. Gessler, S., Kotulla, A., PDAs as mobile WWW browsers. Proc. of
`2nd International WWW Conference, Chicago, Oct. 1994.
`
`Motorola PX 1012_13
`
`