THINC: A Virtual Display Architecture for Thin-Client Computing

Text-only Preview

THINC: A Virtual Display Architecture
for Thin-Client Computing
Ricardo A. Baratto, Leonard N. Kim, and Jason Nieh
Department of Computer Science
Columbia University, New York, NY, USA
{ricardo, lnk2101, nieh}
Rapid improvements in network bandwidth, cost, and ubiq-
In the last two decades, the centralized computing model
uity combined with the security hazards and high total cost
of mainframe computing has shifted toward the more dis-
of ownership of personal computers have created a growing
tributed model of desktop computing. However, as these
market for thin-client computing. We introduce THINC, a
personal desktop computers become prevalent in today’s
virtual display architecture for high-performance thin-client
large corporate and government organizations, the total cost
computing in both LAN and WAN environments. THINC
of owning and maintaining them is becoming unmanageable.
virtualizes the display at the device driver interface to trans-
The use of mobile laptops and handheld devices to store and
parently intercept application display commands and trans-
process information poses additional administration and se-
late them into a few simple low-level commands that can be
curity issues. These devices often contain sensitive data that
easily supported by widely used client hardware. THINC’s
must be carefully secured, yet the devices themselves must
translation mechanism efficiently leverages display semantic
travel in insecure environments where they can be easily
information through novel optimizations such as offscreen
damaged, lost, or stolen. This management and security
drawing awareness, native video support, and server-side
problem is particularly important for the medical commu-
screen scaling. This is integrated with an update delivery ar-
nity, given the increasing use of computing in medicine, the
chitecture that uses shortest command first scheduling and
urgent need to comply with HIPAA regulations [17], and the
non-blocking operation. THINC leverages existing display
huge privacy consequences for lost patient data.
system functionality and works seamlessly with unmodified
Thin-client computing offers a solution to the rising man-
applications, window systems, and operating systems.
agement complexity and security hazards of the current com-
We have implemented THINC in an X/Linux environ-
puting model by leveraging continued improvements in net-
ment and compared its performance against widely used
work bandwidth, cost, and ubiquity to return to a more
commercial approaches, including Citrix MetaFrame, Mi-
centralized, secure, and easier-to-manage computing strat-
crosoft RDP, GoToMyPC, X, NX, VNC, and Sun Ray. Our
egy. A thin-client computing system consists of a server and
experimental results on web and audio/video applications
a client that communicate over a network using a remote dis-
demonstrate that THINC can provide up to 4.8 times faster
play protocol. The protocol allows graphical displays to be
web browsing performance and two orders of magnitude bet-
virtualized and served across a network to a client device,
ter audio/video performance. THINC is the only thin client
while application logic is executed on the server. Using the
capable of transparently playing full-screen video and au-
remote display protocol, the client transmits user input to
dio at full frame rate in both LAN and WAN environments.
the server, and the server returns screen updates of the user
Our results also show for the first time that thin clients can
interface of the applications to the client.
even provide good performance using remote clients located
The thin-client approach provides several significant ad-
in other countries around the world.
vantages over traditional desktop computing. Clients can
Categories and Subject Descriptors: C.2.4 Computer-
be essentially stateless appliances that do not need to be
Communication-Networks: Distributed Systems – client/
backed up or restored, require almost no maintenance or
upgrades, and do not store any sensitive data that can be
lost or stolen. Mobile users can access the server from any
General Terms: Design, Experimentation, Performance
client and obtain the same persistent, personalized comput-
Keywords: thin-client computing, remote display, virtual-
ing environment. Server resources can be physically secured
ization, mobility
in protected data centers and centrally administered, with
all the attendant benefits of easier maintenance and cheaper
upgrades. Computing resources can be consolidated and
shared across many users, resulting in more effective utiliza-
Permission to make digital or hard copies of all or part of this work for
tion of computing hardware. It is not surprising that the
personal or classroom use is granted without fee provided that copies are
popularity of thin clients continues to rise [12] and the mar-
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, to
ket for thin-client systems is expected to grow substantially
republish, to post on servers or to redistribute to lists, requires prior specific
over the next five years [38, 43].
permission and/or a fee.
Moreover, the key remote display functionality used in
SOSP’05, October 23–26, 2005, Brighton, United Kingdom.
thin clients to decouple display from application execution
Copyright 2005 ACM 1-59593-079-5/05/0010 ...$5.00.

over a network enables a myriad of other benefits. Remote
tions encode display updates in a manner that is more com-
users can travel and access their full desktop computing en-
putationally and bandwidth efficient than commonly used
vironment from anywhere [16, 37]. Applications written for
compression-based approaches [41, 16].
one platform can be remotely displayed on a completely dif-
THINC carefully partitions functionality between client
ferent one without rewrite. Scientists can gain full access at
and server using low-latency mechanisms to provide fast per-
their desktops to specialized computer-controlled scientific
formance even in high latency WAN environments. THINC
instrumentation located at remote locations. Since display
introduces a shortest-job-first display command scheduling
output can be arbitrarily redirected and multiplexed over
to improve response time for interactive applications, a low-
the network, screen sharing among multiple clients becomes
latency push display update model that minimizes synchro-
possible. Thus, groups of users distributed over large geo-
nization costs between client and server, and a non-blocking
graphical locations can seamlessly collaborate using a single
drawing pipeline that integrates well with and maximizes
shared computing session. Furthermore, by mirroring local
the performance of today’s single-threaded window servers.
display output and redirecting it over the network, instant
THINC also provides server-side screen scaling, which min-
technical support can be provided [11, 14, 16] with the abil-
imizes display bandwidth and processing requirements for
ity to see exactly what the user sees on the desktop, enabling
small display handheld devices.
problems to be diagnosed and corrected much more quickly.
We have implemented THINC as a virtual display driver
Finally, virtualized computing infrastructure such as virtual
in the X window system and measured its performance on
machines can leverage remote display to avoid display de-
real applications. We have compared our THINC proto-
vice dependencies and display anywhere, further decoupling
type system against the most current and widely-used thin-
their execution from underlying hardware.
client systems, including Citrix MetaFrame XP [10], Mi-
However, thin clients face a number of technical challenges
crosoft Remote Desktop [11], Sun Ray [36], VNC [41], Go-
before achieving mass acceptance. The most salient of these
ToMyPC [16], X [35], and NX [28]. All of these systems
is the need to provide a high fidelity visual and interactive
are commercially-deployed, continue to be heavily devel-
experience for end users across the vast spectrum of graph-
oped and supported by substantial engineering efforts, and
ical and multimedia applications commonly found on the
have recently released versions available. Rather than just
computing desktop. While previous thin-client approaches
discussing the benefits of THINC compared to other ap-
have focused on supporting office productivity tools in LAN
proaches, we conducted a direct comparison with highly-
environments and reducing data transfer for low bandwidth
tuned commercial and open-source thin-client systems to
links such as ISDN and modem lines, they do not effectively
quantitatively measure the performance of THINC versus
support more display-intensive applications such as multi-
the current state-of-the-art.
media video, and they are not designed to operate effectively
Our experimental results on web and multimedia appli-
in higher latency WAN environments. WAN performance is
cations in various network environments demonstrate that
particularly important given the growing number of thin-
THINC can provide an order of magnitude better perfor-
client application service providers attempting to provide
mance than many of these other approaches. These results
desktop computing services over the Internet [5, 16, 33].
illustrate the importance of not just the choice of display
We introduce THINC (THin-client InterNet Computing),
commands used, but also how the mapping of application-
a virtual display architecture for thin-client computing that
level drawing commands to protocol primitives can affect
provides high fidelity display and interactive performance
performance. THINC’s approach results in superior over-
in both LAN and WAN environments. THINC leverages
all application performance and network bandwidth usage.
the standard video driver interface, a well-defined, low-level,
Most notably, it is the only system capable of providing
device-dependent layer that exposes the video hardware to
audio/video playback with full-screen video at full frame
the display system. Instead of providing a real driver for a
rate. As a result, THINC provides a key technology for en-
particular display hardware, THINC introduces a simple vir-
abling application service providers and utility computing
tual display driver that intercepts drawing commands at the
with modern, multimedia-oriented applications [4].
device layer, packetizes them, and sends them over the net-
This paper presents the design and implementation of
work to a client device to display. Using a standard interface
THINC. Section 2 provides background and discusses re-
enables THINC to work seamlessly with existing unmodified
lated work. Section 3 presents an overview of the system ar-
applications, window systems, and operating systems. As a
chitecture and display protocol. Section 4 discusses the key
result, THINC avoids reimplementing existing display func-
translation mechanisms used in THINC. Section 5 presents
tionality in window servers and can leverage any continuing
THINC’s scheduling mechanisms to improve system interac-
advances in window server technology.
tivity. Section 6 describes THINC’s screen scaling support
THINC’s remote display protocol consists of a small set
for heterogeneous display devices. The implementation of
of efficient low-level commands that mirror the video dis-
THINC as a virtual display driver in the X window system
play driver interface and are easy to implement and acceler-
is discussed in Section 7. Section 8 presents experimen-
ate using widely-available commodity client video hardware.
tal results measuring THINC performance and comparing
THINC avoids the complexity and overhead of directly im-
it against other popular commercial thin-client systems on
plementing higher-level graphics commands used by appli-
a variety of web and multimedia application workloads. Fi-
cations. Instead, it transparently maps them to its protocol
nally, we present some concluding remarks.
command set. This is done by taking advantage of informa-
tion available at the video display driver interface and and
preserving that information throughout display processing
Because of the importance of developing effective thin-
using transformation optimizations such as offscreen draw-
client systems, many alternative designs have been proposed.
ing awareness and native video support. These transforma-
These approaches can be loosely classified based on three re-

lated design choices: (1) where the graphical user interface
device. It maintains a local copy of the framebuffer state
of applications is executed, (2) how display commands from
used to refresh its display and forwards all user input di-
applications are intercepted so that display updates can be
rectly to the server for processing. When applications gener-
sent from server to client, and (3) what display primitives
ate display commands, the server processes those commands
are used for sending display updates over the network.
and sends screen updates over the network to the client to
Older thin-client systems such as Plan 9 [29] and X [35]
update the client’s local framebuffer. The server maintains
provide remote display functionality by pushing all user in-
the true application and display state, while the client only
terface processing to the client computer. Application-level
contains transient soft state.
display commands are not processed on the server, but sim-
This approach provides several important benefits. First,
ply forwarded to the client. This division of work is more
synchronization overhead across the network between the
apparent in X, where, the client is referred to as the “X
user interface and applications can be eliminated since both
server”, and applications running on the server are called
components run on the server. Second, no window server
“X clients”. X applications perform graphics operations by
software needs to run on the client, allowing for less com-
calling library functions in charge of forwarding application-
plex client implementation. Third, client processing require-
level display commands over the network to the X server. X
ments can scale with display size instead of graphical user
commands present a high-level model of the overall charac-
interface complexity, enabling clients to be designed as fixed-
teristics of the display system, including descriptions of the
function devices for a given display resolution. Fourth, since
operation and management of windows, graphics state, in-
all persistent state resides on the server, mobile users can
put mechanisms, and display capabilities of the system. By
easily obtain the same persistent and consistent computing
running the user interface on the client, user interface inter-
environment by connecting to the server from any client.
actions that do not involve application logic can be processed
Achieving these benefits with good system performance
locally without incurring network latencies. The use of high-
remains a key challenge. Citrix MetaFrame, Tarantella [34],
level application display commands for sending updates over
and Microsoft Remote Desktop translate application display
the network is also widely thought to be bandwidth efficient.
commands into a rich set of low-level graphics commands
However, there are several important drawbacks to this
that encode display updates sent over the network. These
approach. First, since application user interfaces and ap-
commands are similar to many of the commands used in
plication logic are usually tightly coupled, running the user
X. However, performance studies [22, 45] of these systems
interface on the client and application logic on the server of-
indicate that using a richer set of display primitives does
ten results in a need for continuous synchronization between
not necessarily provide substantial gains in bandwidth effi-
client and server. In high-latency WAN environments, this
ciency, particularly in the presence of multimedia content.
kind of synchronization causes substantial interactive per-
Furthermore, the added overhead of supporting a complex
formance degradation [22]. Second, the use of high-level
set of display primitives results in slower responsiveness and
application display commands, such as those used by X, in
degraded performance in WAN environments. MetaFrame
practice turns out to be not very bandwidth efficient [22, 36].
recently added video support by taking advantage of the
Third, the window server software used to process applica-
Windows media architecture to capture the encoded media
tion user interfaces is large and complex. Maintaining that
stream from the application, transmit it over the network,
software on the client requires frequent updates and software
and decode and playback on the client. However, video sup-
maintenance costs contrary to the zero administration goals
port is limited only to certain video formats and applications
of thin clients. Fourth, as application user interfaces become
that use the necessary Windows media framework. While
richer, they impose more complex processing requirements.
this approach reduces network utilization, it increases client
Running the user interface on the client necessitates replac-
complexity, as media decoding and support for different for-
ing clients at the same high frequency as traditional desktop
mats relies heavily on local software components, which need
PCs in order to run the latest applications effectively. Other-
to be bundled with and maintained on the client. A special-
wise, they grow outdated quickly as X-based terminals did
ized version of Remote Desktop in Windows Media Center
when web browsers came out in the early 1990s. Finally,
Edition takes a similar approach, but requires a specialized
storing and managing all display state at the client makes it
Windows server and hardware clients; video support is not
difficult to support seamless user mobility across locations.
available in standard Remote Desktop.
Proxy extensions such as low-bandwidth X (LBX) [44]
Sun Ray uses simpler 2D drawing primitives for sending
and NoMachine’s NX [28] have been developed to address
updates over the network. The original command set de-
some of these problems and improve X performance. LBX
veloped was simple and easy to implement, and was thor-
has been shown to have poor performance [21] compared
oughly evaluated [36], which motivated the use of a similar
to other thin-client systems [42]. NX is a more recent de-
command set for THINC. Sun Ray has since evolved and is
velopment that provides good X protocol compression and
now in its third major product version. However, it lacks
reduces the need for network round trips to improve X per-
efficient and transparent mechanisms to translate applica-
formance in WAN environments. Neither of these systems
tion display commands into its command set. For example,
address the maintenance costs associated with executing a
some application commands need to be reduced to pixel data
complete window system on the client.
then sampled to determine which drawing primitives to use.
More recent thin-client systems such as Citrix MetaFrame
This can be difficult to do effectively, and processing over-
[10], Microsoft Remote Desktop [11] which comes standard
head can adversely affect overall performance. Applications
with Windows, Sun Ray [36], and VNC [32], run the graph-
which generate display commands that Sun Ray cannot ef-
ical user interface of applications at the server, avoiding the
ficiently translate need to be explicitly modified to deliver
need to maintain and run complex window server software at
adequate performance. In particular, Sun Ray lacks trans-
the client. The client functions simply as an input-output
parent support for video playback. Sun Ray intercepts appli-

cation commands using a customized X server, which causes
Command Description
difficulty in keeping up with continuing advances in more
Display raw pixel data at a given location
widely supported window server implementations, such as
Copy frame buffer area to specified coor-
XFree86 and
VNC [41] and GoToMyPC [16] reduce everything to raw
Fill an area with a given pixel color value
pixel values for representing display updates, then read the
Tile an area with a given pixel pattern
resulting framebuffer pixel data and encode or compress it,
Fill a region using a bitmap image
a process sometimes called screen scraping. Other similar
systems include Laplink [24] and PC Anywhere [37], which
Table 1: THINC Protocol Display Commands
have been previously shown to perform poorly [25]. Screen
scraping is a simple process, and decouples the processing
need to maintain and update its own window server code
of application display commands from the generation of dis-
base. THINC can also support new video hardware features
play updates sent to the client. Servers do the full trans-
with at most the same amount of work necessary to support
lation from application display commands to actual pixel
them in traditional hardware-specific display drivers. Fi-
data, while clients can be very simple and stateless, allow-
nally, as the video device driver layer still provides semantic
ing for maximum portability of the system across client plat-
information regarding display commands, THINC utilizes
forms. However, display commands consisting of raw pixels
those semantics to encode application commands and trans-
alone are typically too bandwidth-intensive. For example,
mit them from the server to the client in a manner that is
using raw pixels to encode display updates for a video player
both computationally and bandwidth efficient.
displaying at 30 frames per second (fps) full-screen video
With this virtual display approach, THINC uses a small
clip on a typical 1024x768 24-bit resolution screen would
set of low-level display commands for encoding display up-
require over 0.5 Gbps of network bandwidth. As a result,
dates, inspired by the core commands originally used in
the raw pixel data must be compressed. Many compression
Sun Ray [36]. The display commands mirror a subset of
techniques have been developed for this purpose, including
the video display driver interface. The five commands used
FABD [15], PWC [3], and TCC [7, 8, 6]. Generating display
in THINC’s display protocol are listed in Table 1. These
updates in this manner is computationally intensive since
commands are ubiquitously supported, simple to implement,
the original application display semantics are lost and can-
and easily portable to a range of environments. They mimic
not be used in the process. However, these inefficiencies
operations commonly found in client display hardware and
may be less important in the context of providing a user
represent a subset of operations accelerated by most graph-
with remote access to an otherwise idle PC [16].
ics subsystems.
Graphics acceleration interfaces such as
The aforementioned systems focus on providing thin-client
XAA and KAA for X and Microsoft Windows’ GDI Video
computing for general-purpose applications. Specialized ar-
Driver interface use a set of operations which can be syn-
chitectures that provide remote access to specific applica-
thesized using THINC’s commands. In this manner, clients
tions have also been proposed. The topic of remote mul-
need only translate protocol commands into hardware calls,
timedia access has been extensively explored; the Infopad
and servers avoid the need to do full translation to actual
project [39] created a device optimized for wireless access
pixel data, reducing display processing latency.
to multimedia content. Commercial systems such as SGI’s
THINC display commands are as follows. RAW is used to
VizServer [40] provide remote access to 3D content. Simi-
transmit unencoded pixel data to be displayed verbatim on
larly, WireGL and Chromium [18] enable cluster rendering
a region of the screen. This command is invoked as a last
systems that distribute the 3D processing load, but require
resort if the server is unable to employ any other command,
high bandwidth environments to operate efficiently.
and it is the only command that may be compressed to mit-
igate its impact on the network. COPY instructs the client
to copy a region of the screen from its local framebuffer to
The architecture of THINC virtualizes the display at the
another location. This command improves the user expe-
video device abstraction layer, which sits below the window
rience by accelerating scrolling and opaque window move-
server and above the framebuffer. This is a well-defined, low-
ment without having to resend screen data from the server.
level, device-dependent layer that exposes the video hard-
SFILL, PFILL, and BITMAP are commands that paint a
ware to the display system. The layer is typically used by
fixed-size region on the screen. They are useful for accel-
implementing a device-specific display driver that enables
erating the display of solid window backgrounds, desktop
the use of a particular display hardware. THINC instead
patterns, backgrounds of web pages, text drawing, and cer-
introduces a simple virtual display driver that intercepts
tain operations in graphics manipulation programs. SFILL
drawing commands at the device layer, packetizes them, and
fills a sizable region on the screen with a single color. PFILL
sends them over the network to a client device to display.
replicates a tile over a screen region. BITMAP performs a
THINC’s virtual display approach brings with it some key
fill using a bitmap of ones and zeros as a stipple to apply a
implementation and architectural advantages. Because the
foreground and background color.
display device layer sits below the window server proper,
For high fidelity display, all THINC commands are de-
THINC avoids reimplementing display system functional-
signed to support full 24-bit color as well as an alpha chan-
ity already available, resulting in a simpler system that can
nel, a feature not supported by thin-client systems that
leverage existing investments in window server technology.
execute the graphical user interface of applications on the
In addition, using a standard interface enables THINC to
server. The alpha channel enables THINC to support graph-
work seamlessly with existing unmodified applications, win-
ics compositing operations [31] and work with more ad-
dow systems, and operating systems. In particular, THINC
vanced window system features that depend on these op-
can operate within unmodified window servers, avoiding the
erations, such as anti-aliased text. Although graphics com-

positing operations have been used in the 3D graphics world
THINC commands were reduced to raw pixels at any time,
for some time, only recently have they been used in the con-
semantic information regarding those commands is lost and
text of 2D desktop graphics. As a result, there is currently
it becomes difficult to revert to the original commands in
a dearth of support for hardware acceleration of these oper-
order to efficiently transmit them over the network.
ations, particularly with low-end 2D only cards commonly
THINC’s virtual video device translation layer builds on
used in more modest machines.
these three design principles by utilizing two basic objects:
THINC provides support for graphics composition by tak-
the protocol command object, and the command queue ob-
ing advantage of available client hardware acceleration sup-
ject. Protocol command objects, or just command objects,
port only when it is present. In the absence of such sup-
are implemented in an object-oriented fashion. They are
port, THINC’s virtual device driver approach allows it to
based on a generic interface that allows the THINC server
transparently fall back to the software implementation pro-
to operate on the commands, without having to know each
vided by the window system precisely for video cards lack-
command’s specific details. On top of this generic interface,
ing hardware support. By doing so, THINC guarantees the
each protocol command provides its own concrete implemen-
simplicity of the client while utilizing the faster server CPU
to perform the software rendering. In contrast, thin-client
As previously mentioned, translated commands are not
systems which push functionality to the client may need to
instantly dispatched to the client. Instead, depending on
perform the software rendering using the limited resources
where drawing occurs and current conditions in the system,
of the client computer.
commands normally need to be stored and groups of com-
mands may need to be manipulated as a single entity. To
handle command processing, THINC introduces the notion
of a command queue. A command queue is a queue where
The key aspect of THINC’s design is not the choice of
commands drawing to a particular region are ordered ac-
display primitives that it uses, but rather how it utilizes the
cording to their arrival time. The command queue keeps
virtual display approach to effectively and transparently in-
track of commands affecting its draw region, and guaran-
tercept application display commands and translate them
tees that only those commands relevant to the current con-
efficiently into THINC commands. There are three impor-
tents of the region are in the queue. As application drawing
tant principles in how THINC translation is performed.
occurs, the contents of the region may be overwritten. In
First, as the window server processes application requests,
the same manner, as commands are generated in response
THINC intercepts display commands as they occur, and
to these new draw operations, they may overwrite existing
translates the result into its own commands.
By translat-
commands either partially or fully. As commands are over-
ing at this point, THINC can use the semantic information
written they may become irrelevant, and thus are evicted
available about the command (and which is lost once pro-
from the queue. Command queues provide a powerful mech-
cessing is finished), to identify which commands should be
anism for THINC to manage groups of commands as a single
used. In particular, THINC can know precisely what dis-
entity. For example, queues can be merged and the resulting
play primitives are used, instead of attempting to infer those
queue will maintain the queue properties automatically.
primitives after the fact. Translation in many cases becomes
To guarantee correct drawing as commands are overwrit-
a simple one-to-one mapping from display command to the
ten, the queue distinguishes among three types of commands
respective THINC command. For example, a fill operation
based on how they overwrite and are overwritten by other
to color a region of the screen a given color is easily mapped
commands: complete, partial, and transparent. Partial com-
to a SFILL command.
mands are opaque commands which can be partially or com-
Second, THINC decouples the processing of application
pletely overwritten by other commands. Complete com-
display commands and their network transmission. This al-
mands are opaque commands that can only be completely
lows THINC to aggregate small display updates into larger
overwritten. Transparent commands are commands that de-
ones before they are sent to the client, and is helpful in
pend on commands previously executed and do not over-
many situations. For example, sending a display update
write commands already in the queue. The command queue
for rendering a single character can result in high overhead
guarantees that the overlap properties of each command
when there are many small display updates being generated.
type are preserved at all times.
Similarly, some application display commands can result in
many small display primitives being generated at the dis-
play device layer. Rasterizing a large image is often done
Offscreen Drawing
by rendering individual scan lines. The cost of processing
Today’s graphic applications use a drawing model where
and sending each scan line can degrade system performance
the user interface is prepared using offscreen video memory;
when an application does extensive image manipulation.
that is, the interface is computed offscreen and copied on-
Third, THINC preserves command semantics throughout
screen only when it is ready to present to the user. This
its processing of application display commands and manip-
idea is similar to the double- and triple-buffering methods
ulation of the resulting THINC commands. Since THINC
used in video and 3D-intensive applications. Though this
commands are not immediately dispatched as they are gen-
practice provides the user with a more pleasant experience
erated by the server, it is important to ensure that they are
on a regular local desktop client, it can pose a serious perfor-
correctly queued and their semantic information preserved
mance problem for thin-client systems. Thin-client systems
throughout the command’s lifetime. For example, it is not
typically ignore all offscreen commands since they do not di-
uncommon for regions of display data to be copied and ma-
rectly result in any visible change to the framebuffer. Only
nipulated. If copying from one display region to another
when offscreen data are copied onscreen does the thin-client
is done by simply copying the raw pixel values, the origi-
server send a corresponding display update to the client.
nal command semantics will be lost in the copied region. If
However, all semantic information regarding the offscreen

data has been lost at this point and the server must resort to
Video delivery imposes rather high requirements on the un-
using raw pixel drawing commands for the onscreen display
derlying remote display architecture. If the video is com-
update. This can be very bandwidth-intensive if there are
pletely decoded by applications on the server, there is little
many offscreen operations that result in large onscreen up-
the thin-client server can do to provide a scalable solution.
dates. Even if the updates can be successfully compressed,
Real-time re-encoding of the video data is computationally
this process can be computationally expensive and would
expensive, even with modern CPUs. At the same time, de-
impose additional load on the server.
livering 24fps of raw RGB data can rapidly overwhelm the
To deliver effective performance for applications that use
capacity of a typical network. Further hampering the abil-
offscreen drawing operations, THINC provides a translation
ity of thin-client systems to support video playback are the
optimization that tracks drawing commands as they occur
lack of well-defined application interfaces for video decoding.
in offscreen memory. The server then sends only those com-
Most video players use ad-hoc methods for video decoding,
mands that affect the display when offscreen data are copied
and providing support in this environment would require
onscreen. THINC implements this by keeping a command
prohibitive per-application modifications.
queue for each offscreen region where drawing occurs. When
While full video decoding in desktop computers is still
a draw command is received by THINC with an offscreen
confined to the realm of software applications, video hard-
destination, a THINC protocol command object is generated
ware manufacturers have been slowly adding hardware ac-
and added to the command queue associated with the desti-
celeration capabilities to video cards for specific stages of
nation offscreen region. The queue guarantees that only rel-
the decoding process. For example, the ability to do hard-
evant commands are stored for each offscreen region, while
ware color space conversion and scaling (the last stage of
allowing new commands to be merged with existing com-
the decoding process) is present in almost all of today’s com-
mands of the same kind that draw next to each other.
modity video cards. To allow applications to take advantage
THINC’s offscreen awareness mechanism also accounts for
of these advancements, interfaces have been created in dis-
applications that create a hierarchy of offscreen regions to
play systems that allow video device drivers to expose their
help them manage the drawing of their graphical interfaces.
hardware capabilities back to the applications. With its ap-
Smaller offscreen regions are used to draw simple elements,
proach, THINC provides a virtual “bridge” between the re-
which are then combined with larger offscreen regions to
mote client hardware and the local applications, and allows
form more complex elements. This is accomplished by copy-
applications to transparently use the hardware capabilities
ing the contents of one offscreen region to another. To pre-
of the client to perform video playback across the network.
serve display content semantics across these copy operations,
THINC supports the transmission of video data using
THINC mimics the process by copying the group of com-
widely supported YUV pixel formats. A wide range of YUV
mands that draw on the source region to the destination
pixel formats exist that provide efficient encoding of video
region’s queue and modifying them to reflect their new lo-
content. For example, the preferred pixel format in the
cation. Note that the commands cannot simply be moved
MPEG decoding process is YV12, which allows normal true
from one queue to the other since an offscreen region may
color pixels to be represented with only 12 bits. YUV for-
be used multiple times as source for a copy.
mats are able to efficiently compress RGB data without loss
When offscreen data are copied onscreen, THINC executes
of quality by taking advantage of the human eye’s ability
the queue of display commands associated with the respec-
to better distinguish differences in brightness than in color.
tive offscreen region. Because the display primitives in the
When using YUV, the client can simply transfer the data to
queue are already encoded as THINC commands, THINC’s
its hardware, which automatically does color space conver-
execution stage normally entails little more than extracting
sion and scaling. Hardware scaling decouples the network
the relevant data from the command’s structure and passing
transfer requirements of the video from the size at which it
it to the functions in charge of formatting and outputting
is viewed. In other words, playing back a video at full screen
THINC protocol commands to be sent to the client.
resolution does not incur any additional overhead over play-
simplicity of this stage is crucial to the performance of the
ing it at its original size, because the client hardware trans-
offscreen mechanism since it should behave equivalently to a
parently transforms the stream to the desired view size.
local desktop client that transfers pixel data from offscreen
THINC’s video architecture is built around the notion of
to onscreen memory.
video stream objects. Each stream object represents a video
In monitoring offscreen operations, THINC incurs some
being displayed. All streams share a common set of char-
tracking and translation overhead compared to systems that
acteristics that allow THINC to manipulate them such as
completely ignore offscreen operations. However, the domi-
their format, position on the screen, and the geometry of
nant cost of offscreen operations is the actual drawing that
the video. In addition, each stream encapsulates informa-
occurs, which is the same regardless of whether the opera-
tion and state for its respective format. The THINC server
tions are tracked or ignored. As a result, THINC’s offscreen
uses its translation architecture to seamlessly translate from
awareness imposes negligible overhead and yields substan-
application requests to video commands which are forwarded
tial improvements in overall system performance, as demon-
to the client. Additional protocol messages are used to ma-
strated in Section 8.
nipulate video streams, and they allow operations such as
initialization and tearing down of a video stream, and ma-
Audio/Video Support
nipulation of the stream’s position and size.
Audio streams are not as resource intensive as video streams
From video conferencing and presentations to movie and
and THINC supports audio by simply applying its virtual
music entertainment, multimedia applications play an ev-
display driver approach to the audio device to create a vir-
eryday role in desktop computing. However, existing thin-
tual audio driver that takes audio input, packetizes it, and
client platforms have little or no support for multimedia ap-
sends it over the network to a client device to display. By
plications, and in particular for video delivery to the client.

operating at the device layer, THINC provides transpar-
T will depend on all those commands with which it overlaps.
ent support for audio applications that can use many dif-
Second, from the set of dependencies, the largest command
ferent audio libraries. THINC timestamps both audio and
L is chosen, and the new command is added to the back of
video data at the server to ensure they are delivered to the
the queue where L currently resides. As queues are flushed
client with the same synchronization characteristics present
in increasing order, THINC’s approach guarantees that all
at the server. Due to space constraints and our primary fo-
commands upon which T depends will have been completely
cus here on remote graphical display, further details regard-
drawn before T itself is sent to the client. Although more
ing THINC audio/video synchronization support are beyond
sophisticated approaches could be used to allow the reorder-
the scope of this paper.
ing of transparent commands, we found that their additional
complexity outweighed any potential benefits to the perfor-
mance of the system.
In addition to the queues for normal commands, the sched-
THINC schedules commands to be sent from server to
uler has a real-time queue for commands with high interac-
client with interactive responsiveness and latency tolerance
tivity needs. Commands in the real-time queue take priority
as a top priority. THINC maintains a per-client command
and preempt commands in the normal queues. Real-time
buffer based on the command queue structure described in
commands are small to medium-sized and are issued in di-
Section 4 to keep track of commands that need to be sent
rect response to user interaction with the applications. For
to the client. While the client buffer maintains command
example, when the user clicks on a button or enters key-
ordering based on arrival time, THINC does not necessarily
board input, she expects immediate feedback from the sys-
follow this ordering when delivering commands over the net-
tem in the form of a pressed button image. Because a video
work. Instead, alongside the client buffer THINC provides a
driver does not have a notion of a button or other high-
multi-queue Shortest-Remaining-Size-First (SRSF) preemp-
level primitives, THINC defines a small-sized region around
tive scheduler, analogous to Shortest-Remaining-Processing-
the location of the last received input event. By marking
Time (SRPT). SRPT is known to be optimal for minimizing
updates which overlap these regions as real-time and deliv-
mean response time, a primary goal in improving the inter-
ering them sooner as opposed to later, THINC improves the
activity of a system. The size of a command refers to its
user-perceived responsiveness of the system.
size in bytes, not its size in terms of the number of pixels
THINC sends commands to the client using a server-push
it updates. THINC uses remaining size instead of the com-
architecture, where display updates are pushed to the client
mand’s original size to shorten the delay between delivery of
as soon as they are generated. In contrast to the client-pull
segments of a display update and to minimize artifacts due
model used by popular systems such as VNC [41] and GoTo-
to partially sent commands. Commands are sorted in mul-
MyPC [16], server-push maximizes display response time by
tiple queues in increasing order with respect to the amount
obviating the need for a round trip delay on every update.
of data needed to deliver them to the client. Each queue
This is particularly important for display-intensive applica-
represents a size range, and commands within the queue are
tions such as video playback since updates are generated
ordered by arrival time. The current implementation uses 10
faster than the rate at which the client can send update re-
queues with powers of 2 representing queue size boundaries.
quests back to the server. Furthermore, a server-push model
When a command is added to the client’s command buffer,
minimizes the impact of network latency on the responsive-
the scheduler chooses the appropriate queue to store it. The
ness of the system because it requires no client-server syn-
commands are then flushed in increasing queue order.
chronization, whereas a client-driven system has an update
Reordering of commands is possible with guaranteed cor-
delay of at least half the round-trip time in the network.
rect final output as long as any dependencies between a com-
Although a push mechanism can outperform client-pull
mand and commands issued before it are handled correctly.
systems, a server blindly pushing data to clients can quickly
To explain how THINC’s scheduler guarantees correct draw-
overwhelm slow or congested networks and slowly respond-
ing, we distinguish between partial, complete, and trans-
ing clients. In this situation, the server may have to block or
parent commands. Opaque commands completely overwrite
buffer updates. If updates are not buffered carefully and the
their destination region. Therefore, dependency problems
state of the display continues to change, outdated content is
can arise after reordering only if an earlier-queued command
sent to the client before relevant updates can be delivered.
can draw over the output of a later-queued command. How-
Blocking can have potentially worse effects. Display sys-
ever, this situation cannot occur for partial commands be-
tems are commonly built around a monolithic core server
cause the command queue guarantees that no overlap exists
which manages display and input events, and where display
among these types of commands, as discussed in Section 4.
drivers are integrated. If the video device driver blocks, the
Furthermore, since complete commands are typical of var-
core display server also blocks. As a result, the system be-
ious types of fills such as solid fills, their size is constantly
comes unresponsive since neither application requests nor
small and they are guaranteed to end up in the first sched-
user input events can be serviced. In display systems where
uler queue. Since each queue is ordered by arrival time, it is
applications send requests to the window system using IPC
not possible for these commands to overwrite later similar
mechanisms, blocking may eventually cause applications to
also block after the IPC buffers are filled.
On the other hand, transparent commands need to be
The THINC server guarantees correct buffering and low
handled more carefully because they explicitly depend on
overhead display update management by using its command
the output of commands drawn before them. To guarantee
queue-based client buffer. The client buffer ensures that out-
efficient scheduling, THINC schedules a transparent com-
dated commands are automatically evicted. THINC period-
mand T using a two step process. First, dependencies are
ically attempts to flush the buffer using its SRSF scheduler
found by computing the overlap between the output region
in a two-stage process. First, each command in the buffer’s
of T and the output region of existing buffered commands.

queue is committed to the network layer by using the com-
mand’s flush handler. Since the server can detect if it will
We have implemented a prototype THINC server in Linux
block when attempting to write to a socket, it can postpone
as a video device driver that works with all existing open
the command until the next flush period. Second, to pro-
source X server implementations, including XFree86 4.3, 4.4,
tect the server from blocking on large updates, a command’s
and 6.8. We have implemented a number of THINC
flush handler is required to guarantee non-blocking opera-
clients, including a simple X application, a Java client (both
tion during the commit by breaking large commands into
as a standalone application and a web browser applet), a
smaller updates. When the handler detects that it cannot
Windows client, and a Windows PDA client, demonstrating
continue without blocking, it reformats the command to re-
THINC’s client portability and simplicity.
THINC seam-
flect the portion that was committed and informs the server
lessly hooks into X’s existing driver infrastructure, and no
to stop flushing the buffer. Commands are not broken up
changes are required to applications or the window system.
in advance to minimize overhead and allow the system to
XFree86 and derived implementations are designed around
adapt to changing conditions.
a single-user workstation model where a server has exclu-
sive access to the computer’s display hardware, and multiple
server instances are not allowed to be active simultaneously.
The promise of ubiquitous computing access has been a
Because the THINC server does not access local hardware,
major driving force in the growing popularity of thin-client
THINC modifies the window server’s behavior from within
systems. To deliver on this promise, THINC decouples the
the video driver to allow multiple servers to be active at the
session’s original framebuffer size, from the size at which a
same time.
particular client may view it. In this way, THINC enables
RAW commands are the only commands where we ap-
access from a variety of devices by supporting variable client
ply additional compression to mitigate its impact on the
display sizes and dynamic resizing. For instance, to view a
network. The current prototype uses PNG[30] for this pur-
desktop session through a small-screen mobile device such
pose. To support resizing, we use a simplified version of
as a PDA, THINC initially presents a zoomed-out version
Fant’s resampling algorithm [13], which produces high qual-
of the user’s desktop, from where the user can zoom in on
ity, anti-aliased results with very low overhead. To provide
particular sections of the display. In sharp contrast to sim-
video support, THINC leverages the standard XVideo ex-
ilar client-only approaches in existing thin-client systems,
tension by implementing the necessary XVideo device driver
THINC’s display resizing is fully supported by the server.
hooks. THINC primarily exports the YV12 format to appli-
After a client reports its screen size to the server, subse-
cations, which we chose not only for its intrinsic compression
quent updates are automatically resized by the server to fit
characteristics, but more importantly, for the wide range of
in the client’s smaller viewport. When the user zooms in on
applications supporting it, and its use as one of the preferred
the desktop, the client presents a temporary magnified view
formats in MPEG codecs. For audio support, we use a vir-
of the desktop while it requests updated content from the
tualized ALSA audio driver implemented as a kernel module
server. The server updates are necessary when the display
to intercept audio data. The THINC audio driver utilizes
size increases, because the client has only a small-size ver-
ALSA’s driver interfaces and multiplexes its resources across
sion of the display, with not enough content to provide an
multiple THINC users. Applications interact with the driver
accurate view of the desktop.
using ALSA’s audio library.
The driver works in tandem
Server resize support is designed to minimize processing
with a per client daemon which is automatically signaled as
and network overhead while maintaining display quality and
audio data becomes available.
client simplicity. For this reason, resizing is supported differ-
For improved network security, THINC encrypts all traf-
ently for each protocol command. RAW updates can be easily
fic using RC4, a streaming cipher particularly suited for the
resized because they consist of pure pixel data which can be
kind of traffic prevalent in thin-client environments. Al-
reliably resampled, and more importantly, the bandwidth
though block ciphers can have a significant effect in the per-
savings are significant. Similarly for PFILL updates the tile
formance of the system, we have found the cost of RC4 to
image is resized. On the other hand, BITMAP updates cannot
be rather minimal, and the benefits far outweigh any mi-
be resized without incurring significant loss of display infor-
nor overhead in overall system performance. Our prototype
mation and generating display artifacts. Traditionally, anti-
also implements authentication using the standard UNIX
aliasing techniques are used to minimize the loss of infor-
authentication facilities provided by PAM (Pluggable Au-
mation from the downsize operation. However, anti-aliasing
thentication Modules). Our authentication model requires
requires the use of intermediate pixel values which bitmap
the user to have a valid account on the server system and to
data cannot represent. In this case, BITMAP updates are con-
be the owner of the session she is connecting to. To support
verted to RAW and resampled by the server. While this may
multiple users collaborating in a screen-sharing session, the
increase bandwidth usage, requiring the client to do resizing
authentication model is extended to allow host users to spec-
would be prohibitively expensive. Finally, resizing SFILL
ify a session password that is then used by peers connecting
updates represents no savings with respect to bandwidth or
to the shared session.
computation, and therefore they are sent unmodified.
As shown in Section 8, our approach provides substan-
tial performance benefits by leveraging server resources and
reducing bandwidth consumption, vastly outperforming the
To demonstrate the effectiveness of THINC, we conducted
client-only support present in other systems. Furthermore,
a direct comparison with a number of state-of-the-art and
since THINC can use the powerful server CPU to do most
widely used thin-client platforms. These were Citrix Meta-
of the resize work, it can use high quality resampling algo-
FrameXP, Microsoft Remote Desktop, GoToMyPC, X, NX,
rithms to provide superior display content to the user.
Sun’s Sun Ray, and VNC. We follow common practice and

To minimize application environment differences, we used
450MHz PII
common thin-client configuration options whenever possible.
Client display was set to 24-bit color except for GoToMyPC
which is limited to 8-bit color. To mimic realistic usage of
the systems over public and insecure networks, we enabled
RC4 encryption with 128-bit keys on all platforms which
IBM Netfinity 4500R 2x933MHz PIII 512MB RAM
supported it. For those which did not, namely X and VNC,
we used ssh to provide a secure tunnel through which all
SunRay I
traffic was forwarded. The ssh tunnel was configured to use
100 MHz uSPARC
Figure 1: Experimental Testbed
RC4. Following common practice, we configured X’s ssh
tunnel to also compress all traffic [20]. Any remaining thin-
refer to Citrix MetaFrameXP and Microsoft Remote Desk-
client configuration settings were set to their defaults for a
top by their respective remote display protocols, ICA (Inde-
particular network environment. ICA, RDP, and NX were
pendent Computing Architecture) and RDP (Remote Desk-
set to LAN settings when used in the LAN and WAN set-
top Protocol). Our measurements present the first quanti-
tings when used in the WAN. Some thin-client systems used
tative performance comparison of these systems. We mea-
a persistent disk cache in addition to a per-session cache.
sured their performance on common web and multimedia
To minimize variability, we left the persistent cache turned
audio/video applications in LAN, WAN, and 802.11g wire-
on but cleared it before every test was run.
less network environments. We also used a PC running all
We considered three different client display resolution and
applications locally as a baseline for representing today’s
network configurations: LAN Desktop, WAN Desktop, and
prevalent desktop computer model. Section 8.1 describes
802.11g PDA. LAN Desktop represents a client with a 1024 x
our experimental setup. Section 8.2 describes the applica-
768 display resolution and a 100 Mbps LAN network. WAN
tion benchmarks used for our studies. Section 8.3 presents
Desktop represents a client with a 1024 x 768 display resolu-
our measurement results.
tion and a 100 Mbps WAN network with a 66 ms RTT, which
emulates Internet2 connectivity to a US cross-country re-
Experimental Setup
mote server [22]. We conducted our WAN experiments using
the kind of high-bandwidth network environment that is be-
We compared the performance of various thin-client sys-
coming increasingly available in public settings [1]. 802.11g
tems using an isolated network testbed, and we measured
PDA represents a client with a 320 x 240 display resolution
wide-area THINC performance using PlanetLab [9] nodes
for a PDA-like viewing experience and a 24 Mbps network,
and other remote sites located around the world. As shown
which emulates an idealized 802.11g wireless network [2].
in Figure 1, our testbed consisted of six computers connected
We chose 802.11g over 802.11b to reflect 802.11g’s emergence
on a switched FastEthernet network: two thin clients, a
as the next standard for wireless networks. The added band-
packet monitor, a network emulator for emulating various
width capacity provides a more conservative comparison for
network environments, a thin-client server, and a web server
bandwidth intensive applications, such as video playback.
used for testing web applications. Except for the thin clients,
Since the purpose of the test was to measure performan