How to test the Performance of a Thin Client Terminal

This article covers various considerations to be taken into account when testing thin client terminals. The aim of this document is to cover comparing different competing terminals – not different protocols – ie not RDP vs. ICA.
How to Test
The purpose of a thin client is to display the graphics primitives sent from the server in a way the user finds comfortable to use.

To make a meaningful comparison between different products it is important to remove the "human" element ie the use of the keyboard and mouse. There are many script based benchmarks that serve this purpose. (WinBench, Sysmarks, WinTach, Winstone...).
Considerations when using these Benchmarks
1 - To guarantee the reliability and repeatability of the tests we recommend using a dedicated machine (i.e. to be the only user) and to connect terminal under test directly to the server using a cross-over cable. This will avoid the effect of varying network load from the corporate network).

2. Most benchmarks are designed to test PCs or servers – the only relevant tests for thin-clients are the tests that measure graphical performance. (Disk, CPU, memory tests are irrelevant).

3 - The tests should lasts several minutes. This reduces measurement errors and reduces the effect of unavoidable non graphical functions such as opening files etc.

4 - Carry out the tests immediately after the logon, and reboot between each test. This will eliminate the effect of caching.
Interpretation of the Results
These benchmarks give a final score which is generally meaningless in a client/server environment. This score is based on 'system' performance and does not take into account network performance, which is an inherent factor in thin-client computing.

It is essential to consider impact of data-compression, data encryption and local caching when testing thin-clients, as these factors are not relevant when testing PCs and servers, so are not reflected in the benchmarks.

The only way to obtain a meaningful measurement from these benchmarks is to physically measure the time taken to complete with a stopwatch.

Note: The clock must be independent to the system being tested.
Please find below details of where the benchmarks mentioned in this article can be downloaded.

WinBench :
http://www.veritest.com/benchmarks/winbench/default.asp
(Download WinBench 99 version 2.0 with graphics)

Sysmarks :
http://www.bapco.com/

WinTach :
http://active-hardware.com/english/benchmarks/Wintach12.zip

WinStone :
http://www.veritest.com/benchmarks/bwinstone/default.asp

Other references
"Server Based Computing: Elements of Performance" published by Citrix describes the principles of thin client performance testing. Download from:  ftp.citrix.com/doclib/elemperf.pdf


http://www.axel.com/articles/wp0001_e.html