You need to enable JavaScript to fully utilise this page.

Testbed and Methodology

Assessing a clock accuracy consists in comparing it to a trusted reference clock. In the context of software clocks, comparing two clocks boils down to comparing respective timestamps of the same event. This comparison faces two fundamental problems.

A full account of the problem and our approach to clock performance assessment can be found in our methodology paper. Here is a quick summary.

Generic Testbed

The performance of the RADclock (and other synchronisation algorithms) is assessed using real data captured on our testbed at the University of Melbourne. The testbed is composed of a variety of PC hosts providing hardware but also operating system diversity. The hosts run several clock synchronisation algorithms simultaneously. The clocks synchronise to a given time reference server.

We run a variety of Stratum-1 NTP servers on CUBINlab, using a Trimble Acutime Gold GPS receiver and a Stanford Research Systems PRS-10 Rubidium oscillator. We also use benefit from a Tekron International NTS-01 NTP server and a RIPE TTM box. Country wide, we have access to the National Measurement Institute NTP servers.

Different DAG cards from Endace are used to accurately tap and timestamp packets transiting through a 100Mbps hub.

Methodology

Each host generates an independent bi-directional flow of UDP packets to assess the clock performance. The DAG card timestamps each packet to provide an absolute time reference (the trusted one). On the host side, each packet is timestamped by each clock from within the kernel. The timestamps are created "back-to-back" to minimise the latency of timestamping and avoid being scheduled out.

Because of the fundamental problems faced by clock validation, we implemented a two-fold methodology to disambiguate clock performance from noise, up to certain bounds.

External Validation

The external validation compares one clock timestamps to the DAG card timestamps. This shows the absolute performance of a host clock, but only up to an error, due to system and NIC noise, which can be large depending on system and network load (see NIC Noise).

This noise has two distinct components, one due to noise variability, and the other to a constant term arising from the fact that the DAG card and the host clock are not collocated. The variable component can be approximated by the RTThost variability, while the constant component is bounded by the interval [-RTThost, +RTThost].

Internal Validation

The internal validation is the comparison of the timestamps of two host clocks running simultaneously. It is implemented with the Fair Compare strategy (back-to-back timestamping), which allows quasi noise free comparison: even if timestamping errors are large and highly variable, they will always cancel almost perfectly. The disadvantage is that this comparison is relative only. It provides insight into both the relative offset and stability of the two clocks, but can not tell us their absolute performance with respect to true time.

Testbed hosts

bandicoot

Intel Pentium III 664.51-MHz

256 MB

FreeBSD 8.1-RELEASE i386

RADclock version 0.3.0

3Com Corporation 3c905C-TX/TX-M [Tornado] (rev 78)

bettong

Intel(R) Pentium(R) 4 CPU 1400MHz 1395.76-MHz

511 MB

FreeBSD 7.3-RELEASE i386

RADclock version Not Installed

3Com Corporation 3c905C-TX/TX-M [Tornado] (rev 78)

bilby

Intel(R) Pentium(R) 4 CPU 1.90GHz 1893.86-MHz

128 MB

FreeBSD 8.0-RELEASE i386

RADclock version 0.2.3

3Com Corporation 3c905C-TX/TX-M [Tornado] (rev 78)

boodie

Intel(R) Pentium(R) 4 CPU 1.70GHz 1694.241MHz

497 MB

Linux 2.6.32 i686

RADclock version 0.4.0

3Com Corporation 3c905C-TX/TX-M [Tornado] (rev 78)

cuscus

Pentium III (Katmai) 548.638MHz

374 MB

Linux 2.6.26 i686

RADclock version 0.1.0

3Com Corporation 3c905B 100BaseTX [Cyclone] (rev 24)

echidna

Intel(R) Pentium(R) D CPU 3.00GHz 3000.525MHz

1012 MB

Linux 2.6.15-1-686-smp i686

RADclock version Not Installed

Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet PCI Express (rev 11)

Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet PCI Express (rev 11)

Endace Measurement Systems, Ltd Unknown device 378e (rev 04)

green

Pentium III/Pentium III Xeon/Celeron 548.63-MHz

384 MB

FreeBSD 8.1-RELEASE i386

RADclock version 0.3.0

3Com Corporation 3c905B 100BaseTX [Cyclone] (rev 24)

koala

Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz 2992.50-MHz

4736 MB

FreeBSD 8.1-RELEASE amd64

RADclock version Not Installed

lspci not found

kultarr

Intel(R) Core(TM)2 CPU 6400 2.13GHz

923 MB

Linux 2.6.35-22-server x86_64

RADclock version 0.3.0

Realtek Semiconductor Co., Ltd. RTL-8169 Gigabit Ethernet (rev 10)

morse

Intel(R) Pentium(R) 4 CPU 2.40GHz 2400.483MHz

504 MB

Linux 2.6.8-2-686 i686

RADclock version Not Installed

Intel Corp. 82540EM Gigabit Ethernet Controller (rev 02)

Unknown device 378e (rev 04)

mulgara

Intel(R) Pentium(R) 4 CPU 3.40GHz 3391.57-MHz

512 MB

FreeBSD 9.0-CURRENT amd64

RADclock version 0.4.x

Broadcom Corporation NetXtreme BCM5751 Gigabit Ethernet PCI Express (rev 01)

3Com Corporation 3c905C-TX/TX-M [Tornado] (rev 74)

numbat

Pentium III (Katmai) 596.894MHz

376 MB

Linux 2.6.32 i686

RADclock version 0.3.0

3Com Corporation 3c905B 100BaseTX [Cyclone]

platypus

Intel(R) Pentium(R) D CPU 3.00GHz 3000.12-MHz

2035 MB

FreeBSD 7.3-RELEASE amd64

RADclock version 0.3.0

Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06)

3Com Corporation 3c905C-TX/TX-M [Tornado] (rev 74)

Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet PCI Express (rev 11)

Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet PCI Express (rev 11)

possum

Intel(R) Pentium(R) D CPU 3.00GHz 3000.13-MHz

1024 MB

FreeBSD 8.1-RELEASE amd64

RADclock version 0.3.0

Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06)

Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet PCI Express (rev 11)

Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet PCI Express (rev 11)

potoroo

Pentium III/Pentium III Xeon/Celeron 596.92-MHz

255 MB

FreeBSD 5.3-RELEASE i386

RADclock version 0.0.6

3Com Corporation 3c905B 100BaseTX [Cyclone]

sugar-glider

Pentium III/Pentium III Xeon/Celeron 596.92-MHz

512 MB

FreeBSD 8.1-RELEASE i386

RADclock version 0.3.x

3Com Corporation 3c905B 100BaseTX [Cyclone]

tastiger

Intel(R) Pentium(R) D CPU 3.40GHz 3391.768MHz

936 MB

Linux 2.6.32.26 x86_64

RADclock version 0.4.0

Broadcom Corporation NetXtreme BCM5751 Gigabit Ethernet PCI Express (rev 01)

Realtek Semiconductor Co., Ltd. RTL-8169 Gigabit Ethernet (rev 10)

wallaby

Pentium III/Pentium III Xeon/Celeron 596.92-MHz

255 MB

FreeBSD 7.2-RELEASE i386

RADclock version 0.3.0

3Com Corporation 3c905B 100BaseTX [Cyclone]

wombat

Pentium III (Katmai) 548.659MHz

245 MB

Linux 2.6.32 i686

RADclock version 0.4.0

3Com Corporation 3c905C-TX/TX-M [Tornado] (rev 74)

3Com Corporation 3c905B 100BaseTX [Cyclone] (rev 24)