The inescapable problem of Latency within Data-based systems 2

It’s long troubled me people touting Real-Time Business Intelligence when its physically impossible to achieve true real-time; I’ve jotted down some thoughts in an attempt to bring order and better understanding to the reality of “Real-Time” – hopefully the post will help you in dealing with vendors, giving you some insight into what questions to ask and what answers you should expect.

Basically there are two ways of looking at real-time, the first is the capability of the hardware (what us IT folk understand and relate to) and second is business workflow, for example connectivity with the customer – for example as the customer traverses the store picking their shopping can business through IT e.g. a smart phone interact with the customer based on what they have put in their basket e.g. if I had just bought a jar of Bolognese  sauce the system would in “real-time” i.e. as the customer shops look at my previous buying habits, combine that with recipes that might include Bolognese sauce and in “real-time” i.e. as I shop give me suggestions as I walk in store – as opposed to via email “batch” that I’d likely get at home later that day having a dramatically weaker impact.

Definition: Latency (aka Delay)

Latency is simply the amount of time between the request and a response – the measured delay. A given operation will have many levels of latency some you can control and some you can’t be it physics, reliance on third parties or simply cost.


Network ping: when a client ping’s a server the round trip time is measured. Wired Local Area Networks tend to have sub-millisecond latency, WiFi varies depending on load and external interference, a ping over the internet can vary widely from a few milliseconds to 10’s of milliseconds – for example to ping from my base here in Harpenden to my co-loc server in Milton Keynes for 1KiB of data is 35 milliseconds which is significant if what I was doing relied on low latency for example synchronous database mirroring.

Hard Disk latency: given a request for a specific block on the disk the time it takes to get that block back, that total delay (latency) can be broken down into many factors like the protocol used from the host to the hard drive (SAS, SATA, USB etc.), mechanical latency because of the geometric properties of a hard drive (why we short stroke); to retrieve 1KiB of information from the disk would involve head movement latency for track seek, rotational latency waiting for the disk to rotate to the position within the track where it can commence reading and then protocol latency and so forth – the latency on a 15Krpm SAS drive will typically be from 1ms to 5ms, however, that again relies on just how busy the disk is and the disk queue!

Digital clock: if operationally (software/hardware) we measure time in milliseconds but the clock display only shows precision to seconds then we always have up-to 999 milliseconds of latency because of the granularity of the clock display and the measurement of the data. I suppose if bothered about consistency then we would make the clock display blank unless the seconds or milliseconds were in sync, or simply show milliseconds. Using the same example, perhaps the display had a 10 millisecond latency, if we show milliseconds then in reality we would always be 10 milliseconds behind!

Reporting: the time it takes from the user request and it being displayed to them.

SQL Server Database Synchronous Mirroring: a classic example where the client’s write (insert, update or delete) cannot continue until the mirror server signals the data is on disk, so the latency involved is numerous – network and storage (on multiple machines).

Data Based Latency – what we really mean

What a business “means” and “needs” in regard to “Real Time” are two entirely separate notions.

If you are a High Frequency Trader then the term real-time means extremely low latencies because milliseconds count when it comes to market movements – if you can analyse movements milliseconds before your competitors potentially you can make more money.

What level of latency would you consider vehicle tracking to be real-time? Standard consultancy answer: well, it depends :).  If the car is monitored via GPS then the accuracy will vary depending on the number of available satellites, speed of car, conditions etc. So the accuracy will be variable, it will also depend on the velocity the vehicle is travelling at. For navigation that is probably acceptable, however, what about telemetry from say a F1 car? Greater accuracy is required for example tracking the path through a corner. Final example, consider your speedometer – what latency is acceptable there? Little use latency measured in seconds because on a powerful car you can end up breaking the speed limit in no time.

So what is “Real-Time”

The definition of “real-time” in the business context must be defined by the business and for the workflow process point being modelled.

For IT equipment – you will never achieve true “real-time”; so folk touting “real time business intelligence” are either misguided, liars or just miss connecting in explanation.

My advice to anybody is to ignore vendor claims when they state “real-time”, do not be fooled by the marketing strap line, instead, look deeper, understand what you are trying to achieve and put workflow time slicer in place that gives a maximum “latency” for each workflow operation group and ask the vendor if it is achievable with their solution and at what cost.

What on earth do I mean by a workflow time slicer?

The workflow for me shopping is – enter store, get basket, [repeat until finished…. pick], queue at till, pack, pay, leave store; for each of those workflow operations a duration can be put in place, for example {queue at till} + {pack} gives enough time for a process to access your data, do some data mining and provide you with offers to entice you to visit again.

Anyway, hopefully my ramblings above give you some food for thought on the notation of “real-time”