This week I was previewing a post by our Chief Scientist about why giving RAM to Infinio is better than giving it to the guest OS or application.

One of the arguments he makes is what he calls “statistical multiplexing” which he explains with this analogy:

Think of lines at a grocery checkout. If you have one line per checkout clerk, then some lines would move faster and some slower, because of the variation of both the load seen by (i.e. number of grocery items) as well as processing speed of individual clerks. Some faster clerks can have nobody in their line, and their processing capacity gets wasted – it could have been better utilized in helping out the other clerks who are overloaded.

This leads to one of my biggest pet peeves: the people at a busy set of lines who say things like “please form X number of lines – it will go faster.”  To all the people at the TSA, the grocery store, CVS, the RMV, and the bank, listen up:  That.  Is.  Not.  True.  

When the amount of time it takes to execute each transaction is variable and unpredictable, there are very few instances where it is faster to make separate lines rather than distribute the work as late in the process as possible.  The only exceptions I can think of are:

(1) if the process of distributing the work later rather than sooner is significantly more time consuming than the benefit, or

(2) if there isn’t enough scratch space to wait on allocating the work

In the grocery example, this would mean

(1) the person saying “you line 3, you line 5” takes too long – or the people in line don’t go directly to the checkout whose light is on as soon as it goes on.  But the math would have to be pretty extreme. I mean, the traffic cop would have to be *really* *really* slow for this still not to be beneficial for the people waiting.

(2)  the store (or airport) isn’t designed to have one long windy line, it’s designed to have 10 shorter straight lines.

Not having the space can be a limiting factor, but that rarely seems to be the case.  If it were really the problem, then I wouldn’t be in so many dang lines with someone telling me that it would be faster if I ignored the natural rules of, you know, MATH.

In practice, there is a supermarket who actually implemented this.  Bravo.