Monday, August 18, 2008

Differential Benchmarking

I wanted to find out the fastest way to return multiple values in the Microsoft CLR. In order to measure this, I needed to have two pieces of code that `do the same thing' except for the way values are returned. For a baseline, I used the Takeuchi function:
        static int Tak (int x, int y, int z)
        {
            return (!(y < x))
                ? z
                : Tak (Tak (x - 1, y, z),
                       Tak (y - 1, z, x),
                       Tak (z - 1, x, y));
        }
Tak (18, 12, 6) makes 63609 function calls and 47706 subtractions.

This version uses an out variable to return the value:
        void TakOut (out int answer, int x, int y, int z)
        {
            if (!(y < x)) {
                answer = z;
                return;
            }
            int x1;
            int y1;
            int z1;
            TakOut (out x1, x - 1, y, z);
            TakOut (out y1, y - 1, z, x);
            TakOut (out z1, z - 1, x, y);
            TakOut (out answer, x1, y1, z1);
        }
But the version using an out variable has an additional argument. I wanted to separate the cost of passing an additional argument from the total cost of using an out parameter, so I wrote another version of Tak that took an extra dummy argument:
        static int Tak1 (int dummy, int x, int y, int z)
        {
            return (!(y < x))
            ? z
            : Tak1 (z, Tak1 (y, x - 1, y, z),
                       Tak1 (x, y - 1, z, x),
                       Tak1 (dummy, z - 1, x, y));
        }
These run pretty quick, so I put them in a loop that runs them ten times and measures the total time. I ran this loop a thousand times and recorded the results.

I didn't expect to find Tak1 to be substantially faster than Tak, so I did some investigating. Look at the graph for Tak: You can clearly see a change in performance (there are two vertical segments). I had forgotten that my machine was in power save mode and was dynamically tuning the CPU clock speed. Somewhere in there the machine decided to hit the gas. I put the machine in performance mode to max out the clock and got a much more expected result.
In this graph, Tak is the normal Tak function. Tak1a and Tak1b are defined as follows:
        static int Tak1a (int dummy, int x, int y, int z)
        {
            return (!(y < x))
            ? z
            : Tak1a (dummy, Tak1a (dummy, x - 1, y, z),
                            Tak1a (dummy, y - 1, z, x),
                            Tak1a (dummy, z - 1, x, y));
        }

        static int Tak1b (int dummy, int x, int y, int z)
        {
            return (!(y < x))
            ? z
            : Tak1b (z, Tak1b (y, x - 1, y, z),
                        Tak1b (x, y - 1, z, x),
                        Tak1b (dummy, z - 1, x, y));
        }
We can see from the graph that it takes an additional 8-10 ns to process the additional argument, depending on whether we change the value or keep it the same. I'll speculate that this is because the processor can notice the re-use of the argument and avoid a register rename or something along those lines.

So what about passing an out variable? From this graph we can see that returning an answer in an out variable makes the code a factor of 2 slower. Using a ref variable is slightly worse, but that may be because we have to initialize the ref variable before we use it. In either case, that's quite a hit.

It isn't likely that returning an object from the heap will be fast, but we ought to actually mesaure it. This is easily done by simply declaring the return value to be of type object. The CLR will be forced to box the object on the heap.
Surprisingly, this isn't all that much slower than using an out or ref variable. What about returning a struct on the stack? It seems that this is quicker than using an out variable.

If you look closely at these benchmarks you will see that the baseline performance of Tak varies a bit from run to run. This could be because of a number of reasons. It doesn't matter all that much because I was interested in the difference between the various mechanisms of returning values.

All the code above was run with these settings:
# LENOVO ThinkPad T61
# 2 processors
#     GenuineIntel
#     Intel(R) Core(TM)2 Duo CPU     T7700  @ 2.40GHz
#     x64 Family 6 Model 15 Stepping 11
#     2401 MHz
# Working set:  9969664
# Overflow checking is disabled.
# Compiled in RELEASE mode.
# Debugger is not attached.
# Stopwatch is high resolution.  14318180 ticks per second.
# 
# 2008-Aug-18 20:50:11
# Microsoft Windows NT 6.0.6001 Service Pack 1
# CLR 2.0.50727.3053
# 
# See http://eval.apply.googlepages.com/ for further information.
# 
# TakTest, Version=1.0.3152.23099, Culture=neutral, PublicKeyToken=null
# Debugging Flags: IgnoreSymbolStoreSequencePoints
# JIT Optimizer enabled
# JIT Tracking disabled
I've put the code online if you want to play with it.

I changed the search engine at the top of the page to search a whole bunch of Lisp oriented web sites in addition to the Hyperspec.

Monday, August 11, 2008

Interpreter Overhead

Consider this client/server model of computing. The server is a general purpose virtual machine/runtime system. The client is the program which makes ‘requests’ to the server and receives responses. And let's be completely abstract here. The client is capable of nothing but making requests. The server does all the work. If the client needs temporary storage for a variable, it requests that the server allocate it. If it wants to load a value into that variable, it requests that the server load the location. The client isn't much more than an instruction stream, but keep the image of an active, but extraordinarily primitive client in your mind.

The server, on the other hand, is amazingly powerful compared to the client, but totally ignorant. It simply does not know what to do or what it is doing. It just mechanically follows the requests of the client.

This is simply a description of the Harvard architecture, but I want to emphasize that each component --- the program (client) and the processor (server) --- sees the other as a black box. And although the processor is usually responsible for fetching instructions, I've made the client responsible for feeding them to the processor.

A general purpose processing machine combined with a program gives you a special purpose machine. Your x86 processor is general purpose, but Excel turns it into a spreadsheet processor. We can consider a program as a ‘meta-operator’ that transforms one machine into another. Instead of making requests directly to the machine, we make them to the program/machine combination. The program makes requests of the underlying machine and arranges for the underlying machine to deliver the answer in a useful format.

If you have a program written in a language that your machine doesn't natively understand you have two options to make it run. You can translate the program to a language the machine does understand, or you can translate the machine to a machine that understands the program. The first technique is compiling, the second is interpretation. An interpreter is a program that converts a machine in one language to a machine in another.

Interpretation almost always performs slower than compilation. The usual reason is given as ‘interpreter overhead’. But what is interpreter overhead? If we look at the model I described above, we have this situation if we have compiled our program:
                    compiler
           program ---------> program'

               +--------------------
               | program' | machine
               +--------------------
But if it is interpreted, we have this situation:
           +---------------------------------
           |         +-----------------------
           | program | interpreter | machine
           |         +-----------------------
           +---------------------------------
Obviously the extra layer introduced by interpretation is a problem. But in our model, the client doesn't do anything. The machine is ultimately the thing that takes action. The client is essentially passive. The only reason that things would take longer is because the machine is doing more work in the second situation.
           +---------------------------------------
           |         +-----------------------------
           | program | interpreter | machine
           |         |             |  + excess work
           |         +-----------------------------
           +---------------------------------------
The interpreter must be making more requests of the machine than the compiled program. (I never miss the obvious.) That's the interpreter overhead.

Let's turn this around. Now we're take the point of view of the machine and we are getting instructions fed to us. The picture is like this:
            -------------------+
                       program | machine
            -------------------+
or perhaps it is like this:
                     +-------------+
           ----------+             |
            program  | interpreter | machine
           ----------+             |
                     +-------------+
Another picture:
                     +--------------+
           ----------+              |
            program  | interpreter  |   machine
           ----------+              |
                     +--------------+
                          |    |
                     +--------------+
                     | co processor |
                     +--------------+
I've attached an ‘interpreter co-processor’ to the picture. What does it do? It handles all the ‘interpreter overhead’. If the interpreter needs some temporary storage to hold a variable, or if it has to maintain a lookup table to find subroutines, the co-processor is the machine that handles that. On the other hand, if the interpreter needs to perform an addition because the user's program calls ‘+’, this is given directly to the machine. The co-processor is only there to help the interpreter, not to perform any work requested by the program.

From the machine's point of view, the only way we can tell if two programs are different is if the series of instructions being fed to us is different. We don't really know, or even care what is going on outside. We're just following orders, whether they come directly from a program or via an interpreter (which, after all, is just another program as far as we're concerned), or whether the interpreter has a funny co-processor attached.

Suppose we have two programs, X and Y, that have only a small difference between them. They both feed the exact same instructions in the exact same order to the machine, but program Y uses the interpreter co-processor half as much as program X. Since the instruction stream to the machine is identical in both, it should be the case that both programs ‘do’ the same thing. The machine is the only component that can take action, and you've told it to take the same action in both cases. But program Y uses fewer resources than program X. By making fewer demands of the co-processor, perhaps our machine can be cheaper or run cooler. The instructions that are given to the co-processor are not essential to running the program.

Our machine is universal, so we really don't need to buy a separate co-processor. We can emulate it.
                     +--------------+
           ----------+              +--+---------+
            program  | interpreter  |  | machine |
           ----------+              +--+--+---+--+
                     +--------------+     |   |
                          |    |          |   |
                     +--------------------+   |
                     | co processor emulator  |
                     +------------------------+
How does this differ from the picture that looked like this?
                     +-------------+
           ----------+             |
            program  | interpreter | machine
           ----------+             |
                     +-------------+
It doesn't, except that I can point at the instructions coming through the ‘emulated co-processor’ and identify them as ‘interpreter overhead’. And as I mentioned just above, those instructions aren't essential to running the program.

If we are not trying to ‘optimize’ the program, then a compiled program should be identical to an interpreted one, but without as much ‘interpreter overhead’. The overhead comes from the fact that we are emulating some of the behavior that the interpreter relies on. Compilation wins because we remove as much of the need for the emulation as we can. But some machines these days are pretty sophisticated. Do we need to emulate the desired behavior, or can we co-opt the machine's built-in behavior to approximate the desired behavior?

Saturday, August 9, 2008

Real life nastiness

Here's an example I stole from the support code for PLT Scheme. This function seems to check whether we've hit a symbol delimiter or comment.
int wxMediaStreamIn::IsDelim(char c)
{
  if (scheme_isspace((unsigned char)c))
    return 1;
  else if (c == '#') {
    long pos;
    char next[1];
    pos = f->Tell();
    f->Read(next, 1);
    if (next[0] == '|') {
      f->Seek(pos - 1);
      return 1;
    } else {
      f->Seek(pos);
      return 0;
    }
  } else if (c == ';') {
    long pos;
    pos = f->Tell();
    f->Seek(pos - 1);
    return 1;
  } else
    return 0;
}
This function might be skipping a token in the input stream. I'm not really sure.
void wxMediaStreamIn::SkipOne(int recur)
{
  char buf[1];

  if (recur) {
    buf[0] = '#';
  } else {
    SkipWhitespace(buf);
  }

  if (!bad) {
    if (buf[0] == '#') {
      /* Byte string */
      if (f->Read(buf, 1) == 1) {
 if (buf[0] != '"') {
   bad = 1;
   BAD_PRINTF(("bad 12\n"));
 } else {
   while (1) {
     if (f->Read(buf, 1) != 1) {
       bad = 1;
       BAD_PRINTF(("bad 13\n"));
       break;
     }
     if (buf[0] == '"') {
       break;
     } else if (buf[0] == '\\') {
       if (f->Read(buf, 1) != 1) {
  bad = 1;
  BAD_PRINTF(("bad 14\n"));
  break;
       }
     }
   }
 }
      } else {
 bad = 1;
 BAD_PRINTF(("bad 15\n"));
      }
    } else if (buf[0] == '(') {
      /* List of byte strings */
      while (!bad) {
 do {
   if (f->Read(buf, 1) != 1) {
     bad = 1;
     BAD_PRINTF(("bad 16\n"));
     break;
   }
 } while (!IsDelim(buf[0]));
 if (buf[0] == ')')
   break;
 else if (buf[0] == '#') {
   SkipOne(TRUE);
 } else {
   bad = 1;
   break;
 }
      }
    } else {
      /* Number */
      do {
 if (f->Read(buf, 1) != 1) {
   bad = 1;
   BAD_PRINTF(("bad 16\n"));
   break;
 }
      } while (!IsDelim(buf[0]));
    }

    if (!bad && !recur)
      IncItemCount();
  }
}
Now suppose we wanted to change this code by in-lining the calls to IsDelim. This is not a trivial task. In fact, the calls to BAD_PRINTF seem to indicate that the author of the code had no easy time getting to this point. The duplication of code blocks shows that some in-lining has already been done.

Now consider this code. It is part of a compiler. It establishes an escape continuation upon entry to a block, and if possible, optimizes the code if it can determine either that the continuation is not used or that the call to the continuation is at the same dynamic depth as the continuation.
  (define (bind-escape-continuation variable-name binder-type)
    (lambda (block receiver)
      ;; Grab a continuation to return to if necessary.
      (let* ((block (block/make block '()))
             (cont-var (binding/variable (variable/make&bind! block variable-name #f #f)))
             (result (receiver block)))
        (cond ((and (non-local-exit? result)
                    (eq? (non-local-exit/variable result) cont-var)
                    (not (variable/referenced? cont-var (non-local-exit/body result))))
               (evil-message "Optimize:  Coalesce catch/throw" cont-var)
               (non-local-exit/body result))
              ((not (variable/referenced? cont-var result))
               (evil-message "Optimize:  Remove binding for non-local-exit" cont-var)
               result)
              (else (make binder-type
                      :block    block
                      :variable cont-var
                      :body     result))))))
And this function compiles a for statement. An escape continuation is necessary to handle calls to break and continuue.
  (defmethod (compile-expression (target <void-target>) block (statement <c-for>))
    (cons-expression
     block
     (compile-expression (make <effect-target> :parent target) block (for/initialize statement))
     ((bind-escape-continuation 'break <compiled-bind-break>)
      block
      (lambda (break-block)
        (let* ((for-block (block/make break-block '()))
               (for-var   (binding/variable (variable/make&bind! block 'for #f #f))))
          (make-iteration
           (variable/name for-var)
           (make-sequence
            (list (make-conditional
                   (compile-expression (make <predicate-target> :parent target)
                                       for-block (for/predicate statement))
                   ((bind-escape-continuation 'continue <compiled-bind-continue>)
                    for-block
                    (lambda (continue-block)
                      (let ((continue-binding (block/lookup-name continue-block 'continue #f)))
                        ;; stow the step part.
                        (setf! (binding/value continue-binding)
                               (compile-expression (make <statement-target> :parent target)
                                                   break-block (for/step statement)))
                        (cons-expression continue-block
                                         (compile-expression (make <statement-target> :parent target)
                                                             continue-block (for/body statement))
                                         (make <non-local-exit>
                                           :block continue-block
                                           :variable (binding/variable continue-binding)
                                           :body (binding/value continue-binding))))))
                   (make <non-local-exit>
                     :block for-block
                     :variable (binding/variable
                                (block/lookup-name break-block 'break #f))
                     :body #f))
                  (make <compiled-combination>
                    :operator (make <compiled-reference> :variable for-var)
                    :operands '()))
            )))))))
Inlining the call to bind-escape-continuation is trivial. (Try it.) This isn't due to the problem domain unless character at a time parsing is that much more difficult than flow-control in a compiler.

One reason I chose Lisp or Scheme for experimental code is because I can move things around so much easier than in other languages. When I tinker with code I often decide to pull a chunk out, combine it with another, change my mind, put it back, change something to a function, move a binding, etc. Lisp simply does not get in my way as much as other languages.

Done testing?

The previous post looked good to me on Blogger and under Google reader. Anyone have problems with it?

Another test.

Some of the posts coming through a feed seem to be completely screwed up with this new stylesheet. I changed the stylesheet tag to use <pre> instead of a custom class and this seems to work on the blog itself. But when I went to the reader, one of the posts looked ok, but another looked wrong. I'm encouraged by the first, but maybe I'm being fooled by some cacheing that's going on somewhere. I'm going to make a few more test posts. If it works or fails, I'd like to know. (If you have hints, etc. It'd be nice, too.)
(define (iter tree))
     (car efficiently
  ;; wite-char #\space)))
     (if (> (cdr entrees)
 (displambda (term)
      (scand (pair? tree)
  (if (leaf-node? thing)
   (n (right-child tree)))
  (n (right-child tree tree)
  (if (leaf-node? tree tree)
    (probabilith the assumption that all-terms))))))

 (for-each-list-points terms an intep trees)
         (begin
        (define (do-term term)
     (display b)
   (let ((best-a alpha beta m N)
    (do ((i 0 (+ i 1))) (if (/ (* *alpha* (gamma-point dp)
      (gamma-ratio num 0)
   (if (> rl (d (right-child right-child)
     (do-it level tree)
         (set ((pi_k))
    (num den)
  ;; Compute gamma (n tree))
        (hash-tree (cluster-data-points terms tree)
  (d (left-child tree)))

Self-learning methodologies are particularly key when it comes to replicated theory. While conventional wisdom states that this issue is always surmounted by the understanding of congestion control, we believe that a different solution is necessary. Despite the fact that conventional wisdom states that this problem is always surmounted by the refinement of the UNIVAC computer, we believe that a different approach is necessary. We emphasize that CARPER requests XML, without investigating DNS. indeed, public-private key pairs and Moore's Law have a long history of connecting in this manner.

A test post.

I've gotten sick of the way blogger is showing my code. I'm playing with the stylesheet to see if it can be improved.
        void Run ()
        {
            // The etc argument isn't used.
            // We want EvalStep to be an expression rather
            // than a statement so we chain the return value
            // around in case we think of a use in the
            // future.
            object etc = null;
            while (true)
                etc = this.expression.EvalStep (this, etc);
        }
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.