benchmark

benchmark
    n 1: a standard by which something can be measured or judged;
         "his painting sets the benchmark of quality"
    2: a surveyor's mark on a permanent object of predetermined
       position and elevation used as a reference point [syn:
       {benchmark}, {bench mark}]
    

benchmark \benchmark\, bench mark \bench mark\ (Surveying)
   1. Any permanent mark to which other levels may be referred.
      such as:
      (a) A horizontal mark at the water's edge with reference
          to which the height of tides and floods may be
          measured.
      (b) a surveyer's mark on a permanent object of
          predetermined position and elevation used as a
          reference point.
          [Webster 1913 Suppl. + WordNet 1.5]

   2. something serving as a standard by which related items may
      be judged; as, his painting sets the benchmark of quality.
      [PJC + WordNet 1.5]

   3. a test or series of tests designed to compare the
      qualities or performance of different devices of the same
      type. Certain sets of computer programs are much used as
      benchmarks for comparing the performance of different
      computers, especially by comparing the time it takes to
      complete a test.
      [PJC]
    

benchmark
 n.

   [techspeak] An inaccurate measure of computer performance. "In the
   computer industry, there are three kinds of lies: lies, damn lies, and
   benchmarks." Well-known ones include Whetstone, Dhrystone, Rhealstone
   (see {h}), the Gabriel LISP benchmarks, the SPECmark suite, and
   LINPACK. See also {machoflops}, {MIPS}, {smoke and mirrors}.
    

benchmark

   <benchmark> A standard program or set of programs which can be
   run on different computers to give an inaccurate measure of
   their performance.

   "In the computer industry, there are three kinds of lies:
   lies, damn lies, and benchmarks."

   A benchmark may attempt to indicate the overall power of a
   system by including a "typical" mixture of programs or it may
   attempt to measure more specific aspects of performance, like
   graphics, I/O or computation (integer or {floating-point}).
   Others measure specific tasks like {rendering} polygons,
   reading and writing files or performing operations on
   matrices.  The most useful kind of benchmark is one which is
   tailored to a user's own typical tasks.  While no one
   benchmark can fully characterise overall system performance,
   the results of a variety of realistic benchmarks can give
   valuable insight into expected real performance.

   Benchmarks should be carefully interpreted, you should know
   exactly which benchmark was run (name, version); exactly what
   configuration was it run on (CPU, memory, compiler options,
   single user/multi-user, peripherals, network); how does the
   benchmark relate to your workload?

   Well-known benchmarks include {Whetstone}, {Dhrystone},
   {Rhealstone} (see {h}), the {Gabriel benchmarks} for {Lisp},
   the {SPECmark} suite, and {LINPACK}.

   See also {machoflops}, {MIPS}, {smoke and mirrors}.

   Usenet newsgroup: news:comp.benchmarks.

   Tennessee BenchWeb (http://netlib.org/benchweb/).

   [{Jargon File}]

   (2002-03-26)