View source for The Fastest Checksum


<br/>
<b>Question: Which checksum algorithm is the fastest?

<pre>Answer: the System V sum algorithm using 512 byte blocks.</pre></b>
<br/>

Use it by running <code class="C">sum -s filename</code>. The <code>-s</code> switch tells it to use the System V algorithm instead of the default BSD algorithm.

It can also read from standard input like <code class="C">cat filename | sum -s -</code> (the hyphen at the end tells sum to read from stdin).
<br/><br/>

<b>Here's how it compares to other algorithms:</b>

I chose a relatively large file and put it in my <i>tmpfs</i> to eliminate the hard drive bottleneck.

<pre class="C">
winter@wintermute:/run$ ls -lh linuxmint-17-mate-64bit-v2.iso
-rwxr-xr-x 1 root root 1.4G Apr  5 08:57 linuxmint-17-mate-64bit-v2.iso

winter@wintermute:/run$ time sha1sum linuxmint-17-mate-64bit-v2.iso
7ea275072dc77e8006df879105dee9eb3b8ecb53  linuxmint-17-mate-64bit-v2.iso

real    0m4.869s
user    0m4.689s
sys     0m0.176s

winter@wintermute:/run$ time md5sum linuxmint-17-mate-64bit-v2.iso
215edb6ab70d2250cae99b6f02dce45a  linuxmint-17-mate-64bit-v2.iso

real    0m2.670s
user    0m2.479s
sys     0m0.192s

winter@wintermute:/run$ time crc32 linuxmint-17-mate-64bit-v2.iso
5bb66ab6

real    0m1.752s
user    0m1.578s
sys     0m0.172s

winter@wintermute:/run$ time sum linuxmint-17-mate-64bit-v2.iso
07152 1431936

real    0m2.577s
user    0m2.363s
sys     0m0.212s

winter@wintermute:/run$ time sum -s linuxmint-17-mate-64bit-v2.iso
61984 2863872 linuxmint-17-mate-64bit-v2.iso

real    0m0.767s
user    0m0.587s
sys     0m0.180s

winter@wintermute:/run$ sum --version|head -1
sum (GNU coreutils) 8.21

winter@wintermute:/run$ lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                4
On-line CPU(s) list:   0-3
Thread(s) per core:    1
Core(s) per socket:    4
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 42
Stepping:              7
CPU MHz:               1638.656
BogoMIPS:              6600.22
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              6144K
NUMA node0 CPU(s):     0-3

winter@wintermute:/run$ sudo lshw -short -C memory|grep MHz
/0/f/0                    memory         8GiB DIMM DDR3 Synchronous 1333 MHz (0.8 ns)
/0/f/1                    memory         4GiB DIMM DDR3 Synchronous 1333 MHz (0.8 ns)
/0/f/2                    memory         8GiB DIMM DDR3 Synchronous 1333 MHz (0.8 ns)
</pre>

I've run this test several times and the results are very consistent.