Beard at 06:03, 8 August 2018

2018-08-08T06:03:39Z

New page

Question: Which checksum algorithm is the fastest?

<pre>Answer: the System V sum algorithm using 512 byte blocks.</pre>
 

Use it by running <code class="C">sum -s filename</code>. The <code>-s</code> switch tells it to use the System V algorithm instead of the default BSD algorithm.

It can also read from standard input like <code class="C">cat filename | sum -s -</code> (the hyphen at the end tells sum to read from stdin).
 

Here's how it compares to other algorithms:

I chose a relatively large file and put it in my tmpfs to eliminate the hard drive bottleneck.

<pre class="C">
winter@wintermute:/run$ ls -lh linuxmint-17-mate-64bit-v2.iso
-rwxr-xr-x 1 root root 1.4G Apr 5 08:57 linuxmint-17-mate-64bit-v2.iso

winter@wintermute:/run$ time sha1sum linuxmint-17-mate-64bit-v2.iso
7ea275072dc77e8006df879105dee9eb3b8ecb53 linuxmint-17-mate-64bit-v2.iso

real 0m4.869s
user 0m4.689s
sys 0m0.176s

winter@wintermute:/run$ time md5sum linuxmint-17-mate-64bit-v2.iso
215edb6ab70d2250cae99b6f02dce45a linuxmint-17-mate-64bit-v2.iso

real 0m2.670s
user 0m2.479s
sys 0m0.192s

winter@wintermute:/run$ time crc32 linuxmint-17-mate-64bit-v2.iso
5bb66ab6

real 0m1.752s
user 0m1.578s
sys 0m0.172s

winter@wintermute:/run$ time sum linuxmint-17-mate-64bit-v2.iso
07152 1431936

real 0m2.577s
user 0m2.363s
sys 0m0.212s

winter@wintermute:/run$ time sum -s linuxmint-17-mate-64bit-v2.iso
61984 2863872 linuxmint-17-mate-64bit-v2.iso

real 0m0.767s
user 0m0.587s
sys 0m0.180s

winter@wintermute:/run$ sum --version|head -1
sum (GNU coreutils) 8.21

winter@wintermute:/run$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 42
Stepping: 7
CPU MHz: 1638.656
BogoMIPS: 6600.22
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 6144K
NUMA node0 CPU(s): 0-3

winter@wintermute:/run$ sudo lshw -short -C memory|grep MHz
/0/f/0 memory 8GiB DIMM DDR3 Synchronous 1333 MHz (0.8 ns)
/0/f/1 memory 4GiB DIMM DDR3 Synchronous 1333 MHz (0.8 ns)
/0/f/2 memory 8GiB DIMM DDR3 Synchronous 1333 MHz (0.8 ns)
</pre>

I've run this test several times and the results are very consistent.

The Fastest Checksum - Revision history

Beard at 06:03, 8 August 2018