A guide to benchmarking nbdkit
==============================


General comments
----------------

* The plugin matters!  Different plugins have completely different
  uses, implementations and threading models.  There is little point
  in talking generically about “the performance of nbdkit” without
  mentioning what plugin you are testing.

* The client matters!  Does the client support multi-conn?  Does the
  client use the oldstyle or newstyle protocol?  Has the client been
  written with performance in mind?  The currently best clients are
  (a) the Linux kernel (nbd.ko), (b) qemu, and (c) fio.  Make sure you
  are using recent versions and have multi-conn enabled.

* Filters affect performance.  When benchmarking you should never use
  filters unless filters are what you are trying to benchmark.


Testing using fio
-----------------

FIO is a Flexible I/O tester written by Jens Axboe, and it is the
primary tool used for generating the load to test filesystems and
block devices.

(1) Install libnbd.

(2) Clone and compile fio:

    https://github.com/axboe/fio

using:

    ./configure --enable-libnbd

(3) Edit the test file in examples/nbd.fio, if required.

(4) Run nbdkit and fio together.  From the fio source directory:

    rm -f /tmp/socket
    nbdkit -f -U /tmp/socket null 1G --run './fio examples/nbd.fio'

If you want to use nbdkit from the source directory too, change
‘nbdkit’ to the path of the wrapper, eg:

    rm -f /tmp/socket
    ../nbdkit/nbdkit -f -U /tmp/socket null 1G --run './fio examples/nbd.fio'


Variations
----------

* Try adjusting the number of fio jobs (threads).

* Try adjusting the number of nbdkit threads (nbdkit -t option).

* Use other plugins.  Both nbdkit-memory-plugin and nbdkit-file-plugin
  are important ones to test.

* Run nbdkit under perf:

  perf record -a -g --call-graph=dwarf -- \
      server/nbdkit  -f -U /tmp/socket \
          ./plugins/null/.libs/nbdkit-null-plugin.so 1G


Testing using the Linux kernel client
-------------------------------------

Step (1) is the same as above - obtain or compile fio.

(2) Create the fio configuation file.

Create /var/tmp/test.fio containing:

----------------------------------------------------------------------
[test]
rw=randrw
size=64m
directory=/var/tmp/nbd
ioengine=libaio
iodepth=4
direct=1
numjobs=8
group_reporting
time_based
runtime=120
----------------------------------------------------------------------

(3) Run nbdkit.

From the nbdkit source directory:

rm -f /tmp/socket
./nbdkit -f -U /tmp/socket memory 1G

(4) Loop mount the NBD server:

modprobe nbd
nbd-client -C 8 -b 512 -unix /tmp/socket /dev/nbd0
mkfs.xfs -f /dev/nbd0
mkdir /var/tmp/nbd
mount /dev/nbd0 /var/tmp/nbd

(5) Run the fio test:

fio /var/tmp/test.fio
