Performance of virtionfs

[Peter-JanGootzen]

Currently with XLIO sequential write performance with bs=4k, iodepth=16, numjobs=1 is ~244MB/s, while sequential read only gives ~20MB/s.

• Investigate why this is happening (there are already some flamegraphs ran on commit 7fc9e18)
• Implement a fix

Furthermore blocksizes larger than the page size (4k) don't do anything (i.e. no performance increase).

• Investage why
• Impl fix

[Peter-JanGootzen]

Latency has now also regressed somehow.
statfs 50000 runs numactl pinned to right NUMA node and interrupt core
XLIO + libnfs -O2 used to be: 82usec
NFS: 48usec
Host<>DPU: 34usec

Now:
Without XLIO: 160usec
With XLIO + libnfs -O2: 137usec
So we lost 55usec somewhere!
Further testing:
With XLIO + libnfs -O3: 126usec
With XLIO + libnfs -O3 + 2 Virtio queues: 112usec
With XLIO + libnfs -O3 + 2 Virtio queues + DPU pinning to core 6,7: 100usec

--enable-latency-measuring
Reports that with the last configuration:
NFS: 68usec
Host<>DPU: 32usec

TODO:

• DPU core pinning = good gains
• Removing -g3 (shouldn't matter) = no change
• NFS remote side things (the latency measuring results indicate that NFS is the culprit)

[Peter-JanGootzen]

It would be interesting to test whether doing the NFS polling on the Virtio queue polling thread would improve performance. This requires a non-existent libnfs patch.

[Peter-JanGootzen]

• Try running the NFS remote on a newer server (flex21?)

[Peter-JanGootzen]

TODO disable c-states p-states