AMD EPYC Genoa-X Linux 6.9 Kernel Benchmarks

Benchmarks by Michael Larabel for a future article.

HTML result view exported from: https://openbenchmarking.org/result/2403282-NE-AMDEPYCGE98&sor.

AMD EPYC Genoa-X Linux 6.9 Kernel BenchmarksProcessorMotherboardChipsetMemoryDiskGraphicsNetworkOSKernelCompilerFile-SystemScreen ResolutionLinux 6.8Linux 6.9 27 Mar2 x AMD EPYC 9684X 96-Core @ 2.55GHz (192 Cores / 384 Threads)AMD Titanite_4G (RTI1007B BIOS)AMD Device 14a41520GB3201GB Micron_7450_MTFDKCB3T2TFS + 257GB Flash DriveASPEEDBroadcom NetXtreme BCM5720 PCIeUbuntu 23.106.8.0-060800-generic (x86_64)GCC 13.2.0ext41920x12006.9.0-060900rc1daily20240327-generic (x86_64)OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa10113ePython Details- Python 3.11.6Security Details- Linux 6.8: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of Safe RET + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - Linux 6.9 27 Mar: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of Safe RET + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

AMD EPYC Genoa-X Linux 6.9 Kernel Benchmarkssockperf: Throughputsockperf: Latency Under Loadquicksilver: CORAL2 P1quicksilver: CORAL2 P2namd: ATPase with 327,506 Atomsnamd: STMV with 1,066,628 Atomssvt-av1: Preset 4 - Bosphorus 4Ksvt-av1: Preset 8 - Bosphorus 4Ksvt-av1: Preset 12 - Bosphorus 4Ksvt-av1: Preset 13 - Bosphorus 4Kcompress-7zip: Compression Ratingcompress-7zip: Decompression Ratingbuild-godot: Time To Compilebuild-linux-kernel: defconfigbuild-linux-kernel: allmodconfigbuild-llvm: Ninjaospray-studio: 1 - 4K - 1 - Path Tracer - CPUospray-studio: 2 - 4K - 1 - Path Tracer - CPUospray-studio: 3 - 4K - 1 - Path Tracer - CPUospray-studio: 1 - 4K - 32 - Path Tracer - CPUospray-studio: 2 - 4K - 32 - Path Tracer - CPUospray-studio: 3 - 4K - 32 - Path Tracer - CPUhackbench: 32 - Processopenssl: SHA256openssl: SHA512openssl: RSA4096openssl: RSA4096openssl: ChaCha20openssl: AES-128-GCMopenssl: AES-256-GCMopenssl: ChaCha20-Poly1305mysqlslap: 1024tensorflow: CPU - 256 - ResNet-50tensorflow: CPU - 512 - ResNet-50stress-ng: NUMAstress-ng: Pthreadstress-ng: Semaphoresstress-ng: Matrix Mathstress-ng: Vector Mathstress-ng: AVX-512 VNNIstress-ng: Context Switchingstress-ng: System V Message Passingopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Face Detection Retail FP16-INT8 - CPUopenvino: Face Detection Retail FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Noise Suppression Poconet-Like FP16 - CPUopenvino: Noise Suppression Poconet-Like FP16 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUrocksdb: Overwriterocksdb: Rand Readrocksdb: Update Randrocksdb: Read While Writingrocksdb: Read Rand Write Randspeedb: Rand Readspeedb: Update Randspeedb: Read Rand Write Randllamafile: llava-v1.5-7b-q4 - CPUllamafile: mistral-7b-instruct-v0.2.Q8_0 - CPULinux 6.8Linux 6.9 27 Mar31938616.356177133331587333320.753826.460628.43186.617159.513157.918833319135735187.48226.914198.99086.39955856365917788178342098410.8362843498040939201557317797844.23232989.310955646645572003432456830173598466780377716398799351117.77136.82646.3458297.64264112681.09915179.071306706.1918968297.9440075791.1325534145.8711218.124.2634785.755.503864.6512.401086.4844.137703.0323.595765.9133.2538152310472908633757142066532431825721084511886293746160389112.0312.7932402916.109179033331629444420.684416.389038.56588.143164.308159.118821061134562187.39226.880196.88186.71955956365517746178772091410.4252839713830209131561355797661.73230787.610931887499472001406545920173204881171377476188128752119.29136.37691.5459057.15292541552.77937042.031349057.5119594674.6741854298.6828384870.1511719.994.0835740.915.354063.1111.801119.5642.827762.0923.505898.0632.4341686010960819514062582241814034848771112167042313791157210921.1013.46OpenBenchmarking.org

Sockperf

Test: Throughput

OpenBenchmarking.orgMessages Per Second, More Is BetterSockperf 3.7Test: ThroughputLinux 6.9 27 MarLinux 6.870K140K210K280K350KSE +/- 3244.28, N = 6SE +/- 2015.52, N = 53240293193861. (CXX) g++ options: --param -O3 -rdynamic

Sockperf

Test: Latency Under Load

OpenBenchmarking.orgusec, Fewer Is BetterSockperf 3.7Test: Latency Under LoadLinux 6.9 27 MarLinux 6.848121620SE +/- 0.12, N = 24SE +/- 0.13, N = 1016.1116.361. (CXX) g++ options: --param -O3 -rdynamic

Quicksilver

Input: CORAL2 P1

OpenBenchmarking.orgFigure Of Merit, More Is BetterQuicksilver 20230818Input: CORAL2 P1Linux 6.9 27 MarLinux 6.84M8M12M16M20MSE +/- 156027.06, N = 3SE +/- 17638.34, N = 317903333177133331. (CXX) g++ options: -fopenmp -O3 -march=native

Quicksilver

Input: CORAL2 P2

OpenBenchmarking.orgFigure Of Merit, More Is BetterQuicksilver 20230818Input: CORAL2 P2Linux 6.9 27 MarLinux 6.83M6M9M12M15MSE +/- 272810.02, N = 9SE +/- 81921.37, N = 316294444158733331. (CXX) g++ options: -fopenmp -O3 -march=native

NAMD

Input: ATPase with 327,506 Atoms

OpenBenchmarking.orgns/day, More Is BetterNAMD 3.0b6Input: ATPase with 327,506 AtomsLinux 6.8Linux 6.9 27 Mar510152025SE +/- 0.03, N = 3SE +/- 0.07, N = 320.7520.68

NAMD

Input: STMV with 1,066,628 Atoms

OpenBenchmarking.orgns/day, More Is BetterNAMD 3.0b6Input: STMV with 1,066,628 AtomsLinux 6.8Linux 6.9 27 Mar246810SE +/- 0.00857, N = 3SE +/- 0.00170, N = 36.460626.38903

SVT-AV1

Encoder Mode: Preset 4 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 4 - Input: Bosphorus 4KLinux 6.9 27 MarLinux 6.8246810SE +/- 0.054, N = 3SE +/- 0.052, N = 38.5658.4311. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

SVT-AV1

Encoder Mode: Preset 8 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 8 - Input: Bosphorus 4KLinux 6.9 27 MarLinux 6.820406080100SE +/- 0.66, N = 15SE +/- 0.60, N = 1588.1486.621. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

SVT-AV1

Encoder Mode: Preset 12 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 12 - Input: Bosphorus 4KLinux 6.9 27 MarLinux 6.84080120160200SE +/- 0.71, N = 3SE +/- 1.75, N = 3164.31159.511. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

SVT-AV1

Encoder Mode: Preset 13 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 13 - Input: Bosphorus 4KLinux 6.9 27 MarLinux 6.84080120160200SE +/- 1.96, N = 15SE +/- 1.80, N = 15159.12157.921. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

7-Zip Compression

Test: Compression Rating

OpenBenchmarking.orgMIPS, More Is Better7-Zip Compression 22.01Test: Compression RatingLinux 6.8Linux 6.9 27 Mar200K400K600K800K1000KSE +/- 3228.95, N = 3SE +/- 6757.68, N = 38333198210611. (CXX) g++ options: -lpthread -ldl -O2 -fPIC

7-Zip Compression

Test: Decompression Rating

OpenBenchmarking.orgMIPS, More Is Better7-Zip Compression 22.01Test: Decompression RatingLinux 6.8Linux 6.9 27 Mar300K600K900K1200K1500KSE +/- 3867.90, N = 3SE +/- 9219.74, N = 3135735113456211. (CXX) g++ options: -lpthread -ldl -O2 -fPIC

Timed Godot Game Engine Compilation

Time To Compile

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Godot Game Engine Compilation 4.0Time To CompileLinux 6.9 27 MarLinux 6.820406080100SE +/- 0.09, N = 3SE +/- 0.07, N = 387.3987.48

Timed Linux Kernel Compilation

Build: defconfig

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Linux Kernel Compilation 6.8Build: defconfigLinux 6.9 27 MarLinux 6.8612182430SE +/- 0.22, N = 8SE +/- 0.16, N = 1426.8826.91

Timed Linux Kernel Compilation

Build: allmodconfig

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Linux Kernel Compilation 6.8Build: allmodconfigLinux 6.9 27 MarLinux 6.84080120160200SE +/- 0.54, N = 3SE +/- 1.25, N = 3196.88198.99

Timed LLVM Compilation

Build System: Ninja

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed LLVM Compilation 16.0Build System: NinjaLinux 6.8Linux 6.9 27 Mar20406080100SE +/- 0.59, N = 3SE +/- 1.00, N = 386.4086.72

OSPRay Studio

Camera: 1 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 1 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPULinux 6.8Linux 6.9 27 Mar120240360480600SE +/- 1.00, N = 3SE +/- 1.15, N = 3558559

OSPRay Studio

Camera: 2 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 2 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPULinux 6.8Linux 6.9 27 Mar120240360480600SE +/- 1.33, N = 3SE +/- 1.53, N = 3563563

OSPRay Studio

Camera: 3 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 3 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPULinux 6.9 27 MarLinux 6.8140280420560700SE +/- 0.58, N = 3SE +/- 3.28, N = 3655659

OSPRay Studio

Camera: 1 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 1 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPULinux 6.9 27 MarLinux 6.84K8K12K16K20KSE +/- 64.98, N = 3SE +/- 60.00, N = 31774617788

OSPRay Studio

Camera: 2 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 2 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPULinux 6.8Linux 6.9 27 Mar4K8K12K16K20KSE +/- 30.37, N = 3SE +/- 68.53, N = 31783417877

OSPRay Studio

Camera: 3 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 3 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPULinux 6.9 27 MarLinux 6.84K8K12K16K20KSE +/- 48.89, N = 3SE +/- 40.08, N = 32091420984

Hackbench

Count: 32 - Type: Process

OpenBenchmarking.orgSeconds, Fewer Is BetterHackbenchCount: 32 - Type: ProcessLinux 6.9 27 MarLinux 6.83691215SE +/- 0.14, N = 3SE +/- 0.13, N = 410.4310.841. (CC) gcc options: -lpthread

OpenSSL

Algorithm: SHA256

OpenBenchmarking.orgbyte/s, More Is BetterOpenSSL 3.1Algorithm: SHA256Linux 6.8Linux 6.9 27 Mar60000M120000M180000M240000M300000MSE +/- 6021947.24, N = 3SE +/- 91878369.01, N = 32843498040932839713830201. (CC) gcc options: -pthread -m64 -O3 -ldl

OpenSSL

Algorithm: SHA512

OpenBenchmarking.orgbyte/s, More Is BetterOpenSSL 3.1Algorithm: SHA512Linux 6.8Linux 6.9 27 Mar20000M40000M60000M80000M100000MSE +/- 24428915.02, N = 3SE +/- 200003199.85, N = 392015573177913156135571. (CC) gcc options: -pthread -m64 -O3 -ldl

OpenSSL

Algorithm: RSA4096

OpenBenchmarking.orgsign/s, More Is BetterOpenSSL 3.1Algorithm: RSA4096Linux 6.8Linux 6.9 27 Mar20K40K60K80K100KSE +/- 54.75, N = 3SE +/- 73.99, N = 397844.297661.71. (CC) gcc options: -pthread -m64 -O3 -ldl

OpenSSL

Algorithm: RSA4096

OpenBenchmarking.orgverify/s, More Is BetterOpenSSL 3.1Algorithm: RSA4096Linux 6.8Linux 6.9 27 Mar700K1400K2100K2800K3500KSE +/- 1592.51, N = 3SE +/- 1390.93, N = 33232989.33230787.61. (CC) gcc options: -pthread -m64 -O3 -ldl

OpenSSL

Algorithm: ChaCha20

OpenBenchmarking.orgbyte/s, More Is BetterOpenSSL 3.1Algorithm: ChaCha20Linux 6.8Linux 6.9 27 Mar200000M400000M600000M800000M1000000MSE +/- 5401967811.92, N = 3SE +/- 4778305950.15, N = 3109556466455710931887499471. (CC) gcc options: -pthread -m64 -O3 -ldl

OpenSSL

Algorithm: AES-128-GCM

OpenBenchmarking.orgbyte/s, More Is BetterOpenSSL 3.1Algorithm: AES-128-GCMLinux 6.8Linux 6.9 27 Mar400000M800000M1200000M1600000M2000000MSE +/- 5046453117.89, N = 3SE +/- 483814569.81, N = 3200343245683020014065459201. (CC) gcc options: -pthread -m64 -O3 -ldl

OpenSSL

Algorithm: AES-256-GCM

OpenBenchmarking.orgbyte/s, More Is BetterOpenSSL 3.1Algorithm: AES-256-GCMLinux 6.8Linux 6.9 27 Mar400000M800000M1200000M1600000M2000000MSE +/- 826670837.74, N = 3SE +/- 1466705411.41, N = 3173598466780317320488117131. (CC) gcc options: -pthread -m64 -O3 -ldl

OpenSSL

Algorithm: ChaCha20-Poly1305

OpenBenchmarking.orgbyte/s, More Is BetterOpenSSL 3.1Algorithm: ChaCha20-Poly1305Linux 6.8Linux 6.9 27 Mar170000M340000M510000M680000M850000MSE +/- 372177165.42, N = 3SE +/- 709551963.95, N = 37771639879937747618812871. (CC) gcc options: -pthread -m64 -O3 -ldl

MariaDB mariadb-slap

Clients: 1024

OpenBenchmarking.orgQueries Per Second, More Is BetterMariaDB mariadb-slap 11.5Clients: 1024Linux 6.9 27 MarLinux 6.81224364860SE +/- 0.26, N = 3SE +/- 0.46, N = 352511. (CXX) g++ options: -fPIC -pie -fstack-protector -O3 -shared -lrt -lpthread -lz -ldl -lm -lstdc++

TensorFlow

Device: CPU - Batch Size: 256 - Model: ResNet-50

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.16.1Device: CPU - Batch Size: 256 - Model: ResNet-50Linux 6.9 27 MarLinux 6.8306090120150SE +/- 0.86, N = 3SE +/- 0.34, N = 3119.29117.77

TensorFlow

Device: CPU - Batch Size: 512 - Model: ResNet-50

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.16.1Device: CPU - Batch Size: 512 - Model: ResNet-50Linux 6.8Linux 6.9 27 Mar306090120150SE +/- 0.10, N = 3SE +/- 0.14, N = 3136.82136.37

Stress-NG

Test: NUMA

OpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.16.04Test: NUMALinux 6.9 27 MarLinux 6.8150300450600750SE +/- 0.95, N = 3SE +/- 2.59, N = 3691.54646.341. (CXX) g++ options: -O2 -std=gnu99 -lc

Stress-NG

Test: Pthread

OpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.16.04Test: PthreadLinux 6.9 27 MarLinux 6.813K26K39K52K65KSE +/- 308.02, N = 2SE +/- 649.26, N = 359057.1558297.641. (CXX) g++ options: -O2 -std=gnu99 -lc

Stress-NG

Test: Semaphores

OpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.16.04Test: SemaphoresLinux 6.9 27 MarLinux 6.860M120M180M240M300MSE +/- 1842285.12, N = 3SE +/- 1107000.14, N = 3292541552.77264112681.091. (CXX) g++ options: -O2 -std=gnu99 -lc

Stress-NG

Test: Matrix Math

OpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.16.04Test: Matrix MathLinux 6.9 27 MarLinux 6.8200K400K600K800K1000KSE +/- 4156.75, N = 3SE +/- 3353.49, N = 3937042.03915179.071. (CXX) g++ options: -O2 -std=gnu99 -lc

Stress-NG

Test: Vector Math

OpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.16.04Test: Vector MathLinux 6.9 27 MarLinux 6.8300K600K900K1200K1500KSE +/- 1326.11, N = 3SE +/- 1312.02, N = 31349057.511306706.191. (CXX) g++ options: -O2 -std=gnu99 -lc

Stress-NG

Test: AVX-512 VNNI

OpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.16.04Test: AVX-512 VNNILinux 6.9 27 MarLinux 6.84M8M12M16M20MSE +/- 25357.28, N = 3SE +/- 19220.66, N = 319594674.6718968297.941. (CXX) g++ options: -O2 -std=gnu99 -lc

Stress-NG

Test: Context Switching

OpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.16.04Test: Context SwitchingLinux 6.9 27 MarLinux 6.89M18M27M36M45MSE +/- 110197.66, N = 3SE +/- 130933.36, N = 341854298.6840075791.131. (CXX) g++ options: -O2 -std=gnu99 -lc

Stress-NG

Test: System V Message Passing

OpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.16.04Test: System V Message PassingLinux 6.9 27 MarLinux 6.86M12M18M24M30MSE +/- 9139.64, N = 3SE +/- 770160.74, N = 1228384870.1525534145.871. (CXX) g++ options: -O2 -std=gnu99 -lc

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16-INT8 - Device: CPULinux 6.9 27 MarLinux 6.83K6K9K12K15KSE +/- 24.42, N = 3SE +/- 5.76, N = 311719.9911218.121. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16-INT8 - Device: CPULinux 6.9 27 MarLinux 6.80.95851.9172.87553.8344.7925SE +/- 0.01, N = 3SE +/- 0.00, N = 34.084.26MIN: 3.64 / MAX: 20.85MIN: 3.76 / MAX: 20.91. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Face Detection Retail FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16-INT8 - Device: CPULinux 6.9 27 MarLinux 6.88K16K24K32K40KSE +/- 83.35, N = 3SE +/- 19.55, N = 335740.9134785.751. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Face Detection Retail FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16-INT8 - Device: CPULinux 6.9 27 MarLinux 6.81.23752.4753.71254.956.1875SE +/- 0.01, N = 3SE +/- 0.00, N = 35.355.50MIN: 4.84 / MAX: 24.91MIN: 4.83 / MAX: 271. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Road Segmentation ADAS FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16-INT8 - Device: CPULinux 6.9 27 MarLinux 6.89001800270036004500SE +/- 3.68, N = 3SE +/- 1.98, N = 34063.113864.651. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Road Segmentation ADAS FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16-INT8 - Device: CPULinux 6.9 27 MarLinux 6.83691215SE +/- 0.01, N = 3SE +/- 0.01, N = 311.8012.40MIN: 11.09 / MAX: 37.4MIN: 11.56 / MAX: 38.831. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Machine Translation EN To DE FP16 - Device: CPULinux 6.9 27 MarLinux 6.82004006008001000SE +/- 0.57, N = 3SE +/- 1.18, N = 31119.561086.481. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Machine Translation EN To DE FP16 - Device: CPULinux 6.9 27 MarLinux 6.81020304050SE +/- 0.02, N = 3SE +/- 0.05, N = 342.8244.13MIN: 34.68 / MAX: 192.44MIN: 36.04 / MAX: 216.031. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Noise Suppression Poconet-Like FP16 - Device: CPULinux 6.9 27 MarLinux 6.817003400510068008500SE +/- 55.96, N = 3SE +/- 90.39, N = 37762.097703.031. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Noise Suppression Poconet-Like FP16 - Device: CPULinux 6.9 27 MarLinux 6.8612182430SE +/- 0.26, N = 3SE +/- 0.34, N = 323.5023.59MIN: 9.59 / MAX: 65.24MIN: 9.67 / MAX: 85.051. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Handwritten English Recognition FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16-INT8 - Device: CPULinux 6.9 27 MarLinux 6.813002600390052006500SE +/- 26.57, N = 3SE +/- 9.86, N = 35898.065765.911. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Handwritten English Recognition FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16-INT8 - Device: CPULinux 6.9 27 MarLinux 6.8816243240SE +/- 0.13, N = 3SE +/- 0.05, N = 332.4333.25MIN: 29.07 / MAX: 50.08MIN: 29.55 / MAX: 52.691. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

RocksDB

Test: Overwrite

OpenBenchmarking.orgOp/s, More Is BetterRocksDB 9.0Test: OverwriteLinux 6.9 27 MarLinux 6.890K180K270K360K450KSE +/- 1320.84, N = 3SE +/- 180.51, N = 34168603815231. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti -lpthread

RocksDB

Test: Random Read

OpenBenchmarking.orgOp/s, More Is BetterRocksDB 9.0Test: Random ReadLinux 6.9 27 MarLinux 6.8200M400M600M800M1000MSE +/- 2941599.90, N = 3SE +/- 4458137.36, N = 3109608195110472908631. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti -lpthread

RocksDB

Test: Update Random

OpenBenchmarking.orgOp/s, More Is BetterRocksDB 9.0Test: Update RandomLinux 6.9 27 MarLinux 6.890K180K270K360K450KSE +/- 1155.95, N = 3SE +/- 379.66, N = 34062583757141. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti -lpthread

RocksDB

Test: Read While Writing

OpenBenchmarking.orgOp/s, More Is BetterRocksDB 9.0Test: Read While WritingLinux 6.9 27 MarLinux 6.85M10M15M20M25MSE +/- 390827.28, N = 12SE +/- 155882.11, N = 322418140206653241. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti -lpthread

RocksDB

Test: Read Random Write Random

OpenBenchmarking.orgOp/s, More Is BetterRocksDB 9.0Test: Read Random Write RandomLinux 6.9 27 MarLinux 6.8700K1400K2100K2800K3500KSE +/- 3987.34, N = 3SE +/- 3761.31, N = 3348487731825721. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti -lpthread

Speedb

Test: Random Read

OpenBenchmarking.orgOp/s, More Is BetterSpeedb 2.7Test: Random ReadLinux 6.9 27 MarLinux 6.8200M400M600M800M1000MSE +/- 848602.66, N = 3SE +/- 5405476.66, N = 3111216704210845118861. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti -lpthread

Speedb

Test: Update Random

OpenBenchmarking.orgOp/s, More Is BetterSpeedb 2.7Test: Update RandomLinux 6.9 27 MarLinux 6.870K140K210K280K350KSE +/- 2172.75, N = 3SE +/- 945.90, N = 33137912937461. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti -lpthread

Speedb

Test: Read Random Write Random

OpenBenchmarking.orgOp/s, More Is BetterSpeedb 2.7Test: Read Random Write RandomLinux 6.8Linux 6.9 27 Mar300K600K900K1200K1500KSE +/- 2371.05, N = 3SE +/- 7976.44, N = 3160389115721091. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti -lpthread

Llamafile

Test: llava-v1.5-7b-q4 - Acceleration: CPU

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.6Test: llava-v1.5-7b-q4 - Acceleration: CPULinux 6.9 27 MarLinux 6.8510152025SE +/- 0.49, N = 15SE +/- 0.42, N = 1521.1012.03

Llamafile

Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.6Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPULinux 6.9 27 MarLinux 6.83691215SE +/- 0.04, N = 3SE +/- 0.04, N = 313.4612.79

Phoronix Test Suite v10.8.5