df2888

Intel Xeon E E-2488 testing with a Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS) and llvmpipe on Ubuntu 22.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2403242-NE-DF288805368&grs&sor.

df2888ProcessorMotherboardChipsetMemoryDiskGraphicsMonitorNetworkOSKernelDesktopDisplay ServerVulkanCompilerFile-SystemScreen ResolutionabcIntel Xeon E E-2488 @ 3.20GHz (8 Cores / 16 Threads)Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS)Intel Device 7a272 x 32GB DRAM-4400MT/s Micron MTC20C2085S1EC48BA1960GB Micron_7450_MTFDKBA960TFR + 3201GB Micron_7450_MTFDKCC3T2TFSllvmpipeVE2282 x Intel I210Ubuntu 22.046.2.0-26-generic (x86_64)GNOME Shell 42.9X Server 1.21.1.41.3.238GCC 11.4.0ext41920x1080OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0x11f - Thermald 2.4.9 Python Details- Python 3.10.12Security Details- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

df2888openvino: Road Segmentation ADAS FP16 - CPUopenvino: Road Segmentation ADAS FP16 - CPUpgbench: 1 - 800 - Read Onlypgbench: 1 - 800 - Read Only - Average Latencysvt-vp9: Visual Quality Optimized - Bosphorus 4Ksvt-av1: Preset 12 - Bosphorus 4Kpgbench: 1000 - 800 - Read Writepgbench: 1000 - 800 - Read Write - Average Latencyuvg266: Bosphorus 1080p - Super Fastsvt-av1: Preset 8 - Bosphorus 1080ppgbench: 1000 - 1000 - Read Writepgbench: 1000 - 1000 - Read Write - Average Latencypgbench: 1 - 800 - Read Writepgbench: 1 - 800 - Read Write - Average Latencyuvg266: Bosphorus 1080p - Very Fastx265: Bosphorus 1080psvt-vp9: PSNR/SSIM Optimized - Bosphorus 4Kpgbench: 100 - 800 - Read Only - Average Latencydeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Streampgbench: 100 - 800 - Read Onlysvt-av1: Preset 13 - Bosphorus 4Kpgbench: 1 - 1000 - Read Writepgbench: 1 - 1000 - Read Only - Average Latencypgbench: 1 - 1000 - Read Onlysvt-hevc: 7 - Bosphorus 4Kpgbench: 1000 - 800 - Read Onlypgbench: 1000 - 800 - Read Only - Average Latencyspecfem3d: Layered Halfspaceuvg266: Bosphorus 1080p - Ultra Fastbuild-ffmpeg: Time To Compileopenvino: Person Detection FP32 - CPUopenvino: Person Detection FP32 - CPUdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamcompress-7zip: Decompression Ratingpgbench: 1000 - 1000 - Read Onlypgbench: 1000 - 1000 - Read Only - Average Latencydeepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Streamdeepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamdeepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Streamsvt-hevc: 10 - Bosphorus 4Kclickhouse: 100M Rows Hits Dataset, First Run / Cold Cachex264: Bosphorus 4Kpgbench: 100 - 1000 - Read Only - Average Latencypgbench: 100 - 1000 - Read Onlysvt-av1: Preset 4 - Bosphorus 1080puvg266: Bosphorus 4K - Super Fastpgbench: 100 - 800 - Read Writepgbench: 100 - 800 - Read Write - Average Latencyopenvino: Face Detection Retail FP16 - CPUopenvino: Face Detection Retail FP16 - CPUdeepsparse: ResNet-50, Baseline - Asynchronous Multi-Streamdeepsparse: ResNet-50, Baseline - Asynchronous Multi-Streamsvt-vp9: VMAF Optimized - Bosphorus 1080pdeepsparse: ResNet-50, Baseline - Synchronous Single-Streamdeepsparse: ResNet-50, Baseline - Synchronous Single-Streamclickhouse: 100M Rows Hits Dataset, Second Runvvenc: Bosphorus 1080p - Fasteropenvino: Person Detection FP16 - CPUnamd: ATPase with 327,506 Atomsopenvino: Person Detection FP16 - CPUdeepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamclickhouse: 100M Rows Hits Dataset, Third Rundeepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Streamspecfem3d: Tomographic Modelsvt-av1: Preset 8 - Bosphorus 4Kdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamdeepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Streamdeepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Streamdeepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Streamdeepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Streamspecfem3d: Mount St. Helenssvt-vp9: VMAF Optimized - Bosphorus 4Knamd: STMV with 1,066,628 Atomsdeepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Streambuild-linux-kernel: defconfigopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUdeepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Streamopenvino: Weld Porosity Detection FP16-INT8 - CPUuvg266: Bosphorus 1080p - Slowgromacs: MPI CPU - water_GMX50_barecompress-7zip: Compression Ratingdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Streambrl-cad: VGR Performance Metricopenvino: Face Detection FP16-INT8 - CPUuvg266: Bosphorus 4K - Ultra Fastsvt-hevc: 1 - Bosphorus 1080popenvino: Face Detection FP16-INT8 - CPUuvg266: Bosphorus 1080p - Mediumopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUpgbench: 100 - 1000 - Read Write - Average Latencypgbench: 100 - 1000 - Read Writebuild-llvm: Unix Makefilesbuild-llvm: Ninjaopenfoam: drivaerFastback, Small Mesh Size - Mesh Timebuild-godot: Time To Compileopenvino: Age Gender Recognition Retail 0013 FP16 - CPUvvenc: Bosphorus 4K - Fasterspecfem3d: Water-layered Halfspacex265: Bosphorus 4Kvvenc: Bosphorus 1080p - Fastopenvino: Face Detection FP16 - CPUopenvino: Vehicle Detection FP16 - CPUsvt-av1: Preset 4 - Bosphorus 4Kopenvino: Vehicle Detection FP16 - CPUdeepsparse: Llama2 Chat 7b Quantized - Asynchronous Multi-Streamdeepsparse: Llama2 Chat 7b Quantized - Asynchronous Multi-Streamopenvino: Noise Suppression Poconet-Like FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUblender: BMW27 - CPU-Onlyopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Noise Suppression Poconet-Like FP16 - CPUsvt-vp9: Visual Quality Optimized - Bosphorus 1080pblender: Fishy Cat - CPU-Onlyopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Face Detection FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUuvg266: Bosphorus 4K - Mediumopenvino: Person Vehicle Bike Detection FP16 - CPUsvt-hevc: 1 - Bosphorus 4Kuvg266: Bosphorus 4K - Very Fastopenvino: Handwritten English Recognition FP16 - CPUopenvino: Handwritten English Recognition FP16 - CPUopenvino: Person Re-Identification Retail FP16 - CPUopenvino: Person Re-Identification Retail FP16 - CPUblender: Classroom - CPU-Onlybuild-nodejs: Time To Compileopenfoam: drivaerFastback, Medium Mesh Size - Mesh Timesvt-vp9: PSNR/SSIM Optimized - Bosphorus 1080popenvino: Face Detection Retail FP16-INT8 - CPUblender: Barbershop - CPU-Onlybuild-linux-kernel: allmodconfigsvt-hevc: 10 - Bosphorus 1080pbuild-gcc: Time To Compileopenvino: Face Detection Retail FP16-INT8 - CPUuvg266: Bosphorus 4K - Slowopenfoam: drivaerFastback, Small Mesh Size - Execution Timevvenc: Bosphorus 4K - Fastdeepsparse: Llama2 Chat 7b Quantized - Synchronous Single-Streamdeepsparse: Llama2 Chat 7b Quantized - Synchronous Single-Streamsvt-av1: Preset 12 - Bosphorus 1080pblender: Pabellon Barcelona - CPU-Onlysvt-av1: Preset 13 - Bosphorus 1080popenfoam: drivaerFastback, Medium Mesh Size - Execution Timeopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUpgbench: 1 - 1000 - Read Write - Average Latencyx264: Bosphorus 1080psvt-hevc: 7 - Bosphorus 1080pspecfem3d: Homogeneous Halfspaceabc44.8089.409658130.82859.67105.7832754129.10598.26136.0882518939.7252347341.05291.5884.8583.760.952446.58878.9566840809106.95317431.16485951448.386197381.292150.981664264120.8842.18628.53140.16109.83189.1063111.06309.0036651805995141.668802.07404.972719.234251.9778109.270036.595926.4927150.8967103.05226.1031.001.41170888716.22620.313497422.8759.88808.93109.338336.5726281.0810.269597.3359234.7140.203137.850.6720829.0110.271397.317851.320477.9417243.23334.711911.938356.66165837135.47195.444714.828310.4758362.274311.041167.43131.49269.8776101.1551667.993056.75468608781.760.209373.8825257.331791.8507.68210.1938.0472.630955.06411040.6832.901.38795609445.64028.9756189384371.8224.899.5410.7437.150.9321990.1933.39629945646.617636.00030.329564307.1158563.7611.604139.21004261819.4815.4482.6816.604.354240.785.0711781.119515.1933.68123.52118.74524.22224.63165.95497.2729.22273.6216.081495.1010.3843.92182.106.68384.902.3819.3152.04153.6618.18439.76336.83523.986221.90624291.271634.311297.101311.874381.27791.2664.895.97226.483294.954124.61958.0234532.480414.01644.2812182.83870.36575.609155.70171.8972.92466691439.65100.8310783770.74265.11117.0322597830.795108.39135.7052473240.4342236357.81198.6491.0389.470.948419.60259.5325844015109.00818461.10890221050.985881991.36151.953370514126.9140.22229.06137.62106.1979.416106.58219.3819679345810011.721813.73364.901218.553853.8839112.977735.394825.5887156.2254101.69226.1930.621.38672151715.72120.93438323.26810.06794.56112.375835.5844274.2710.02199.7371240.3840.747135.480.6673329.510.060199.327250.255479.5793239.55340.821411.724356.61069074436.02493.845414.550710.6542355.52711.250268.70721.48029.7105102.886673.15357.0672556280.440.209273.8228261.269290.4937.71213.2237.571.69955.7791036.6932.491.40696616442.90289.0311187148374.6424.729.5910.6736.750.9321792.1333.50929843639.853629.4730.018778306.6568538.5911.696139.33030273719.515.5662.6616.694.336239.555.1063775.759615.0933.7122.72118.68527.54225.96165.79499.4629.35272.4616.011502.5510.4243.88182.216.68383.452.3719.2351.83154.2818.17439.93335.62522.107222.69332291.651639.461298.171308.543381.92790.8684.885.98226.111034.946124.47198.0329531.906413.66644.7562184.34250.36541.625157.78166.272.25806154845.8787.1610578780.75666.15110.5692868527.889104.61124.1882305843.372413331.59296.7289.7288.881.009441.68929.0559793236113.6318151.17185391749.075947781.345144.466011134122.2840.38329.87133.88111.05899.0038111.31078.9835673345765861.734831.54434.796518.628353.6674109.109736.649426.3635151.6354105.17218.8031.631.43169905015.81520.263397623.5469.78817.16110.662836.1354274.4810.045599.4907237.9839.798138.640.6568328.8510.055399.393950.382979.3779238.31334.201711.955955.57467586336.15693.656814.557710.6756356.213411.228568.6721.46659.7892102.0626679.280956.14319272880.610.206013.8275260.955590.5057.6211.4337.8272.731354.98751051.2232.941.40396892439.84179.0939188833370.2724.69.6510.7936.90.9421757.7733.15630160643.708630.04130.039036304.5828495.0411.65138.25437520219.6315.4942.6616.574.367241.225.0995776.831315.1133.48122.81119.45527.2224.72165.05499.9229.37272.25161499.5710.4344.09181.396.71383.232.3719.2751.99153.8118.24438.29336.08523.423222.19301292.251636.641294.721308.925382.17789.5944.885.98226.417394.952124.55378.0276532.511413.87644.6952183.09580.36551.009152.15185.2471.570884588OpenBenchmarking.org

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16 - Device: CPUbac1020304050SE +/- 0.49, N = 1539.6544.8045.87MIN: 31.4 / MAX: 49.78MIN: 17.6 / MAX: 66.04MIN: 28.97 / MAX: 51.941. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16 - Device: CPUbac20406080100SE +/- 1.01, N = 15100.8389.4087.161. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

PostgreSQL

Scaling Factor: 1 - Clients: 800 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 1 - Clients: 800 - Mode: Read Onlybca200K400K600K800K1000KSE +/- 2188.42, N = 3107837710578789658131. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1 - Clients: 800 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 1 - Clients: 800 - Mode: Read Only - Average Latencybca0.18630.37260.55890.74520.9315SE +/- 0.002, N = 30.7420.7560.8281. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

SVT-VP9

Tuning: Visual Quality Optimized - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-VP9 0.3Tuning: Visual Quality Optimized - Input: Bosphorus 4Kcba1530456075SE +/- 0.69, N = 366.1565.1159.671. (CC) gcc options: -O3 -fcommon -fPIE -fPIC -fvisibility=hidden -pie -rdynamic -lpthread -lrt -lm

SVT-AV1

Encoder Mode: Preset 12 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 12 - Input: Bosphorus 4Kbca306090120150SE +/- 0.98, N = 15117.03110.57105.781. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

PostgreSQL

Scaling Factor: 1000 - Clients: 800 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 800 - Mode: Read Writecab6K12K18K24K30KSE +/- 372.60, N = 122868527541259781. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1000 - Clients: 800 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 800 - Mode: Read Write - Average Latencycab714212835SE +/- 0.39, N = 1227.8929.1130.801. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

uvg266

Video Input: Bosphorus 1080p - Video Preset: Super Fast

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 1080p - Video Preset: Super Fastbca20406080100SE +/- 1.28, N = 12108.39104.6198.26

SVT-AV1

Encoder Mode: Preset 8 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 8 - Input: Bosphorus 1080pabc306090120150SE +/- 0.21, N = 3136.09135.71124.191. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

PostgreSQL

Scaling Factor: 1000 - Clients: 1000 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 1000 - Mode: Read Writeabc5K10K15K20K25KSE +/- 190.45, N = 122518924732230581. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1000 - Clients: 1000 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 1000 - Mode: Read Write - Average Latencyabc1020304050SE +/- 0.30, N = 1239.7340.4343.371. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1 - Clients: 800 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 1 - Clients: 800 - Mode: Read Writecab5001000150020002500SE +/- 33.78, N = 32413234722361. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1 - Clients: 800 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 1 - Clients: 800 - Mode: Read Write - Average Latencycab80160240320400SE +/- 4.95, N = 3331.59341.05357.811. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

uvg266

Video Input: Bosphorus 1080p - Video Preset: Very Fast

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 1080p - Video Preset: Very Fastbca20406080100SE +/- 1.03, N = 1598.6496.7291.58

x265

Video Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is Betterx265 3.4Video Input: Bosphorus 1080pbca20406080100SE +/- 0.71, N = 1591.0389.7284.851. (CXX) g++ options: -O3 -rdynamic -lpthread -lrt -ldl -lnuma

SVT-VP9

Tuning: PSNR/SSIM Optimized - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-VP9 0.3Tuning: PSNR/SSIM Optimized - Input: Bosphorus 4Kbca20406080100SE +/- 0.78, N = 1589.4788.8883.761. (CC) gcc options: -O3 -fcommon -fPIE -fPIC -fvisibility=hidden -pie -rdynamic -lpthread -lrt -lm

PostgreSQL

Scaling Factor: 100 - Clients: 800 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 100 - Clients: 800 - Mode: Read Only - Average Latencybac0.2270.4540.6810.9081.135SE +/- 0.010, N = 30.9480.9521.0091. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Streambca100200300400500SE +/- 0.58, N = 3419.60441.69446.59

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Streambca3691215SE +/- 0.0116, N = 39.53259.05598.9566

PostgreSQL

Scaling Factor: 100 - Clients: 800 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 100 - Clients: 800 - Mode: Read Onlybac200K400K600K800K1000KSE +/- 8610.85, N = 38440158408097932361. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

SVT-AV1

Encoder Mode: Preset 13 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 13 - Input: Bosphorus 4Kcba306090120150SE +/- 1.39, N = 15113.63109.01106.951. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

PostgreSQL

Scaling Factor: 1 - Clients: 1000 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 1 - Clients: 1000 - Mode: Read Writebca400800120016002000SE +/- 28.21, N = 121846181517431. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1 - Clients: 1000 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 1 - Clients: 1000 - Mode: Read Only - Average Latencybac0.26350.5270.79051.0541.3175SE +/- 0.010, N = 31.1081.1641.1711. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1 - Clients: 1000 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 1 - Clients: 1000 - Mode: Read Onlybac200K400K600K800K1000KSE +/- 6896.35, N = 39022108595148539171. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

SVT-HEVC

Tuning: 7 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-HEVC 1.5.0Tuning: 7 - Input: Bosphorus 4Kbca1224364860SE +/- 0.07, N = 350.9849.0748.381. (CC) gcc options: -fPIE -fPIC -O3 -O2 -pie -rdynamic -lpthread -lrt

PostgreSQL

Scaling Factor: 1000 - Clients: 800 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 800 - Mode: Read Onlyacb130K260K390K520K650KSE +/- 5061.88, N = 96197385947785881991. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1000 - Clients: 800 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 800 - Mode: Read Only - Average Latencyacb0.3060.6120.9181.2241.53SE +/- 0.011, N = 91.2921.3451.3601. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

SPECFEM3D

Model: Layered Halfspace

OpenBenchmarking.orgSeconds, Fewer Is BetterSPECFEM3D 4.0Model: Layered Halfspacecab306090120150SE +/- 1.25, N = 3144.47150.98151.951. (F9X) gfortran options: -O2 -fopenmp -std=f2003 -fimplicit-none -fmax-errors=10 -pedantic -pedantic-errors -O3 -finline-functions -lmpi_usempif08 -lmpi_mpifh -lmpi -lopen-rte -lopen-pal -lhwloc -levent_core -levent_pthreads -lm -lz

uvg266

Video Input: Bosphorus 1080p - Video Preset: Ultra Fast

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 1080p - Video Preset: Ultra Fastbca306090120150SE +/- 0.11, N = 3126.91122.28120.88

Timed FFmpeg Compilation

Time To Compile

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed FFmpeg Compilation 6.1Time To Compilebca1020304050SE +/- 0.05, N = 340.2240.3842.19

OpenVINO

Model: Person Detection FP32 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Detection FP32 - Device: CPUcba714212835SE +/- 0.07, N = 329.8729.0628.531. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Detection FP32 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Detection FP32 - Device: CPUcba306090120150SE +/- 0.35, N = 3133.88137.62140.16MIN: 115.87 / MAX: 217.54MIN: 115.29 / MAX: 146.55MIN: 114.55 / MAX: 222.641. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Streambac20406080100SE +/- 1.11, N = 3106.20109.83111.06

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Streambac3691215SE +/- 0.0932, N = 39.41609.10639.0038

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Streambac20406080100SE +/- 0.25, N = 3106.58111.06111.31

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Streambac3691215SE +/- 0.0199, N = 39.38199.00368.9835

7-Zip Compression

Test: Decompression Rating

OpenBenchmarking.orgMIPS, More Is Better7-Zip Compression 22.01Test: Decompression Ratingbca15K30K45K60K75KSE +/- 855.85, N = 36793467334651801. (CXX) g++ options: -lpthread -ldl -O2 -fPIC

PostgreSQL

Scaling Factor: 1000 - Clients: 1000 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 1000 - Mode: Read Onlyabc130K260K390K520K650KSE +/- 5543.80, N = 35995145810015765861. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1000 - Clients: 1000 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 1000 - Mode: Read Only - Average Latencyabc0.39020.78041.17061.56081.951SE +/- 0.016, N = 31.6681.7211.7341. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

Neural Magic DeepSparse

Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Streamcba2004006008001000SE +/- 1.78, N = 3831.54813.73802.07

Neural Magic DeepSparse

Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Streamcba1.11892.23783.35674.47565.5945SE +/- 0.0110, N = 34.79654.90124.9727

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Streambca510152025SE +/- 0.02, N = 318.5518.6319.23

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Streambca1224364860SE +/- 0.05, N = 353.8853.6751.98

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Streambac306090120150SE +/- 0.18, N = 3112.98109.27109.11

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Streambac816243240SE +/- 0.06, N = 335.3936.6036.65

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Streambca612182430SE +/- 0.03, N = 325.5926.3626.49

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Streambca306090120150SE +/- 0.16, N = 3156.23151.64150.90

SVT-HEVC

Tuning: 10 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-HEVC 1.5.0Tuning: 10 - Input: Bosphorus 4Kcab20406080100SE +/- 1.12, N = 14105.17103.05101.691. (CC) gcc options: -fPIE -fPIC -O3 -O2 -pie -rdynamic -lpthread -lrt

ClickHouse

100M Rows Hits Dataset, First Run / Cold Cache

OpenBenchmarking.orgQueries Per Minute, Geo Mean, More Is BetterClickHouse 22.12.3.5100M Rows Hits Dataset, First Run / Cold Cachebac50100150200250SE +/- 0.26, N = 3226.19226.10218.80MIN: 9.16 / MAX: 8571.43MIN: 9.15 / MAX: 10000MIN: 9.11 / MAX: 10000

x264

Video Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is Betterx264 2022-02-22Video Input: Bosphorus 4Kcab714212835SE +/- 0.38, N = 331.6331.0030.621. (CC) gcc options: -ldl -lavformat -lavcodec -lavutil -lswscale -m64 -lm -lpthread -O3 -flto

PostgreSQL

Scaling Factor: 100 - Clients: 1000 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 100 - Clients: 1000 - Mode: Read Only - Average Latencybac0.3220.6440.9661.2881.61SE +/- 0.016, N = 31.3861.4111.4311. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 100 - Clients: 1000 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 100 - Clients: 1000 - Mode: Read Onlybac150K300K450K600K750KSE +/- 8095.75, N = 37215177088876990501. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

SVT-AV1

Encoder Mode: Preset 4 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 4 - Input: Bosphorus 1080pacb48121620SE +/- 0.12, N = 1516.2315.8215.721. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

uvg266

Video Input: Bosphorus 4K - Video Preset: Super Fast

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 4K - Video Preset: Super Fastbac510152025SE +/- 0.02, N = 320.9020.3120.26

PostgreSQL

Scaling Factor: 100 - Clients: 800 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 100 - Clients: 800 - Mode: Read Writeabc7K14K21K28K35KSE +/- 96.04, N = 33497434383339761. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 100 - Clients: 800 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 100 - Clients: 800 - Mode: Read Write - Average Latencyabc612182430SE +/- 0.06, N = 322.8823.2723.551. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16 - Device: CPUcab3691215SE +/- 0.11, N = 39.789.8810.06MIN: 6.25 / MAX: 37.55MIN: 5.09 / MAX: 39.75MIN: 4.92 / MAX: 36.161. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16 - Device: CPUcab2004006008001000SE +/- 9.03, N = 3817.16808.93794.561. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Neural Magic DeepSparse

Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Streambca306090120150SE +/- 0.05, N = 3112.38110.66109.34

Neural Magic DeepSparse

Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Streambca816243240SE +/- 0.02, N = 335.5836.1436.57

SVT-VP9

Tuning: VMAF Optimized - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-VP9 0.3Tuning: VMAF Optimized - Input: Bosphorus 1080pacb60120180240300SE +/- 4.04, N = 3281.08274.48274.271. (CC) gcc options: -O3 -fcommon -fPIE -fPIC -fvisibility=hidden -pie -rdynamic -lpthread -lrt -lm

Neural Magic DeepSparse

Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Baseline - Scenario: Synchronous Single-Streambca3691215SE +/- 0.10, N = 310.0210.0510.27

Neural Magic DeepSparse

Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Baseline - Scenario: Synchronous Single-Streambca20406080100SE +/- 0.95, N = 399.7499.4997.34

ClickHouse

100M Rows Hits Dataset, Second Run

OpenBenchmarking.orgQueries Per Minute, Geo Mean, More Is BetterClickHouse 22.12.3.5100M Rows Hits Dataset, Second Runbca50100150200250SE +/- 0.63, N = 3240.38237.98234.71MIN: 9.44 / MAX: 12000MIN: 9.26 / MAX: 8571.43MIN: 9.36 / MAX: 10000

VVenC

Video Input: Bosphorus 1080p - Video Preset: Faster

OpenBenchmarking.orgFrames Per Second, More Is BetterVVenC 1.11Video Input: Bosphorus 1080p - Video Preset: Fasterbac918273645SE +/- 0.28, N = 1540.7540.2039.801. (CXX) g++ options: -O3 -flto -fno-fat-lto-objects -flto=auto

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Detection FP16 - Device: CPUbac306090120150SE +/- 1.56, N = 3135.48137.85138.64MIN: 115.39 / MAX: 224.16MIN: 112.51 / MAX: 221.34MIN: 114.61 / MAX: 225.731. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

NAMD

Input: ATPase with 327,506 Atoms

OpenBenchmarking.orgns/day, More Is BetterNAMD 3.0b6Input: ATPase with 327,506 Atomsabc0.15120.30240.45360.60480.756SE +/- 0.00274, N = 30.672080.667330.65683

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Detection FP16 - Device: CPUbac714212835SE +/- 0.32, N = 329.5029.0128.851. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Streamcba3691215SE +/- 0.11, N = 310.0610.0610.27

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Streamcba20406080100SE +/- 1.02, N = 399.3999.3397.32

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Streambca1224364860SE +/- 0.49, N = 350.2650.3851.32

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Streambca20406080100SE +/- 0.75, N = 379.5879.3877.94

ClickHouse

100M Rows Hits Dataset, Third Run

OpenBenchmarking.orgQueries Per Minute, Geo Mean, More Is BetterClickHouse 22.12.3.5100M Rows Hits Dataset, Third Runabc50100150200250SE +/- 1.48, N = 3243.23239.55238.31MIN: 9.39 / MAX: 10000MIN: 9.46 / MAX: 10000MIN: 9.23 / MAX: 8571.43

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Streambac70140210280350SE +/- 0.48, N = 3340.82334.71334.20

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Streambac3691215SE +/- 0.02, N = 311.7211.9411.96

SPECFEM3D

Model: Tomographic Model

OpenBenchmarking.orgSeconds, Fewer Is BetterSPECFEM3D 4.0Model: Tomographic Modelcba1326395265SE +/- 0.29, N = 355.5756.6156.661. (F9X) gfortran options: -O2 -fopenmp -std=f2003 -fimplicit-none -fmax-errors=10 -pedantic -pedantic-errors -O3 -finline-functions -lmpi_usempif08 -lmpi_mpifh -lmpi -lopen-rte -lopen-pal -lhwloc -levent_core -levent_pthreads -lm -lz

SVT-AV1

Encoder Mode: Preset 8 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 8 - Input: Bosphorus 4Kcba816243240SE +/- 0.15, N = 336.1636.0235.471. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Streamcba20406080100SE +/- 0.24, N = 393.6693.8595.44

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Streambca48121620SE +/- 0.14, N = 314.5514.5614.83

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Streamcba3691215SE +/- 0.03, N = 310.6810.6510.48

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Streambca80160240320400SE +/- 1.69, N = 3355.53356.21362.27

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Streambca3691215SE +/- 0.05, N = 311.2511.2311.04

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Streambca1530456075SE +/- 0.62, N = 368.7168.6767.43

Neural Magic DeepSparse

Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Streamcba0.33580.67161.00741.34321.679SE +/- 0.0116, N = 101.46651.48021.4926

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Streambca3691215SE +/- 0.0846, N = 39.71059.78929.8776

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Streambca20406080100SE +/- 0.86, N = 3102.89102.06101.16

Neural Magic DeepSparse

Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Streamcba150300450600750SE +/- 5.27, N = 10679.28673.15667.99

SPECFEM3D

Model: Mount St. Helens

OpenBenchmarking.orgSeconds, Fewer Is BetterSPECFEM3D 4.0Model: Mount St. Helenscab1326395265SE +/- 0.60, N = 556.1456.7557.071. (F9X) gfortran options: -O2 -fopenmp -std=f2003 -fimplicit-none -fmax-errors=10 -pedantic -pedantic-errors -O3 -finline-functions -lmpi_usempif08 -lmpi_mpifh -lmpi -lopen-rte -lopen-pal -lhwloc -levent_core -levent_pthreads -lm -lz

SVT-VP9

Tuning: VMAF Optimized - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-VP9 0.3Tuning: VMAF Optimized - Input: Bosphorus 4Kacb20406080100SE +/- 0.99, N = 481.7680.6180.441. (CC) gcc options: -O3 -fcommon -fPIE -fPIC -fvisibility=hidden -pie -rdynamic -lpthread -lrt -lm

NAMD

Input: STMV with 1,066,628 Atoms

OpenBenchmarking.orgns/day, More Is BetterNAMD 3.0b6Input: STMV with 1,066,628 Atomsabc0.04710.09420.14130.18840.2355SE +/- 0.00013, N = 30.209370.209270.20601

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Streambca0.87361.74722.62083.49444.368SE +/- 0.0378, N = 33.82283.82753.8825

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Streambca60120180240300SE +/- 2.52, N = 3261.27260.96257.33

Timed Linux Kernel Compilation

Build: defconfig

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Linux Kernel Compilation 6.8Build: defconfigbca20406080100SE +/- 0.18, N = 390.4990.5191.85

OpenVINO

Model: Weld Porosity Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16-INT8 - Device: CPUcab246810SE +/- 0.03, N = 37.607.687.71MIN: 5.3 / MAX: 31.74MIN: 4.73 / MAX: 36.9MIN: 4.97 / MAX: 32.251. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Road Segmentation ADAS FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16-INT8 - Device: CPUbca50100150200250SE +/- 0.70, N = 3213.22211.43210.191. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Road Segmentation ADAS FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16-INT8 - Device: CPUbca918273645SE +/- 0.13, N = 337.5037.8238.04MIN: 26.78 / MAX: 87.54MIN: 21.82 / MAX: 84.8MIN: 21.24 / MAX: 90.731. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Streambac1632486480SE +/- 0.13, N = 371.7072.6372.73

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Streambac1326395265SE +/- 0.10, N = 355.7855.0654.99

OpenVINO

Model: Weld Porosity Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16-INT8 - Device: CPUcab2004006008001000SE +/- 3.64, N = 31051.221040.681036.691. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

uvg266

Video Input: Bosphorus 1080p - Video Preset: Slow

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 1080p - Video Preset: Slowcab816243240SE +/- 0.40, N = 332.9432.9032.49

GROMACS

Implementation: MPI CPU - Input: water_GMX50_bare

OpenBenchmarking.orgNs Per Day, More Is BetterGROMACS 2024Implementation: MPI CPU - Input: water_GMX50_barebca0.31640.63280.94921.26561.582SE +/- 0.015, N = 51.4061.4031.3871. (CXX) g++ options: -O3 -lm

7-Zip Compression

Test: Compression Rating

OpenBenchmarking.orgMIPS, More Is Better7-Zip Compression 22.01Test: Compression Ratingcba20K40K60K80K100KSE +/- 806.12, N = 39689296616956091. (CXX) g++ options: -lpthread -ldl -O2 -fPIC

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Streamcba100200300400500SE +/- 0.33, N = 3439.84442.90445.64

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Streamcba3691215SE +/- 0.0066, N = 39.09399.03118.9756

BRL-CAD

VGR Performance Metric

OpenBenchmarking.orgVGR Performance Metric, More Is BetterBRL-CAD 7.38.2VGR Performance Metricacb40K80K120K160K200K1893841888331871481. (CXX) g++ options: -std=c++17 -pipe -fvisibility=hidden -fno-strict-aliasing -fno-common -fexceptions -ftemplate-depth-128 -m64 -ggdb3 -O3 -fipa-pta -fstrength-reduce -finline-functions -flto -ltcl8.6 -lnetpbm -lregex_brl -lz_brl -lassimp -ldl -lm -ltk8.6

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection FP16-INT8 - Device: CPUcab80160240320400SE +/- 1.42, N = 3370.27371.82374.64MIN: 284.02 / MAX: 518.25MIN: 283.89 / MAX: 476.9MIN: 283.87 / MAX: 471.91. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

uvg266

Video Input: Bosphorus 4K - Video Preset: Ultra Fast

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 4K - Video Preset: Ultra Fastabc612182430SE +/- 0.14, N = 324.8924.7224.60

SVT-HEVC

Tuning: 1 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-HEVC 1.5.0Tuning: 1 - Input: Bosphorus 1080pcba3691215SE +/- 0.03, N = 39.659.599.541. (CC) gcc options: -fPIE -fPIC -O3 -O2 -pie -rdynamic -lpthread -lrt

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection FP16-INT8 - Device: CPUcab3691215SE +/- 0.05, N = 310.7910.7410.671. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

uvg266

Video Input: Bosphorus 1080p - Video Preset: Medium

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 1080p - Video Preset: Mediumacb918273645SE +/- 0.23, N = 337.1536.9036.75

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUabc0.21150.4230.63450.8461.0575SE +/- 0.00, N = 30.930.930.94MIN: 0.5 / MAX: 10.02MIN: 0.55 / MAX: 9.6MIN: 0.57 / MAX: 17.211. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUabc5K10K15K20K25KSE +/- 20.16, N = 321990.1921792.1321757.771. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

PostgreSQL

Scaling Factor: 100 - Clients: 1000 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 100 - Clients: 1000 - Mode: Read Write - Average Latencycab816243240SE +/- 0.14, N = 333.1633.4033.511. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 100 - Clients: 1000 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 100 - Clients: 1000 - Mode: Read Writecab6K12K18K24K30KSE +/- 122.20, N = 33016029945298431. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

Timed LLVM Compilation

Build System: Unix Makefiles

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed LLVM Compilation 16.0Build System: Unix Makefilesbca140280420560700SE +/- 2.26, N = 3639.85643.71646.62

Timed LLVM Compilation

Build System: Ninja

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed LLVM Compilation 16.0Build System: Ninjabca140280420560700SE +/- 2.72, N = 3629.47630.04636.00

OpenFOAM

Input: drivaerFastback, Small Mesh Size - Mesh Time

OpenBenchmarking.orgSeconds, Fewer Is BetterOpenFOAM 10Input: drivaerFastback, Small Mesh Size - Mesh Timebca71421283530.0230.0430.331. (CXX) g++ options: -std=c++14 -m64 -O3 -ftemplate-depth-100 -fPIC -fuse-ld=bfd -Xlinker --add-needed --no-as-needed -ldynamicMesh -lgenericPatchFields -lOpenFOAM -ldl -lm

Timed Godot Game Engine Compilation

Time To Compile

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Godot Game Engine Compilation 4.0Time To Compilecba70140210280350SE +/- 0.18, N = 3304.58306.66307.12

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUabc2K4K6K8K10KSE +/- 16.19, N = 38563.768538.598495.041. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

VVenC

Video Input: Bosphorus 4K - Video Preset: Faster

OpenBenchmarking.orgFrames Per Second, More Is BetterVVenC 1.11Video Input: Bosphorus 4K - Video Preset: Fasterbca3691215SE +/- 0.01, N = 311.7011.6511.601. (CXX) g++ options: -O3 -flto -fno-fat-lto-objects -flto=auto

SPECFEM3D

Model: Water-layered Halfspace

OpenBenchmarking.orgSeconds, Fewer Is BetterSPECFEM3D 4.0Model: Water-layered Halfspacecab306090120150SE +/- 0.57, N = 3138.25139.21139.331. (F9X) gfortran options: -O2 -fopenmp -std=f2003 -fimplicit-none -fmax-errors=10 -pedantic -pedantic-errors -O3 -finline-functions -lmpi_usempif08 -lmpi_mpifh -lmpi -lopen-rte -lopen-pal -lhwloc -levent_core -levent_pthreads -lm -lz

x265

Video Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is Betterx265 3.4Video Input: Bosphorus 4Kcba510152025SE +/- 0.08, N = 319.6319.5019.481. (CXX) g++ options: -O3 -rdynamic -lpthread -lrt -ldl -lnuma

VVenC

Video Input: Bosphorus 1080p - Video Preset: Fast

OpenBenchmarking.orgFrames Per Second, More Is BetterVVenC 1.11Video Input: Bosphorus 1080p - Video Preset: Fastbca48121620SE +/- 0.04, N = 315.5715.4915.451. (CXX) g++ options: -O3 -flto -fno-fat-lto-objects -flto=auto

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection FP16 - Device: CPUacb0.6031.2061.8092.4123.015SE +/- 0.01, N = 32.682.662.661. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16 - Device: CPUcab48121620SE +/- 0.04, N = 316.5716.6016.69MIN: 7.95 / MAX: 34.05MIN: 8.7 / MAX: 42.87MIN: 11.64 / MAX: 52.911. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

SVT-AV1

Encoder Mode: Preset 4 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 4 - Input: Bosphorus 4Kcab0.98261.96522.94783.93044.913SE +/- 0.009, N = 34.3674.3544.3361. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16 - Device: CPUcab50100150200250SE +/- 0.65, N = 3241.22240.78239.551. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Neural Magic DeepSparse

Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Streambca1.14892.29783.44674.59565.7445SE +/- 0.0090, N = 35.10635.09955.0711

Neural Magic DeepSparse

Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Streambca2004006008001000SE +/- 1.38, N = 3775.76776.83781.12

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Noise Suppression Poconet-Like FP16 - Device: CPUbca48121620SE +/- 0.03, N = 315.0915.1115.19MIN: 9.56 / MAX: 53.46MIN: 10.08 / MAX: 54.61MIN: 9.31 / MAX: 55.491. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Machine Translation EN To DE FP16 - Device: CPUbac816243240SE +/- 0.06, N = 333.7033.6833.481. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Blender

Blend File: BMW27 - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.0Blend File: BMW27 - Compute: CPU-Onlybca306090120150SE +/- 0.23, N = 3122.72122.81123.52

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Machine Translation EN To DE FP16 - Device: CPUbac306090120150SE +/- 0.23, N = 3118.68118.74119.45MIN: 90.15 / MAX: 196.79MIN: 92.44 / MAX: 207.3MIN: 92.57 / MAX: 198.031. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Noise Suppression Poconet-Like FP16 - Device: CPUbca110220330440550SE +/- 0.86, N = 3527.54527.20524.221. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

SVT-VP9

Tuning: Visual Quality Optimized - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-VP9 0.3Tuning: Visual Quality Optimized - Input: Bosphorus 1080pbca50100150200250SE +/- 0.62, N = 3225.96224.72224.631. (CC) gcc options: -O3 -fcommon -fPIE -fPIC -fvisibility=hidden -pie -rdynamic -lpthread -lrt -lm

Blender

Blend File: Fishy Cat - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.0Blend File: Fishy Cat - Compute: CPU-Onlycba4080120160200SE +/- 0.37, N = 3165.05165.79165.95

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16-INT8 - Device: CPUcba110220330440550SE +/- 2.47, N = 3499.92499.46497.271. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16 - Device: CPUabc714212835SE +/- 0.09, N = 329.2229.3529.37MIN: 15.45 / MAX: 72.82MIN: 15.67 / MAX: 72.92MIN: 15.64 / MAX: 69.571. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16 - Device: CPUabc60120180240300SE +/- 0.87, N = 3273.62272.46272.251. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16-INT8 - Device: CPUcba48121620SE +/- 0.08, N = 316.0016.0116.08MIN: 10.21 / MAX: 59.04MIN: 10.53 / MAX: 53.92MIN: 10.36 / MAX: 57.941. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection FP16 - Device: CPUacb30060090012001500SE +/- 5.23, N = 31495.101499.571502.55MIN: 1179.18 / MAX: 1678.08MIN: 1217.27 / MAX: 1680.47MIN: 1181.84 / MAX: 1684.451. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Vehicle Bike Detection FP16 - Device: CPUabc3691215SE +/- 0.01, N = 310.3810.4210.43MIN: 6.09 / MAX: 31.65MIN: 6.07 / MAX: 30.65MIN: 7.42 / MAX: 34.611. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Handwritten English Recognition FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16-INT8 - Device: CPUbac1020304050SE +/- 0.23, N = 343.8843.9244.09MIN: 32.8 / MAX: 111.1MIN: 32.41 / MAX: 102.38MIN: 32.81 / MAX: 104.251. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Handwritten English Recognition FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16-INT8 - Device: CPUbac4080120160200SE +/- 0.96, N = 3182.21182.10181.391. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

uvg266

Video Input: Bosphorus 4K - Video Preset: Medium

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 4K - Video Preset: Mediumcba246810SE +/- 0.00, N = 36.716.686.68

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Vehicle Bike Detection FP16 - Device: CPUabc80160240320400SE +/- 0.46, N = 3384.90383.45383.231. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

SVT-HEVC

Tuning: 1 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-HEVC 1.5.0Tuning: 1 - Input: Bosphorus 4Kacb0.53551.0711.60652.1422.6775SE +/- 0.00, N = 32.382.372.371. (CC) gcc options: -fPIE -fPIC -O3 -O2 -pie -rdynamic -lpthread -lrt

uvg266

Video Input: Bosphorus 4K - Video Preset: Very Fast

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 4K - Video Preset: Very Fastacb510152025SE +/- 0.12, N = 319.3119.2719.23

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16 - Device: CPUbca1224364860SE +/- 0.22, N = 351.8351.9952.04MIN: 39.59 / MAX: 122.23MIN: 39.27 / MAX: 121.44MIN: 38.84 / MAX: 116.681. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16 - Device: CPUbca306090120150SE +/- 0.64, N = 3154.28153.81153.661. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Re-Identification Retail FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Re-Identification Retail FP16 - Device: CPUbac48121620SE +/- 0.03, N = 318.1718.1818.24MIN: 9.62 / MAX: 52.96MIN: 9.82 / MAX: 46.59MIN: 9.91 / MAX: 48.241. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Re-Identification Retail FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Re-Identification Retail FP16 - Device: CPUbac100200300400500SE +/- 0.80, N = 3439.93439.76438.291. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Blender

Blend File: Classroom - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.0Blend File: Classroom - Compute: CPU-Onlybca70140210280350SE +/- 0.33, N = 3335.62336.08336.83

Timed Node.js Compilation

Time To Compile

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Node.js Compilation 19.8.1Time To Compilebca110220330440550SE +/- 0.47, N = 3522.11523.42523.99

OpenFOAM

Input: drivaerFastback, Medium Mesh Size - Mesh Time

OpenBenchmarking.orgSeconds, Fewer Is BetterOpenFOAM 10Input: drivaerFastback, Medium Mesh Size - Mesh Timeacb50100150200250221.91222.19222.691. (CXX) g++ options: -std=c++14 -m64 -O3 -ftemplate-depth-100 -fPIC -fuse-ld=bfd -Xlinker --add-needed --no-as-needed -ldynamicMesh -lgenericPatchFields -lOpenFOAM -ldl -lm

SVT-VP9

Tuning: PSNR/SSIM Optimized - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-VP9 0.3Tuning: PSNR/SSIM Optimized - Input: Bosphorus 1080pcba60120180240300SE +/- 0.30, N = 3292.25291.65291.271. (CC) gcc options: -O3 -fcommon -fPIE -fPIC -fvisibility=hidden -pie -rdynamic -lpthread -lrt -lm

OpenVINO

Model: Face Detection Retail FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16-INT8 - Device: CPUbca400800120016002000SE +/- 5.93, N = 31639.461636.641634.311. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Blender

Blend File: Barbershop - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.0Blend File: Barbershop - Compute: CPU-Onlycab30060090012001500SE +/- 0.90, N = 31294.721297.101298.17

Timed Linux Kernel Compilation

Build: allmodconfig

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Linux Kernel Compilation 6.8Build: allmodconfigbca30060090012001500SE +/- 0.39, N = 31308.541308.931311.87

SVT-HEVC

Tuning: 10 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-HEVC 1.5.0Tuning: 10 - Input: Bosphorus 1080pcba80160240320400SE +/- 0.21, N = 3382.17381.92381.271. (CC) gcc options: -fPIE -fPIC -O3 -O2 -pie -rdynamic -lpthread -lrt

Timed GCC Compilation

Time To Compile

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed GCC Compilation 13.2Time To Compilecba2004006008001000SE +/- 1.92, N = 3789.59790.87791.27

OpenVINO

Model: Face Detection Retail FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16-INT8 - Device: CPUbca1.10032.20063.30094.40125.5015SE +/- 0.02, N = 34.884.884.89MIN: 3.54 / MAX: 21.77MIN: 3.52 / MAX: 27.88MIN: 3.4 / MAX: 23.021. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

uvg266

Video Input: Bosphorus 4K - Video Preset: Slow

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 4K - Video Preset: Slowcba1.34552.6914.03655.3826.7275SE +/- 0.01, N = 35.985.985.97

OpenFOAM

Input: drivaerFastback, Small Mesh Size - Execution Time

OpenBenchmarking.orgSeconds, Fewer Is BetterOpenFOAM 10Input: drivaerFastback, Small Mesh Size - Execution Timebca50100150200250226.11226.42226.481. (CXX) g++ options: -std=c++14 -m64 -O3 -ftemplate-depth-100 -fPIC -fuse-ld=bfd -Xlinker --add-needed --no-as-needed -ldynamicMesh -lgenericPatchFields -lOpenFOAM -ldl -lm

VVenC

Video Input: Bosphorus 4K - Video Preset: Fast

OpenBenchmarking.orgFrames Per Second, More Is BetterVVenC 1.11Video Input: Bosphorus 4K - Video Preset: Fastacb1.11472.22943.34414.45885.5735SE +/- 0.024, N = 34.9544.9524.9461. (CXX) g++ options: -O3 -flto -fno-fat-lto-objects -flto=auto

Neural Magic DeepSparse

Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Streambca306090120150SE +/- 0.06, N = 3124.47124.55124.62

Neural Magic DeepSparse

Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Streambca246810SE +/- 0.0042, N = 38.03298.02768.0234

SVT-AV1

Encoder Mode: Preset 12 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 12 - Input: Bosphorus 1080pcab120240360480600SE +/- 1.61, N = 3532.51532.48531.911. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Blender

Blend File: Pabellon Barcelona - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.0Blend File: Pabellon Barcelona - Compute: CPU-Onlybca90180270360450SE +/- 0.40, N = 3413.66413.87414.01

SVT-AV1

Encoder Mode: Preset 13 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 13 - Input: Bosphorus 1080pbca140280420560700SE +/- 0.52, N = 3644.76644.70644.281. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

OpenFOAM

Input: drivaerFastback, Medium Mesh Size - Execution Time

OpenBenchmarking.orgSeconds, Fewer Is BetterOpenFOAM 10Input: drivaerFastback, Medium Mesh Size - Execution Timeacb50010001500200025002182.842183.102184.341. (CXX) g++ options: -std=c++14 -m64 -O3 -ftemplate-depth-100 -fPIC -fuse-ld=bfd -Xlinker --add-needed --no-as-needed -ldynamicMesh -lgenericPatchFields -lOpenFOAM -ldl -lm

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUabc0.0810.1620.2430.3240.405SE +/- 0.00, N = 30.360.360.36MIN: 0.19 / MAX: 8.34MIN: 0.21 / MAX: 7.95MIN: 0.19 / MAX: 8.11. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

PostgreSQL

Scaling Factor: 1 - Clients: 1000 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 1 - Clients: 1000 - Mode: Read Write - Average Latencybca120240360480600SE +/- 10.12, N = 12541.63551.01575.611. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

x264

Video Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is Betterx264 2022-02-22Video Input: Bosphorus 1080pbac306090120150SE +/- 2.82, N = 15157.78155.70152.151. (CC) gcc options: -ldl -lavformat -lavcodec -lavutil -lswscale -m64 -lm -lpthread -O3 -flto

SVT-HEVC

Tuning: 7 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-HEVC 1.5.0Tuning: 7 - Input: Bosphorus 1080pcab4080120160200SE +/- 3.25, N = 15185.24171.89166.201. (CC) gcc options: -fPIE -fPIC -O3 -O2 -pie -rdynamic -lpthread -lrt

SPECFEM3D

Model: Homogeneous Halfspace

OpenBenchmarking.orgSeconds, Fewer Is BetterSPECFEM3D 4.0Model: Homogeneous Halfspacecba1632486480SE +/- 1.31, N = 1271.5772.2672.921. (F9X) gfortran options: -O2 -fopenmp -std=f2003 -fimplicit-none -fmax-errors=10 -pedantic -pedantic-errors -O3 -finline-functions -lmpi_usempif08 -lmpi_mpifh -lmpi -lopen-rte -lopen-pal -lhwloc -levent_core -levent_pthreads -lm -lz

Phoronix Test Suite v10.8.5