AMD EPYC 4th Gen AVX-512 Comparison

AMD EPYC 9654 Genoa AVX-512 benchmark comparison by Michael Larabel for a future article.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2212195-NE-AVXCOMPAR69
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results
Show Result Confidence Charts

Limit displaying results to tests within:

C/C++ Compiler Tests 3 Tests
CPU Massive 7 Tests
Creator Workloads 9 Tests
Cryptography 2 Tests
Game Development 2 Tests
HPC - High Performance Computing 16 Tests
Machine Learning 11 Tests
Molecular Dynamics 3 Tests
Multi-Core 10 Tests
NVIDIA GPU Compute 3 Tests
Intel oneAPI 7 Tests
OpenMPI Tests 3 Tests
Python 2 Tests
Raytracing 2 Tests
Renderers 2 Tests
Scientific Computing 3 Tests
Server CPU Tests 6 Tests

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Geometric Means Per-Suite/Category
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs
No Box Plots
On Line Graphs With Missing Data, Connect The Line Gaps

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs
Condense Test Profiles With Multiple Version Results Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
AVX-512 On
December 18 2022
  20 Hours, 2 Minutes
AVX-512 Off
December 18 2022
  15 Hours, 29 Minutes
Invert Hiding All Results Option
  17 Hours, 45 Minutes
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

AVX-512 On

Kernel Notes: Transparent Huge Pages: madvise
Environment Notes: CXXFLAGS="-O3 -march=native -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi -mprefer-vector-width=512" CFLAGS="-O3 -march=native -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi -mprefer-vector-width=512"
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-U8K4Qv/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-U8K4Qv/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa10110d
Python Notes: Python 3.10.7
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

AVX-512 Off

Processor: 2 x AMD EPYC 9654 96-Core @ 2.40GHz (192 Cores / 384 Threads), Motherboard: AMD Titanite_4G (RTI1002E BIOS), Chipset: AMD Device 14a4, Memory: 1520GB, Disk: 800GB INTEL SSDPF21Q800GB, Graphics: ASPEED, Monitor: VGA HDMI, Network: Broadcom NetXtreme BCM5720 PCIe

OS: Ubuntu 22.10, Kernel: 6.1.0-phx (x86_64), Desktop: GNOME Shell 43.0, Display Server: X Server 1.21.1.4, Vulkan: 1.3.224, Compiler: GCC 12.2.0 + Clang 15.0.2-1, File-System: ext4, Screen Resolution: 1920x1080

Kernel Notes: Transparent Huge Pages: madvise
Environment Notes: CXXFLAGS="-O3 -march=native -mno-avx512f" CFLAGS="-O3 -march=native -mno-avx512f"
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-U8K4Qv/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-U8K4Qv/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa10110d
Python Notes: Python 3.10.7
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

AMD EPYC 4th Gen AVX-512 ComparisonOpenBenchmarking.orgPhoronix Test Suite2 x AMD EPYC 9654 96-Core @ 2.40GHz (192 Cores / 384 Threads)AMD Titanite_4G (RTI1002E BIOS)AMD Device 14a41520GB800GB INTEL SSDPF21Q800GBASPEEDVGA HDMIBroadcom NetXtreme BCM5720 PCIeUbuntu 22.106.1.0-phx (x86_64)GNOME Shell 43.0X Server 1.21.1.41.3.224GCC 12.2.0 + Clang 15.0.2-1ext41920x1080ProcessorMotherboardChipsetMemoryDiskGraphicsMonitorNetworkOSKernelDesktopDisplay ServerVulkanCompilerFile-SystemScreen ResolutionAMD EPYC 4th Gen AVX-512 Comparison BenchmarksSystem Logs- Transparent Huge Pages: madvise- AVX-512 On: CXXFLAGS="-O3 -march=native -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi -mprefer-vector-width=512" CFLAGS="-O3 -march=native -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi -mprefer-vector-width=512" - AVX-512 Off: CXXFLAGS="-O3 -march=native -mno-avx512f" CFLAGS="-O3 -march=native -mno-avx512f" - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-U8K4Qv/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-U8K4Qv/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa10110d - Python 3.10.7- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

AVX-512 On vs. AVX-512 Off ComparisonPhoronix Test SuiteBaseline+46.4%+46.4%+92.8%+92.8%+139.2%+139.2%50.5%37.6%3.6%CPU - 16 - AlexNet185.4%R.N.N.T - bf16bf16bf16 - CPU155.1%D.T.S153.4%R.N.N.T - f32 - CPU152.5%W.P.D.F - CPU143.4%W.P.D.F - CPU143%F.D.F - CPU132.2%F.D.F - CPU131.9%A.G.R.R.0.F - CPU120%LBC, LBRY Credits117%Device AI Score115.1%M.T.E.T.D.F - CPU114.8%M.T.E.T.D.F - CPU114.7%W.P.D.F.I - CPU104.7%W.P.D.F.I - CPU104.5%F.D.F.I - CPU104.2%F.D.F.I - CPU103.9%V.D.F - CPU97.1%V.D.F - CPU97%CPU - 16 - GoogLeNet93.3%D.I.S92.8%V.D.F.I - CPU84.7%V.D.F.I - CPU84.6%D.B.s - f32 - CPU78.2%P.V.B.D.F - CPU76.5%P.V.B.D.F - CPU76.4%CPU - 16 - ResNet-5073.7%gravity_spheres_volume/dim_512/scivis/real_time71.4%Q.S.2.P70.9%P.D.F - CPU70%P.D.F - CPU69.5%P.D.F - CPU68.4%P.D.F - CPU67.7%gravity_spheres_volume/dim_512/ao/real_time67.4%scrypt62.9%A.G.R.R.0.F.I - CPU58.3%resnet-v2-5056.1%N.Q.A.B.b.u.S.1.P - A.M.S55.2%N.Q.A.B.b.u.S.1.P - A.M.S55.2%inception-v3Garlicoin46.5%OpenMP - BM144.1%OpenMP - BM144.1%N.Q.A.B.b.u.S.1.P - S.S.S41.5%N.Q.A.B.b.u.S.1.P - S.S.S41.5%C.C.R.5.I - A.M.S38.5%C.C.R.5.I - A.M.S38.4%R.N.N.I - bf16bf16bf16 - CPUA.G.R.R.0.F - CPU37.4%Skeincoin36.7%OpenMP - BM235.4%OpenMP - BM235.4%gravity_spheres_volume/dim_512/pathtracer/real_time33.8%x25x33.1%DistinctUserID30.7%PartialTweets30.4%C.D.Y.C - A.M.S24.1%C.D.Y.C - A.M.S24%Kostya23.3%2 - 1080p - 32 - Path Tracer23%2 - 4K - 16 - Path Tracer22.8%2 - 1080p - 16 - Path Tracer22.7%1 - 1080p - 32 - Path Tracer22.4%1 - 1080p - 1 - Path Tracer22.3%2 - 4K - 32 - Path Tracer22.2%1 - 4K - 1 - Path Tracer22.2%1 - 4K - 32 - Path Tracer22.1%1 - 4K - 16 - Path Tracer22.1%1 - 1080p - 16 - Path Tracer22%3 - 1080p - 16 - Path Tracer21.7%3 - 1080p - 32 - Path Tracer21.6%TopTweet21.2%3 - 4K - 1 - Path Tracer21.2%LargeRand21.2%N.T.C.B.b.u.S - A.M.S21%3 - 4K - 16 - Path Tracer21%N.T.C.B.b.u.S - A.M.S20.9%3 - 1080p - 1 - Path Tracer20.9%3 - 4K - 32 - Path Tracer20.7%Pathtracer ISPC - Asian Dragon20.4%Pathtracer ISPC - Asian Dragon Obj20.2%C.D.Y.C - S.S.S20.1%C.D.Y.C - S.S.S20.1%Pathtracer ISPC - Crown19.8%2 - 4K - 1 - Path Tracer19.5%2 - 1080p - 1 - Path Tracer19.5%N.T.C.D.m - A.M.S19%d.M.M.S - Execution Time19%N.T.C.D.m - A.M.S19%OpenMP - Points2Image18.7%super-resolution-10 - CPU - Standard17.7%N.T.C.B.b.u.S - S.S.S15.6%N.T.C.B.b.u.S - S.S.S15.6%vklBenchmark ISPC15.3%CPU - vision_transformer15%ArcFace ResNet-100 - CPU - Standard14.5%A.G.R.R.0.F.I - CPU12.8%N.T.C.B.b.u.c - S.S.S12.8%N.T.C.B.b.u.c - S.S.S12.8%N.D.C.o.b.u.o.I - S.S.S12.2%N.D.C.o.b.u.o.I - S.S.S12.2%CPU - blazeface12.1%Eigen11.4%CPU - regnety_400m9.2%F.x.A9.1%CPU - efficientnet-b08.9%fcn-resnet101-11 - CPU - Standard8.4%BLAS7.8%Fayalite-FIST6.7%SqueezeNetV1.06.6%d.M.M.S - Mesh Time6.3%N.T.C.D.m - S.S.S5.4%N.T.C.D.m - S.S.S5.4%Preset 12 - Bosphorus 4K5%4.9%JPEG - 904.6%CPU - googlenet4.6%PNG - 904.5%CPU - mnasnet4.5%B.C4.3%C.B.S.A - f32 - CPU4.2%RTLightmap.hdr.4096x4096bertsquad-12 - CPU - Standard3.4%Windowed Gaussian3.3%CPU - FastestDet3%Preset 13 - Bosphorus 4K2.8%JPEG - 1002.8%OpenMP - NDT Mapping2.7%TensorFlowoneDNNAI Benchmark AlphaoneDNNOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOCpuminer-OptAI Benchmark AlphaOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOTensorFlowAI Benchmark AlphaOpenVINOOpenVINOoneDNNOpenVINOOpenVINOTensorFlowOSPRayCpuminer-OptOpenVINOOpenVINOOpenVINOOpenVINOOSPRayCpuminer-OptOpenVINOMobile Neural NetworkNeural Magic DeepSparseNeural Magic DeepSparseMobile Neural NetworkCpuminer-OptminiBUDEminiBUDENeural Magic DeepSparseNeural Magic DeepSparseNeural Magic DeepSparseNeural Magic DeepSparseoneDNNOpenVINOCpuminer-OptminiBUDEminiBUDEOSPRayCpuminer-OptsimdjsonsimdjsonNeural Magic DeepSparseNeural Magic DeepSparsesimdjsonOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudiosimdjsonOSPRay StudiosimdjsonNeural Magic DeepSparseOSPRay StudioNeural Magic DeepSparseOSPRay StudioOSPRay StudioEmbreeEmbreeNeural Magic DeepSparseNeural Magic DeepSparseEmbreeOSPRay StudioOSPRay StudioNeural Magic DeepSparseOpenFOAMNeural Magic DeepSparseDarmstadt Automotive Parallel Heterogeneous SuiteONNX RuntimeNeural Magic DeepSparseNeural Magic DeepSparseOpenVKLNCNNONNX RuntimeOpenVINONeural Magic DeepSparseNeural Magic DeepSparseNeural Magic DeepSparseNeural Magic DeepSparseNCNNLeelaChessZeroNCNNSMHasherNCNNONNX RuntimeLeelaChessZeroCP2K Molecular DynamicsMobile Neural NetworkOpenFOAMNeural Magic DeepSparseNeural Magic DeepSparseSVT-AV1Numpy BenchmarkJPEG XL libjxlNCNNJPEG XL libjxlNCNNNumenta Anomaly BenchmarkoneDNNIntel Open Image DenoiseONNX RuntimeNumenta Anomaly BenchmarkNCNNSVT-AV1JPEG XL libjxlDarmstadt Automotive Parallel Heterogeneous SuiteAVX-512 OnAVX-512 Off

AMD EPYC 4th Gen AVX-512 Comparisontensorflow: CPU - 16 - AlexNetonednn: Recurrent Neural Network Training - bf16bf16bf16 - CPUai-benchmark: Device Training Scoreonednn: Recurrent Neural Network Training - f32 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Face Detection FP16 - CPUopenvino: Face Detection FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUcpuminer-opt: LBC, LBRY Creditsai-benchmark: Device AI Scoreopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Vehicle Detection FP16 - CPUopenvino: Vehicle Detection FP16 - CPUai-benchmark: Device Inference Scoreopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUonednn: Deconvolution Batch shapes_3d - f32 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUtensorflow: CPU - 16 - ResNet-50ospray: gravity_spheres_volume/dim_512/scivis/real_timecpuminer-opt: Quad SHA-256, Pyriteopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP32 - CPUopenvino: Person Detection FP32 - CPUospray: gravity_spheres_volume/dim_512/ao/real_timecpuminer-opt: scryptopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUmnn: resnet-v2-50deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Streamdeepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Streammnn: inception-v3cpuminer-opt: Garlicoinminibude: OpenMP - BM1minibude: OpenMP - BM1deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Streamdeepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamonednn: Recurrent Neural Network Inference - bf16bf16bf16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUcpuminer-opt: Skeincoinminibude: OpenMP - BM2minibude: OpenMP - BM2ospray: gravity_spheres_volume/dim_512/pathtracer/real_timecpuminer-opt: x25xsimdjson: DistinctUserIDsimdjson: PartialTweetsdeepsparse: CV Detection,YOLOv5s COCO - Asynchronous Multi-Streamdeepsparse: CV Detection,YOLOv5s COCO - Asynchronous Multi-Streamsimdjson: Kostyaospray-studio: 2 - 1080p - 32 - Path Tracerospray-studio: 2 - 4K - 16 - Path Tracerospray-studio: 2 - 1080p - 16 - Path Tracerospray-studio: 1 - 1080p - 32 - Path Tracerospray-studio: 1 - 1080p - 1 - Path Tracerospray-studio: 2 - 4K - 32 - Path Tracerospray-studio: 1 - 4K - 1 - Path Tracerospray-studio: 1 - 4K - 32 - Path Tracerospray-studio: 1 - 4K - 16 - Path Tracerospray-studio: 1 - 1080p - 16 - Path Tracerospray-studio: 3 - 1080p - 16 - Path Tracerospray-studio: 3 - 1080p - 32 - Path Tracersimdjson: TopTweetospray-studio: 3 - 4K - 1 - Path Tracersimdjson: LargeRanddeepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Streamospray-studio: 3 - 4K - 16 - Path Tracerdeepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Streamospray-studio: 3 - 1080p - 1 - Path Tracerospray-studio: 3 - 4K - 32 - Path Tracerembree: Pathtracer ISPC - Asian Dragonembree: Pathtracer ISPC - Asian Dragon Objdeepsparse: CV Detection,YOLOv5s COCO - Synchronous Single-Streamdeepsparse: CV Detection,YOLOv5s COCO - Synchronous Single-Streamembree: Pathtracer ISPC - Crownospray-studio: 2 - 4K - 1 - Path Tracerospray-studio: 2 - 1080p - 1 - Path Tracerdeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamopenfoam: drivaerFastback, Medium Mesh Size - Execution Timedeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Streamopenvkl: vklBenchmark ISPCopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamncnn: CPU - blazefacelczero: Eigenncnn: CPU - regnety_400msmhasher: FarmHash32 x86_64 AVXncnn: CPU - efficientnet-b0onnx: fcn-resnet101-11 - CPU - Standardlczero: BLAScp2k: Fayalite-FISTmnn: SqueezeNetV1.0openfoam: drivaerFastback, Medium Mesh Size - Mesh Timedeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamnumpy: jpegxl: JPEG - 90ncnn: CPU - googlenetjpegxl: PNG - 90ncnn: CPU - mnasnetnumenta-nab: Bayesian Changepointonednn: Convolution Batch Shapes Auto - f32 - CPUoidn: RTLightmap.hdr.4096x4096onnx: bertsquad-12 - CPU - Standardnumenta-nab: Windowed Gaussianjpegxl: JPEG - 100daphne: OpenMP - NDT Mappingncnn: CPU - resnet50gromacs: MPI CPU - water_GMX50_barenumenta-nab: Relative Entropyonednn: Deconvolution Batch shapes_1d - f32 - CPUoidn: RT.hdr_alb_nrm.3840x2160oidn: RT.ldr_alb_nrm.3840x2160smhasher: FarmHash32 x86_64 AVXdaphne: OpenMP - Points2Imagesvt-av1: Preset 12 - Bosphorus 4Ksvt-av1: Preset 13 - Bosphorus 4Konnx: ArcFace ResNet-100 - CPU - Standardonnx: super-resolution-10 - CPU - Standardncnn: CPU - FastestDetncnn: CPU - vision_transformertensorflow: CPU - 16 - GoogLeNetAVX-512 OnAVX-512 Off157.291955.3927011962.064.799988.44102.04469.290.551066950621150.11956.8819800.209.63193.93246.987452.966.43351011202.624.280.9895919065.345.2922.1543.0711226068043.341100.2543.341101.0044.15664800.610.3615.441761.2241125.786045.812715537299.545291.982100.54589.940849.03281953.02552361.27148967.981951330346.0818652.00554.21877696.066.866.70857.2265111.72824.184764938523844698148188085811854692712353280456086.526941.26616.680411070155.346217722184212.9364184.4701189.56115.2712180.931560415480.0826113.637331195.936784.998811.75861332170652.7134.520628.960934.251829.189225.889096247.3239794.4457.7127190771122.7258.579135.77418187.08375.3413575.669.5572.809.9542.3016.6770.5161671.665164.7270.741371.1866.3418.7649.89822.58753.513.5126.38713677.764494843245.555246.5721051740158.9574.9360.1755.114989.1310664953.9811.664110.4943.941088.501.214917402887107.62445.599672.9419.6994.97503.503782.0812.6718216065.767.901.763365135.569.3312.7525.1354132296025.501864.9325.741845.9526.37942946.080.5724.098490.3658195.165630.436488275065.096202.60471.043714.065567.91181410.80511715.76108449.491427543255.6616391.51940.51175784.255.255.14690.8722138.58613.39586111524292457491812298571022647113212871341368205.388411.04509.446213398187.854021426785176.8831153.4476157.8196.3313151.055872218495.3347135.216481005.161573.538013.59071155151239.6030.606232.664730.526732.750529.008162270.1236483.7062.8625084231198.4089.146144.3478177.53875.6282548.839.1376.139.5244.1917.3890.5376821.724994.8820.721335.6367.4818.46710.03522.83543.503.5126.41211521.38233.944239.881918628860.7386.1631.12OpenBenchmarking.org

TensorFlow

This is a benchmark of the TensorFlow deep learning framework using the TensorFlow reference benchmarks (tensorflow/benchmarks with tf_cnn_benchmarks.py). Note with the Phoronix Test Suite there is also pts/tensorflow-lite for benchmarking the TensorFlow Lite binaries too. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.10Device: CPU - Batch Size: 16 - Model: AlexNetAVX-512 OnAVX-512 Off306090120150SE +/- 2.35, N = 12SE +/- 0.19, N = 3157.2955.11
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.10Device: CPU - Batch Size: 16 - Model: AlexNetAVX-512 OnAVX-512 Off0.11810.23620.35430.47240.59050.5250.173
MinAvgMaxAVX-512 Off369437324340AVX-512 On240034583812OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.10CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On48.9299.7409.2AVX-512 Off169.7318.9350.5OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.10CPU Power Consumption Monitor1102203304405501. AVX-512 On: Approximate power consumption of 4971 Joules per run.2. AVX-512 Off: Approximate power consumption of 11906 Joules per run.
MinAvgMaxAVX-512 On35.040.548.9AVX-512 Off39.041.047.5OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.10CPU Temperature Monitor1428425670

oneDNN

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPUAVX-512 OnAVX-512 Off11002200330044005500SE +/- 19.67, N = 15SE +/- 41.71, N = 31955.394989.13-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 1769.7-mno-avx512f - MIN: 4767.221. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxAVX-512 Off358236274223AVX-512 On240035683704OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 2.7CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 Off195.7412.5498.9AVX-512 On52.5419.2543.1OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 2.7CPU Power Consumption Monitor1402804205607001. AVX-512 Off: Approximate power consumption of 35614 Joules per run.2. AVX-512 On: Approximate power consumption of 34154 Joules per run.
MinAvgMaxAVX-512 Off42.646.650.6AVX-512 On37.846.751.5OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 2.7CPU Temperature Monitor1530456075

AI Benchmark Alpha

AI Benchmark Alpha is a Python library for evaluating artificial intelligence (AI) performance on diverse hardware platforms and relies upon the TensorFlow machine learning library. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgScore, More Is BetterAI Benchmark Alpha 0.1.2Device Training ScoreAVX-512 OnAVX-512 Off600120018002400300027011066

oneDNN

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Recurrent Neural Network Training - Data Type: f32 - Engine: CPUAVX-512 OnAVX-512 Off11002200330044005500SE +/- 20.13, N = 5SE +/- 60.12, N = 41962.064953.98-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 1899.34-mno-avx512f - MIN: 4690.851. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxAVX-512 Off358136233818AVX-512 On240035713711OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 2.7CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 Off215.2412.5499.6AVX-512 On55.5418.6541.1OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 2.7CPU Power Consumption Monitor1402804205607001. AVX-512 Off: Approximate power consumption of 35574 Joules per run.2. AVX-512 On: Approximate power consumption of 34241 Joules per run.
MinAvgMaxAVX-512 Off38.645.950.1AVX-512 On35.546.050.6OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 2.7CPU Temperature Monitor1428425670

OpenVINO

This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Weld Porosity Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off3691215SE +/- 0.01, N = 3SE +/- 0.01, N = 34.7911.66-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 3.98 / MAX: 28.79-mno-avx512f - MIN: 9.72 / MAX: 43.71. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared
MinAvgMaxAVX-512 On240030603695AVX-512 Off267228483946OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 Off221598639AVX-512 On58660712OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.2.devCPU Power Consumption Monitor20040060080010001. AVX-512 Off: Approximate power consumption of 37874 Joules per run.2. AVX-512 On: Approximate power consumption of 42022 Joules per run.
MinAvgMaxAVX-512 Off49.860.061.8AVX-512 On43.060.964.5OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.2.devCPU Temperature Monitor20406080100

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Weld Porosity Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off2K4K6K8K10KSE +/- 6.96, N = 3SE +/- 4.29, N = 39988.444110.49-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Face Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off20406080100SE +/- 0.07, N = 3SE +/- 0.01, N = 3102.0443.94-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Face Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off2004006008001000SE +/- 0.24, N = 3SE +/- 0.51, N = 3469.291088.50-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 410.93 / MAX: 547.33-mno-avx512f - MIN: 937.5 / MAX: 1231.551. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared
MinAvgMaxAVX-512 On240031263696AVX-512 Off265229064086OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 Off218585645AVX-512 On54639725OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.2.devCPU Power Consumption Monitor20040060080010001. AVX-512 Off: Approximate power consumption of 39977 Joules per run.2. AVX-512 On: Approximate power consumption of 43894 Joules per run.
MinAvgMaxAVX-512 Off41.356.261.9AVX-512 On35.957.364.1OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.2.devCPU Temperature Monitor20406080100

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Age Gender Recognition Retail 0013 FP16 - Device: CPUAVX-512 OnAVX-512 Off0.27230.54460.81691.08921.3615SE +/- 0.00, N = 3SE +/- 0.01, N = 40.551.21-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 0.5 / MAX: 29.57-mno-avx512f - MIN: 0.98 / MAX: 47.371. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared
MinAvgMaxAVX-512 On240036053828AVX-512 Off299231934056OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On48566622AVX-512 Off219592636OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.2.devCPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 36764 Joules per run.2. AVX-512 Off: Approximate power consumption of 38208 Joules per run.
MinAvgMaxAVX-512 On42.155.459.5AVX-512 Off49.359.962.6OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.2.devCPU Temperature Monitor20406080100

Cpuminer-Opt

Cpuminer-Opt is a fork of cpuminer-multi that carries a wide range of CPU performance optimizations for measuring the potential cryptocurrency mining performance of the CPU/processor with a wide variety of cryptocurrencies. The benchmark reports the hash speed for the CPU mining performance for the selected cryptocurrency. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: LBC, LBRY CreditsAVX-512 OnAVX-512 Off200K400K600K800K1000KSE +/- 7645.47, N = 3SE +/- 667.26, N = 31066950491740-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -lcurl -lz -lpthread -lssl -lcrypto -lgmp
OpenBenchmarking.orgkH/s Per Watt, More Is BetterCpuminer-Opt 3.20.3Algorithm: LBC, LBRY CreditsAVX-512 OnAVX-512 Off4008001200160020001658.51802.76
MinAvgMaxAVX-512 On240030603703AVX-512 Off272929334111OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 3.20.3CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 Off222613684AVX-512 On59643743OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 3.20.3CPU Power Consumption Monitor20040060080010001. AVX-512 Off: Approximate power consumption of 20623 Joules per run.2. AVX-512 On: Approximate power consumption of 21873 Joules per run.
MinAvgMaxAVX-512 On42.459.063.5AVX-512 Off47.059.262.3OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 3.20.3CPU Temperature Monitor20406080100

AI Benchmark Alpha

AI Benchmark Alpha is a Python library for evaluating artificial intelligence (AI) performance on diverse hardware platforms and relies upon the TensorFlow machine learning library. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgScore, More Is BetterAI Benchmark Alpha 0.1.2Device AI ScoreAVX-512 OnAVX-512 Off1300260039005200650062112887
OpenBenchmarking.orgScore Per Watt, More Is BetterAI Benchmark Alpha 0.1.2Device AI ScoreAVX-512 OnAVX-512 Off51015202520.5229.349
MinAvgMaxAVX-512 Off306837104285AVX-512 On240036554107OpenBenchmarking.orgMegahertz, More Is BetterAI Benchmark Alpha 0.1.2CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On52.9302.7502.0AVX-512 Off130.7308.8452.7OpenBenchmarking.orgWatts, Fewer Is BetterAI Benchmark Alpha 0.1.2CPU Power Consumption Monitor130260390520650
MinAvgMaxAVX-512 On30.139.247.0AVX-512 Off36.440.649.3OpenBenchmarking.orgCelsius, Fewer Is BetterAI Benchmark Alpha 0.1.2CPU Temperature Monitor1428425670

OpenVINO

This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Machine Translation EN To DE FP16 - Device: CPUAVX-512 OnAVX-512 Off20406080100SE +/- 0.21, N = 3SE +/- 0.74, N = 350.11107.62-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 38.98 / MAX: 166.69-mno-avx512f - MIN: 83.48 / MAX: 216.071. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared
MinAvgMaxAVX-512 On240032563707AVX-512 Off259229663926OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 Off213595648AVX-512 On57636701OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.2.devCPU Power Consumption Monitor20040060080010001. AVX-512 Off: Approximate power consumption of 38656 Joules per run.2. AVX-512 On: Approximate power consumption of 41369 Joules per run.
MinAvgMaxAVX-512 Off46.156.361.4AVX-512 On42.658.461.9OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.2.devCPU Temperature Monitor20406080100

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Machine Translation EN To DE FP16 - Device: CPUAVX-512 OnAVX-512 Off2004006008001000SE +/- 4.00, N = 3SE +/- 3.04, N = 3956.88445.59-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Weld Porosity Detection FP16-INT8 - Device: CPUAVX-512 OnAVX-512 Off4K8K12K16K20KSE +/- 12.72, N = 3SE +/- 1.81, N = 319800.209672.94-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Weld Porosity Detection FP16-INT8 - Device: CPUAVX-512 OnAVX-512 Off510152025SE +/- 0.01, N = 3SE +/- 0.01, N = 39.6319.69-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 8.29 / MAX: 57.53-mno-avx512f - MIN: 16.4 / MAX: 74.851. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared
MinAvgMaxAVX-512 Off293030703969AVX-512 On240030373696OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On57639696AVX-512 Off216684729OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.2.devCPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 41120 Joules per run.2. AVX-512 Off: Approximate power consumption of 43997 Joules per run.
MinAvgMaxAVX-512 On42.659.462.8AVX-512 Off45.863.067.1OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.2.devCPU Temperature Monitor20406080100

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Face Detection FP16-INT8 - Device: CPUAVX-512 OnAVX-512 Off4080120160200SE +/- 0.05, N = 3SE +/- 0.02, N = 3193.9394.97-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Face Detection FP16-INT8 - Device: CPUAVX-512 OnAVX-512 Off110220330440550SE +/- 0.01, N = 3SE +/- 0.07, N = 3246.98503.50-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 204.5 / MAX: 299.54-mno-avx512f - MIN: 402.47 / MAX: 569.651. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared
MinAvgMaxAVX-512 On240031293696AVX-512 Off292030813713OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On58612687AVX-512 Off216662733OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.2.devCPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 41609 Joules per run.2. AVX-512 Off: Approximate power consumption of 45253 Joules per run.
MinAvgMaxAVX-512 On43.058.161.9AVX-512 Off47.062.566.8OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.2.devCPU Temperature Monitor20406080100

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Vehicle Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off16003200480064008000SE +/- 3.52, N = 3SE +/- 10.17, N = 37452.963782.08-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Vehicle Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off3691215SE +/- 0.00, N = 3SE +/- 0.03, N = 36.4312.67-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 5.22 / MAX: 61.26-mno-avx512f - MIN: 9.58 / MAX: 77.911. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared
MinAvgMaxAVX-512 On240029723706AVX-512 Off257127304230OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 Off217613655AVX-512 On59633690OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.2.devCPU Power Consumption Monitor20040060080010001. AVX-512 Off: Approximate power consumption of 39059 Joules per run.2. AVX-512 On: Approximate power consumption of 40737 Joules per run.
MinAvgMaxAVX-512 On42.159.062.4AVX-512 Off48.959.962.3OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.2.devCPU Temperature Monitor20406080100

AI Benchmark Alpha

AI Benchmark Alpha is a Python library for evaluating artificial intelligence (AI) performance on diverse hardware platforms and relies upon the TensorFlow machine learning library. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgScore, More Is BetterAI Benchmark Alpha 0.1.2Device Inference ScoreAVX-512 OnAVX-512 Off800160024003200400035101821

OpenVINO

This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Vehicle Detection FP16-INT8 - Device: CPUAVX-512 OnAVX-512 Off2K4K6K8K10KSE +/- 2.53, N = 3SE +/- 1.39, N = 311202.626065.76-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Vehicle Detection FP16-INT8 - Device: CPUAVX-512 OnAVX-512 Off246810SE +/- 0.00, N = 3SE +/- 0.00, N = 34.287.90-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 3.5 / MAX: 42.62-mno-avx512f - MIN: 6.37 / MAX: 41.531. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared
MinAvgMaxAVX-512 Off298830684270AVX-512 On240030363900OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On33598646AVX-512 Off214634676OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.2.devCPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 38248 Joules per run.2. AVX-512 Off: Approximate power consumption of 40552 Joules per run.
MinAvgMaxAVX-512 On44.457.960.1AVX-512 Off48.061.164.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.2.devCPU Temperature Monitor20406080100

oneDNN

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPUAVX-512 OnAVX-512 Off0.39680.79361.19041.58721.984SE +/- 0.001906, N = 9SE +/- 0.004550, N = 90.9895911.763360-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 0.9-mno-avx512f - MIN: 1.551. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxAVX-512 Off356936664064AVX-512 On240030083695OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 2.7CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On54.0248.8512.8AVX-512 Off215.2323.9475.6OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 2.7CPU Power Consumption Monitor1302603905206501. AVX-512 On: Approximate power consumption of 1714 Joules per run.2. AVX-512 Off: Approximate power consumption of 2267 Joules per run.
MinAvgMaxAVX-512 On35.939.344.4AVX-512 Off39.041.443.9OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 2.7CPU Temperature Monitor1224364860

OpenVINO

This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Person Vehicle Bike Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off2K4K6K8K10KSE +/- 4.01, N = 3SE +/- 2.17, N = 39065.345135.56-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Person Vehicle Bike Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off3691215SE +/- 0.00, N = 3SE +/- 0.00, N = 35.299.33-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 4.36 / MAX: 45.25-mno-avx512f - MIN: 7.39 / MAX: 57.571. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared
MinAvgMaxAVX-512 On240030993699AVX-512 Off279929423941OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 Off218588640AVX-512 On51588657OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.2.devCPU Power Consumption Monitor20040060080010001. AVX-512 Off: Approximate power consumption of 38981 Joules per run.2. AVX-512 On: Approximate power consumption of 39006 Joules per run.
MinAvgMaxAVX-512 On43.856.859.6AVX-512 Off46.658.061.4OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.2.devCPU Temperature Monitor20406080100

TensorFlow

This is a benchmark of the TensorFlow deep learning framework using the TensorFlow reference benchmarks (tensorflow/benchmarks with tf_cnn_benchmarks.py). Note with the Phoronix Test Suite there is also pts/tensorflow-lite for benchmarking the TensorFlow Lite binaries too. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.10Device: CPU - Batch Size: 16 - Model: ResNet-50AVX-512 OnAVX-512 Off510152025SE +/- 0.11, N = 3SE +/- 0.01, N = 322.1512.75
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.10Device: CPU - Batch Size: 16 - Model: ResNet-50AVX-512 OnAVX-512 Off0.01620.03240.04860.06480.0810.0720.038
MinAvgMaxAVX-512 Off369437134254AVX-512 On240036644099OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.10CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On53.5307.1397.4AVX-512 Off165.7338.8379.7OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.10CPU Power Consumption Monitor1102203304405501. AVX-512 On: Approximate power consumption of 26926 Joules per run.2. AVX-512 Off: Approximate power consumption of 49350 Joules per run.
MinAvgMaxAVX-512 On36.841.748.6AVX-512 Off40.943.453.5OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.10CPU Temperature Monitor1530456075

OSPRay

Intel OSPRay is a portable ray-tracing engine for high-performance, high-fidelity scientific visualizations. OSPRay builds off Intel's Embree and Intel SPMD Program Compiler (ISPC) components as part of the oneAPI rendering toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 2.10Benchmark: gravity_spheres_volume/dim_512/scivis/real_timeAVX-512 OnAVX-512 Off1020304050SE +/- 0.10, N = 3SE +/- 0.04, N = 343.0725.14
OpenBenchmarking.orgItems Per Second Per Watt, More Is BetterOSPRay 2.10Benchmark: gravity_spheres_volume/dim_512/scivis/real_timeAVX-512 OnAVX-512 Off0.01670.03340.05010.06680.08350.0740.041
MinAvgMaxAVX-512 Off276830834016AVX-512 On240030513695OpenBenchmarking.orgMegahertz, More Is BetterOSPRay 2.10CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On56582632AVX-512 Off177620662OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay 2.10CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 68662 Joules per run.2. AVX-512 Off: Approximate power consumption of 84280 Joules per run.
MinAvgMaxAVX-512 On41.856.860.4AVX-512 Off48.460.363.5OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay 2.10CPU Temperature Monitor20406080100

Cpuminer-Opt

Cpuminer-Opt is a fork of cpuminer-multi that carries a wide range of CPU performance optimizations for measuring the potential cryptocurrency mining performance of the CPU/processor with a wide variety of cryptocurrencies. The benchmark reports the hash speed for the CPU mining performance for the selected cryptocurrency. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: Quad SHA-256, PyriteAVX-512 OnAVX-512 Off500K1000K1500K2000K2500KSE +/- 9040.25, N = 3SE +/- 276.83, N = 322606801322960-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -lcurl -lz -lpthread -lssl -lcrypto -lgmp
OpenBenchmarking.orgkH/s Per Watt, More Is BetterCpuminer-Opt 3.20.3Algorithm: Quad SHA-256, PyriteAVX-512 OnAVX-512 Off80016002400320040003657.532252.99
MinAvgMaxAVX-512 Off300131554285AVX-512 On240031123695OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 3.20.3CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 Off134587654AVX-512 On57618724OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 3.20.3CPU Power Consumption Monitor20040060080010001. AVX-512 Off: Approximate power consumption of 19965 Joules per run.2. AVX-512 On: Approximate power consumption of 20809 Joules per run.
MinAvgMaxAVX-512 Off46.657.961.4AVX-512 On43.058.662.8OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 3.20.3CPU Temperature Monitor20406080100

OpenVINO

This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Person Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off1020304050SE +/- 0.20, N = 3SE +/- 0.22, N = 343.3425.50-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Person Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off400800120016002000SE +/- 4.83, N = 3SE +/- 15.65, N = 31100.251864.93-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 783.9 / MAX: 1824.92-mno-avx512f - MIN: 1371.29 / MAX: 2532.741. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared
MinAvgMaxAVX-512 On240034803726AVX-512 Off270931503863OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 Off114579650AVX-512 On56605699OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.2.devCPU Power Consumption Monitor20040060080010001. AVX-512 Off: Approximate power consumption of 41080 Joules per run.2. AVX-512 On: Approximate power consumption of 42116 Joules per run.
MinAvgMaxAVX-512 On39.557.465.9AVX-512 Off47.058.267.6OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.2.devCPU Temperature Monitor20406080100

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Person Detection FP32 - Device: CPUAVX-512 OnAVX-512 Off1020304050SE +/- 0.18, N = 3SE +/- 0.04, N = 343.3425.74-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Person Detection FP32 - Device: CPUAVX-512 OnAVX-512 Off400800120016002000SE +/- 4.44, N = 3SE +/- 3.32, N = 31101.001845.95-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 840.82 / MAX: 1792.51-mno-avx512f - MIN: 1386.31 / MAX: 27981. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared
MinAvgMaxAVX-512 On240034663698AVX-512 Off272931003993OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 Off215593665AVX-512 On38607700OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.2.devCPU Power Consumption Monitor20040060080010001. AVX-512 Off: Approximate power consumption of 41729 Joules per run.2. AVX-512 On: Approximate power consumption of 41912 Joules per run.
MinAvgMaxAVX-512 On42.158.067.3AVX-512 Off46.158.766.8OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.2.devCPU Temperature Monitor20406080100

OSPRay

Intel OSPRay is a portable ray-tracing engine for high-performance, high-fidelity scientific visualizations. OSPRay builds off Intel's Embree and Intel SPMD Program Compiler (ISPC) components as part of the oneAPI rendering toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 2.10Benchmark: gravity_spheres_volume/dim_512/ao/real_timeAVX-512 OnAVX-512 Off1020304050SE +/- 0.21, N = 3SE +/- 0.02, N = 344.1626.38
OpenBenchmarking.orgItems Per Second Per Watt, More Is BetterOSPRay 2.10Benchmark: gravity_spheres_volume/dim_512/ao/real_timeAVX-512 OnAVX-512 Off0.01710.03420.05130.06840.08550.0760.043
MinAvgMaxAVX-512 On240030813695AVX-512 Off277130734018OpenBenchmarking.orgMegahertz, More Is BetterOSPRay 2.10CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On56580633AVX-512 Off222618662OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay 2.10CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 68242 Joules per run.2. AVX-512 Off: Approximate power consumption of 82804 Joules per run.
MinAvgMaxAVX-512 On39.956.259.5AVX-512 Off45.359.865.0OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay 2.10CPU Temperature Monitor20406080100

Cpuminer-Opt

Cpuminer-Opt is a fork of cpuminer-multi that carries a wide range of CPU performance optimizations for measuring the potential cryptocurrency mining performance of the CPU/processor with a wide variety of cryptocurrencies. The benchmark reports the hash speed for the CPU mining performance for the selected cryptocurrency. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: scryptAVX-512 OnAVX-512 Off10002000300040005000SE +/- 0.45, N = 3SE +/- 0.31, N = 34800.612946.08-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -lcurl -lz -lpthread -lssl -lcrypto -lgmp
OpenBenchmarking.orgkH/s Per Watt, More Is BetterCpuminer-Opt 3.20.3Algorithm: scryptAVX-512 OnAVX-512 Off2468108.0765.279
MinAvgMaxAVX-512 Off293330583896AVX-512 On240030093697OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 3.20.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 Off215558618AVX-512 On55594682OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 3.20.3CPU Power Consumption Monitor20040060080010001. AVX-512 Off: Approximate power consumption of 18973 Joules per run.2. AVX-512 On: Approximate power consumption of 20013 Joules per run.
MinAvgMaxAVX-512 On34.152.258.3AVX-512 Off39.552.456.9OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 3.20.3CPU Temperature Monitor1632486480

OpenVINO

This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUAVX-512 OnAVX-512 Off0.12830.25660.38490.51320.6415SE +/- 0.00, N = 3SE +/- 0.00, N = 30.360.57-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 0.34 / MAX: 33.7-mno-avx512f - MIN: 0.52 / MAX: 47.931. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared
MinAvgMaxAVX-512 Off347636743707AVX-512 On240036303754OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On29514564AVX-512 Off217592630OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.2.devCPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 33242 Joules per run.2. AVX-512 Off: Approximate power consumption of 38057 Joules per run.
MinAvgMaxAVX-512 On41.852.255.1AVX-512 Off48.059.262.3OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.2.devCPU Temperature Monitor20406080100

Mobile Neural Network

MNN is the Mobile Neural Network as a highly efficient, lightweight deep learning framework developed by Alibaba. This MNN test profile is building the OpenMP / CPU threaded version for processor benchmarking and not any GPU-accelerated test. MNN does allow making use of AVX-512 extensions. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 2.1Model: resnet-v2-50AVX-512 OnAVX-512 Off612182430SE +/- 0.08, N = 9SE +/- 0.08, N = 815.4424.10-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 14.79 / MAX: 54.05-mno-avx512f - MIN: 23.44 / MAX: 71.321. (CXX) g++ options: -O3 -march=native -std=c++11 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl

Neural Magic DeepSparse

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off160320480640800SE +/- 0.79, N = 3SE +/- 0.61, N = 3761.22490.37

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off4080120160200SE +/- 0.11, N = 3SE +/- 0.23, N = 3125.79195.17
MinAvgMaxAVX-512 Off256832144450AVX-512 On240029453885OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.1CPU Peak Freq (Highest CPU Core Frequency) Monitor12002400360048006000
MinAvgMaxAVX-512 Off114503700AVX-512 On54537700OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Power Consumption Monitor20040060080010001. AVX-512 Off: Approximate power consumption of 38526 Joules per run.2. AVX-512 On: Approximate power consumption of 28998 Joules per run.
MinAvgMaxAVX-512 Off43.551.559.1AVX-512 On39.351.958.3OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Temperature Monitor1632486480

Mobile Neural Network

MNN is the Mobile Neural Network as a highly efficient, lightweight deep learning framework developed by Alibaba. This MNN test profile is building the OpenMP / CPU threaded version for processor benchmarking and not any GPU-accelerated test. MNN does allow making use of AVX-512 extensions. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 2.1Model: inception-v3AVX-512 OffAVX-512 On1020304050SE +/- 0.13, N = 8SE +/- 0.23, N = 930.4445.81-mno-avx512f - MIN: 28.75 / MAX: 109.22-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 44.13 / MAX: 87.281. (CXX) g++ options: -O3 -march=native -std=c++11 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl
OpenBenchmarking.orgMegahertz, More Is BetterMobile Neural Network 2.1CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 OffAVX-512 On7001400210028003500Min: 3694 / Avg: 3699.72 / Max: 4041Min: 2400 / Avg: 3679.28 / Max: 3704
OpenBenchmarking.orgWatts, Fewer Is BetterMobile Neural Network 2.1CPU Power Consumption MonitorAVX-512 OnAVX-512 Off60120180240300Min: 53.81 / Avg: 327.57 / Max: 340Min: 165 / Avg: 334.93 / Max: 346.271. AVX-512 On: Approximate power consumption of 84551 Joules per run.2. AVX-512 Off: Approximate power consumption of 79001 Joules per run.
OpenBenchmarking.orgCelsius, Fewer Is BetterMobile Neural Network 2.1CPU Temperature MonitorAVX-512 OnAVX-512 Off1122334455Min: 35.88 / Avg: 48.36 / Max: 52Min: 40.38 / Avg: 49.52 / Max: 52.88

Cpuminer-Opt

Cpuminer-Opt is a fork of cpuminer-multi that carries a wide range of CPU performance optimizations for measuring the potential cryptocurrency mining performance of the CPU/processor with a wide variety of cryptocurrencies. The benchmark reports the hash speed for the CPU mining performance for the selected cryptocurrency. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: GarlicoinAVX-512 OnAVX-512 Off15K30K45K60K75KSE +/- 89.69, N = 3SE +/- 13.33, N = 37155348827-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -lcurl -lz -lpthread -lssl -lcrypto -lgmp
OpenBenchmarking.orgkH/s Per Watt, More Is BetterCpuminer-Opt 3.20.3Algorithm: GarlicoinAVX-512 OnAVX-512 Off306090120150119.9783.22
MinAvgMaxAVX-512 On240030793830AVX-512 Off287830303783OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 3.20.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 Off220587657AVX-512 On57596694OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 3.20.3CPU Power Consumption Monitor20040060080010001. AVX-512 Off: Approximate power consumption of 19754 Joules per run.2. AVX-512 On: Approximate power consumption of 19881 Joules per run.
MinAvgMaxAVX-512 On41.356.360.9AVX-512 Off45.857.160.9OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 3.20.3CPU Temperature Monitor20406080100

miniBUDE

MiniBUDE is a mini application for the the core computation of the Bristol University Docking Engine (BUDE). This test profile currently makes use of the OpenMP implementation of miniBUDE for CPU benchmarking. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgGFInst/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM1AVX-512 OnAVX-512 Off16003200480064008000SE +/- 5.79, N = 10SE +/- 26.42, N = 87299.555065.101. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm

OpenBenchmarking.orgBillion Interactions/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM1AVX-512 OnAVX-512 Off60120180240300SE +/- 0.23, N = 10SE +/- 1.06, N = 8291.98202.601. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm
OpenBenchmarking.orgBillion Interactions/s Per Watt, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM1AVX-512 OnAVX-512 Off0.23420.46840.70260.93681.1711.0410.489
MinAvgMaxAVX-512 Off310735054044AVX-512 On240029133739OpenBenchmarking.orgMegahertz, More Is BetterminiBUDE 20210901CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On52281624AVX-512 Off212414630OpenBenchmarking.orgWatts, Fewer Is BetterminiBUDE 20210901CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 1880 Joules per run.2. AVX-512 Off: Approximate power consumption of 3313 Joules per run.
MinAvgMaxAVX-512 On31.538.547.4AVX-512 Off38.645.552.4OpenBenchmarking.orgCelsius, Fewer Is BetterminiBUDE 20210901CPU Temperature Monitor1530456075

Neural Magic DeepSparse

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off20406080100SE +/- 0.17, N = 3SE +/- 0.17, N = 3100.5571.04

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off48121620SE +/- 0.0172, N = 3SE +/- 0.0334, N = 39.940814.0655
MinAvgMaxAVX-512 Off367837024150AVX-512 On240035713764OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.1CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On57.7334.2406.1AVX-512 Off124.0352.9412.6OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Power Consumption Monitor1102203304405501. AVX-512 On: Approximate power consumption of 16262 Joules per run.2. AVX-512 Off: Approximate power consumption of 17645 Joules per run.
MinAvgMaxAVX-512 On38.148.554.8AVX-512 Off43.050.656.9OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Temperature Monitor1632486480

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off1530456075SE +/- 0.11, N = 3SE +/- 0.02, N = 349.0367.91
MinAvgMaxAVX-512 On240030033730AVX-512 Off255229383959OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On56531630AVX-512 Off216558637OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 23349 Joules per run.2. AVX-512 Off: Approximate power consumption of 24384 Joules per run.
MinAvgMaxAVX-512 On38.151.957.3AVX-512 Off44.454.959.1OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Temperature Monitor1632486480

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off400800120016002000SE +/- 4.32, N = 3SE +/- 0.17, N = 31953.031410.81

oneDNN

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPUAVX-512 OffAVX-512 On5001000150020002500SE +/- 22.32, N = 3SE +/- 24.41, N = 41715.762361.27-mno-avx512f - MIN: 1607.13-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 2265.71. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxAVX-512 Off357036124005AVX-512 On240035073700OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 2.7CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On55.2413.8581.1AVX-512 Off204.0413.9482.1OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 2.7CPU Power Consumption Monitor1603204806408001. AVX-512 On: Approximate power consumption of 34038 Joules per run.2. AVX-512 Off: Approximate power consumption of 32976 Joules per run.
MinAvgMaxAVX-512 On37.845.848.9AVX-512 Off42.646.849.8OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 2.7CPU Temperature Monitor1428425670

OpenVINO

This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Age Gender Recognition Retail 0013 FP16 - Device: CPUAVX-512 OnAVX-512 Off30K60K90K120K150KSE +/- 1328.20, N = 3SE +/- 1130.85, N = 4148967.98108449.49-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

Cpuminer-Opt

Cpuminer-Opt is a fork of cpuminer-multi that carries a wide range of CPU performance optimizations for measuring the potential cryptocurrency mining performance of the CPU/processor with a wide variety of cryptocurrencies. The benchmark reports the hash speed for the CPU mining performance for the selected cryptocurrency. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: SkeincoinAVX-512 OnAVX-512 Off400K800K1200K1600K2000KSE +/- 13005.55, N = 3SE +/- 3964.01, N = 319513301427543-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -lcurl -lz -lpthread -lssl -lcrypto -lgmp
OpenBenchmarking.orgkH/s Per Watt, More Is BetterCpuminer-Opt 3.20.3Algorithm: SkeincoinAVX-512 OnAVX-512 Off70014002100280035003137.562381.94
MinAvgMaxAVX-512 Off289530683848AVX-512 On240030023696OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 3.20.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 Off221599665AVX-512 On58622720OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 3.20.3CPU Power Consumption Monitor20040060080010001. AVX-512 Off: Approximate power consumption of 19778 Joules per run.2. AVX-512 On: Approximate power consumption of 20524 Joules per run.
MinAvgMaxAVX-512 On41.357.962.8AVX-512 Off47.058.761.8OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 3.20.3CPU Temperature Monitor20406080100

miniBUDE

MiniBUDE is a mini application for the the core computation of the Bristol University Docking Engine (BUDE). This test profile currently makes use of the OpenMP implementation of miniBUDE for CPU benchmarking. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgBillion Interactions/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM2AVX-512 OnAVX-512 Off80160240320400SE +/- 0.46, N = 4SE +/- 1.27, N = 3346.08255.661. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm
OpenBenchmarking.orgBillion Interactions/s Per Watt, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM2AVX-512 OnAVX-512 Off0.15620.31240.46860.62480.7810.6940.453
MinAvgMaxAVX-512 Off290532883891AVX-512 On240032013697OpenBenchmarking.orgMegahertz, More Is BetterminiBUDE 20210901CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On54499647AVX-512 Off211564661OpenBenchmarking.orgWatts, Fewer Is BetterminiBUDE 20210901CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 9476 Joules per run.2. AVX-512 Off: Approximate power consumption of 13359 Joules per run.
MinAvgMaxAVX-512 On32.847.756.0AVX-512 Off40.453.060.1OpenBenchmarking.orgCelsius, Fewer Is BetterminiBUDE 20210901CPU Temperature Monitor1632486480

OpenBenchmarking.orgGFInst/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM2AVX-512 OnAVX-512 Off2K4K6K8K10KSE +/- 11.62, N = 4SE +/- 31.65, N = 38652.016391.521. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm

OSPRay

Intel OSPRay is a portable ray-tracing engine for high-performance, high-fidelity scientific visualizations. OSPRay builds off Intel's Embree and Intel SPMD Program Compiler (ISPC) components as part of the oneAPI rendering toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 2.10Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_timeAVX-512 OnAVX-512 Off1224364860SE +/- 0.05, N = 3SE +/- 0.03, N = 354.2240.51
OpenBenchmarking.orgItems Per Second Per Watt, More Is BetterOSPRay 2.10Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_timeAVX-512 OnAVX-512 Off0.02050.0410.06150.0820.10250.0910.065
MinAvgMaxAVX-512 Off276631044475AVX-512 On240030553720OpenBenchmarking.orgMegahertz, More Is BetterOSPRay 2.10CPU Peak Freq (Highest CPU Core Frequency) Monitor12002400360048006000
MinAvgMaxAVX-512 On58593646AVX-512 Off221628677OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay 2.10CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 66866 Joules per run.2. AVX-512 Off: Approximate power consumption of 75548 Joules per run.
MinAvgMaxAVX-512 On42.957.860.9AVX-512 Off47.561.364.9OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay 2.10CPU Temperature Monitor20406080100

Cpuminer-Opt

Cpuminer-Opt is a fork of cpuminer-multi that carries a wide range of CPU performance optimizations for measuring the potential cryptocurrency mining performance of the CPU/processor with a wide variety of cryptocurrencies. The benchmark reports the hash speed for the CPU mining performance for the selected cryptocurrency. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: x25xAVX-512 OnAVX-512 Off16003200480064008000SE +/- 16.58, N = 3SE +/- 14.41, N = 37696.065784.25-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -lcurl -lz -lpthread -lssl -lcrypto -lgmp
OpenBenchmarking.orgkH/s Per Watt, More Is BetterCpuminer-Opt 3.20.3Algorithm: x25xAVX-512 OnAVX-512 Off369121513.5489.971
MinAvgMaxAVX-512 Off325633514121AVX-512 On240031053697OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 3.20.3CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On39568650AVX-512 Off221580645OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 3.20.3CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 19314 Joules per run.2. AVX-512 Off: Approximate power consumption of 19529 Joules per run.
MinAvgMaxAVX-512 On42.057.160.4AVX-512 Off46.658.260.9OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 3.20.3CPU Temperature Monitor20406080100

simdjson

This is a benchmark of SIMDJSON, a high performance JSON parser. SIMDJSON aims to be the fastest JSON parser and is used by projects like Microsoft FishStore, Yandex ClickHouse, Shopify, and others. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgGB/s, More Is Bettersimdjson 2.0Throughput Test: DistinctUserIDAVX-512 OnAVX-512 Off246810SE +/- 0.02, N = 3SE +/- 0.01, N = 36.865.25-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native
OpenBenchmarking.orgGB/s Per Watt, More Is Bettersimdjson 2.0Throughput Test: DistinctUserIDAVX-512 OnAVX-512 Off0.00740.01480.02220.02960.0370.0330.024
MinAvgMaxAVX-512 Off369437033925AVX-512 On240036433754OpenBenchmarking.orgMegahertz, More Is Bettersimdjson 2.0CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On35.8209.0219.1AVX-512 Off111.3221.2223.7OpenBenchmarking.orgWatts, Fewer Is Bettersimdjson 2.0CPU Power Consumption Monitor601201802403001. AVX-512 On: Approximate power consumption of 15327 Joules per run.2. AVX-512 Off: Approximate power consumption of 15998 Joules per run.
MinAvgMaxAVX-512 On32.436.237.3AVX-512 Off37.346.948.1OpenBenchmarking.orgCelsius, Fewer Is Bettersimdjson 2.0CPU Temperature Monitor1428425670

OpenBenchmarking.orgGB/s, More Is Bettersimdjson 2.0Throughput Test: PartialTweetsAVX-512 OnAVX-512 Off246810SE +/- 0.04, N = 3SE +/- 0.01, N = 36.705.14-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native
OpenBenchmarking.orgGB/s Per Watt, More Is Bettersimdjson 2.0Throughput Test: PartialTweetsAVX-512 OnAVX-512 Off0.00720.01440.02160.02880.0360.0320.023
MinAvgMaxAVX-512 Off369437024016AVX-512 On240036393702OpenBenchmarking.orgMegahertz, More Is Bettersimdjson 2.0CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On50.2211.4225.1AVX-512 Off113.7224.4230.2OpenBenchmarking.orgWatts, Fewer Is Bettersimdjson 2.0CPU Power Consumption Monitor601201802403001. AVX-512 On: Approximate power consumption of 15293 Joules per run.2. AVX-512 Off: Approximate power consumption of 16157 Joules per run.
MinAvgMaxAVX-512 On36.440.547.5AVX-512 Off39.051.060.1OpenBenchmarking.orgCelsius, Fewer Is Bettersimdjson 2.0CPU Temperature Monitor1632486480

Neural Magic DeepSparse

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: CV Detection,YOLOv5s COCO - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off2004006008001000SE +/- 0.77, N = 3SE +/- 1.12, N = 3857.23690.87

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: CV Detection,YOLOv5s COCO - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off306090120150SE +/- 0.12, N = 3SE +/- 0.25, N = 3111.73138.59
MinAvgMaxAVX-512 Off253529083834AVX-512 On240028793730OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 Off217552656AVX-512 On28553664OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Power Consumption Monitor20040060080010001. AVX-512 Off: Approximate power consumption of 25561 Joules per run.2. AVX-512 On: Approximate power consumption of 24689 Joules per run.
MinAvgMaxAVX-512 On40.152.957.9AVX-512 Off44.954.461.8OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Temperature Monitor20406080100

simdjson

This is a benchmark of SIMDJSON, a high performance JSON parser. SIMDJSON aims to be the fastest JSON parser and is used by projects like Microsoft FishStore, Yandex ClickHouse, Shopify, and others. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgGB/s, More Is Bettersimdjson 2.0Throughput Test: KostyaAVX-512 OnAVX-512 Off0.94051.8812.82153.7624.7025SE +/- 0.02, N = 3SE +/- 0.00, N = 34.183.39-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native
OpenBenchmarking.orgGB/s Per Watt, More Is Bettersimdjson 2.0Throughput Test: KostyaAVX-512 OnAVX-512 Off0.00450.0090.01350.0180.02250.0200.015
MinAvgMaxAVX-512 Off369437043981AVX-512 On240036253702OpenBenchmarking.orgMegahertz, More Is Bettersimdjson 2.0CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On54.4209.9221.6AVX-512 Off112.5223.5225.7OpenBenchmarking.orgWatts, Fewer Is Bettersimdjson 2.0CPU Power Consumption Monitor601201802403001. AVX-512 On: Approximate power consumption of 13082 Joules per run.2. AVX-512 Off: Approximate power consumption of 13635 Joules per run.
MinAvgMaxAVX-512 On33.336.437.3AVX-512 Off37.346.348.6OpenBenchmarking.orgCelsius, Fewer Is Bettersimdjson 2.0CPU Temperature Monitor1428425670

OSPRay Studio

Intel OSPRay Studio is an open-source, interactive visualization and ray-tracing software package. OSPRay Studio makes use of Intel OSPRay, a portable ray-tracing engine for high-performance, high-fidelity visualizations. OSPRay builds off Intel's Embree and Intel SPMD Program Compiler (ISPC) components as part of the oneAPI rendering toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 2 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path TracerAVX-512 OnAVX-512 Off13002600390052006500SE +/- 6.24, N = 3SE +/- 8.25, N = 347645861-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off277129723740AVX-512 On240029353696OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On54604686AVX-512 Off224627702OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 85795 Joules per run.2. AVX-512 Off: Approximate power consumption of 92097 Joules per run.
MinAvgMaxAVX-512 On43.858.963.1AVX-512 Off46.661.365.8OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 2 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path TracerAVX-512 OnAVX-512 Off2K4K6K8K10KSE +/- 2.19, N = 3SE +/- 15.51, N = 3938511524-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off272129603862AVX-512 On240029063696OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On57616688AVX-512 Off114635701OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 101207 Joules per run.2. AVX-512 Off: Approximate power consumption of 103785 Joules per run.
MinAvgMaxAVX-512 On43.559.764.0AVX-512 Off46.161.665.4OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 2 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path TracerAVX-512 OnAVX-512 Off6001200180024003000SE +/- 9.17, N = 3SE +/- 0.88, N = 323842924-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off278329933739AVX-512 On240029223700OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On29599684AVX-512 Off116621701OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 76828 Joules per run.2. AVX-512 Off: Approximate power consumption of 80323 Joules per run.
MinAvgMaxAVX-512 On43.858.462.6AVX-512 Off46.660.664.9OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 1 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path TracerAVX-512 OnAVX-512 Off12002400360048006000SE +/- 3.71, N = 3SE +/- 6.56, N = 346985749-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off280229784050AVX-512 On240029043702OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On58606687AVX-512 Off222629702OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 85852 Joules per run.2. AVX-512 Off: Approximate power consumption of 92016 Joules per run.
MinAvgMaxAVX-512 On44.059.064.0AVX-512 Off46.661.368.0OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 1 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path TracerAVX-512 OnAVX-512 Off4080120160200SE +/- 0.33, N = 3SE +/- 0.33, N = 3148181-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off270829973911AVX-512 On240029393695OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On55593680AVX-512 Off218614692OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 67997 Joules per run.2. AVX-512 Off: Approximate power consumption of 72458 Joules per run.
MinAvgMaxAVX-512 On36.456.362.3AVX-512 Off41.858.964.5OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 2 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path TracerAVX-512 OnAVX-512 Off5K10K15K20K25KSE +/- 20.34, N = 3SE +/- 30.02, N = 31880822985-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off272630074349AVX-512 On240029073700OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 Off224609700AVX-512 On60616683OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 Off: Approximate power consumption of 65407 Joules per run.2. AVX-512 On: Approximate power consumption of 101822 Joules per run.
MinAvgMaxAVX-512 On45.359.864.9AVX-512 Off47.559.964.1OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 1 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path TracerAVX-512 OnAVX-512 Off150300450600750SE +/- 1.00, N = 3SE +/- 2.08, N = 3581710-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off269630013902AVX-512 On240029503701OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On51590687AVX-512 Off225610697OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 62301 Joules per run.2. AVX-512 Off: Approximate power consumption of 66137 Joules per run.
MinAvgMaxAVX-512 On44.157.962.3AVX-512 Off48.060.264.0OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 1 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path TracerAVX-512 OnAVX-512 Off5K10K15K20K25KSE +/- 13.75, N = 3SE +/- 27.41, N = 31854622647-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off276929893982AVX-512 On240028763752OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 Off128610701AVX-512 On58623687OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 Off: Approximate power consumption of 64635 Joules per run.2. AVX-512 On: Approximate power consumption of 112763 Joules per run.
MinAvgMaxAVX-512 Off47.560.366.3AVX-512 On44.960.365.4OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 1 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path TracerAVX-512 OnAVX-512 Off2K4K6K8K10KSE +/- 10.17, N = 3SE +/- 24.46, N = 3927111321-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off270929373983AVX-512 On240028883699OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On56618684AVX-512 Off223638703OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 100105 Joules per run.2. AVX-512 Off: Approximate power consumption of 107587 Joules per run.
MinAvgMaxAVX-512 On43.859.764.9AVX-512 Off46.662.065.4OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 1 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path TracerAVX-512 OnAVX-512 Off6001200180024003000SE +/- 2.33, N = 3SE +/- 1.20, N = 323532871-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off269629794041AVX-512 On240029303699OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On56599685AVX-512 Off223622699OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 75109 Joules per run.2. AVX-512 Off: Approximate power consumption of 81244 Joules per run.
MinAvgMaxAVX-512 On43.558.465.0AVX-512 Off46.660.764.9OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 3 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path TracerAVX-512 OnAVX-512 Off7001400210028003500SE +/- 3.06, N = 3SE +/- 7.00, N = 328043413-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off272129714031AVX-512 On240029263699OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On31602684AVX-512 Off223621699OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 78698 Joules per run.2. AVX-512 Off: Approximate power consumption of 81921 Joules per run.
MinAvgMaxAVX-512 On43.858.765.4AVX-512 Off47.060.864.9OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 3 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path TracerAVX-512 OnAVX-512 Off15003000450060007500SE +/- 9.54, N = 3SE +/- 5.51, N = 356086820-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off271529724286AVX-512 On240029233697OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On58611684AVX-512 Off222630700OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 91244 Joules per run.2. AVX-512 Off: Approximate power consumption of 94869 Joules per run.
MinAvgMaxAVX-512 On43.859.364.0AVX-512 Off46.661.366.3OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

simdjson

This is a benchmark of SIMDJSON, a high performance JSON parser. SIMDJSON aims to be the fastest JSON parser and is used by projects like Microsoft FishStore, Yandex ClickHouse, Shopify, and others. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgGB/s, More Is Bettersimdjson 2.0Throughput Test: TopTweetAVX-512 OnAVX-512 Off246810SE +/- 0.08, N = 4SE +/- 0.01, N = 36.525.38-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native
OpenBenchmarking.orgGB/s Per Watt, More Is Bettersimdjson 2.0Throughput Test: TopTweetAVX-512 OnAVX-512 Off0.0070.0140.0210.0280.0350.0310.024
MinAvgMaxAVX-512 Off369437013912AVX-512 On240036403701OpenBenchmarking.orgMegahertz, More Is Bettersimdjson 2.0CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On55.3209.4219.2AVX-512 Off111.2221.5223.4OpenBenchmarking.orgWatts, Fewer Is Bettersimdjson 2.0CPU Power Consumption Monitor601201802403001. AVX-512 On: Approximate power consumption of 15184 Joules per run.2. AVX-512 Off: Approximate power consumption of 16021 Joules per run.
MinAvgMaxAVX-512 On32.436.137.3AVX-512 Off36.846.247.5OpenBenchmarking.orgCelsius, Fewer Is Bettersimdjson 2.0CPU Temperature Monitor1428425670

OSPRay Studio

Intel OSPRay Studio is an open-source, interactive visualization and ray-tracing software package. OSPRay Studio makes use of Intel OSPRay, a portable ray-tracing engine for high-performance, high-fidelity visualizations. OSPRay builds off Intel's Embree and Intel SPMD Program Compiler (ISPC) components as part of the oneAPI rendering toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 3 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path TracerAVX-512 OnAVX-512 Off2004006008001000SE +/- 1.53, N = 3SE +/- 1.15, N = 3694841-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off272129964398AVX-512 On240029373698OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On39591684AVX-512 Off221612698OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 63863 Joules per run.2. AVX-512 Off: Approximate power consumption of 67108 Joules per run.
MinAvgMaxAVX-512 On44.958.262.8AVX-512 Off46.660.064.0OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

simdjson

This is a benchmark of SIMDJSON, a high performance JSON parser. SIMDJSON aims to be the fastest JSON parser and is used by projects like Microsoft FishStore, Yandex ClickHouse, Shopify, and others. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgGB/s, More Is Bettersimdjson 2.0Throughput Test: LargeRandomAVX-512 OnAVX-512 Off0.28350.5670.85051.1341.4175SE +/- 0.00, N = 3SE +/- 0.00, N = 31.261.04-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native
OpenBenchmarking.orgGB/s Per Watt, More Is Bettersimdjson 2.0Throughput Test: LargeRandomAVX-512 OnAVX-512 Off0.00140.00280.00420.00560.0070.0060.005
MinAvgMaxAVX-512 Off369437013788AVX-512 On240036013700OpenBenchmarking.orgMegahertz, More Is Bettersimdjson 2.0CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On53.8207.0220.8AVX-512 Off111.6222.5224.6OpenBenchmarking.orgWatts, Fewer Is Bettersimdjson 2.0CPU Power Consumption Monitor601201802403001. AVX-512 On: Approximate power consumption of 11387 Joules per run.2. AVX-512 Off: Approximate power consumption of 12237 Joules per run.
MinAvgMaxAVX-512 On33.336.737.8AVX-512 Off37.847.049.8OpenBenchmarking.orgCelsius, Fewer Is Bettersimdjson 2.0CPU Temperature Monitor1428425670

Neural Magic DeepSparse

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off130260390520650SE +/- 1.92, N = 3SE +/- 0.33, N = 3616.68509.45

OSPRay Studio

Intel OSPRay Studio is an open-source, interactive visualization and ray-tracing software package. OSPRay Studio makes use of Intel OSPRay, a portable ray-tracing engine for high-performance, high-fidelity visualizations. OSPRay builds off Intel's Embree and Intel SPMD Program Compiler (ISPC) components as part of the oneAPI rendering toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 3 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path TracerAVX-512 OnAVX-512 Off3K6K9K12K15KSE +/- 27.14, N = 3SE +/- 22.81, N = 31107013398-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off275229564184AVX-512 On240029103697OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On60618690AVX-512 Off114637703OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 102029 Joules per run.2. AVX-512 Off: Approximate power consumption of 102921 Joules per run.
MinAvgMaxAVX-512 On44.659.964.9AVX-512 Off46.661.664.9OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

Neural Magic DeepSparse

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off4080120160200SE +/- 0.47, N = 3SE +/- 0.10, N = 3155.35187.85
MinAvgMaxAVX-512 Off258931354299AVX-512 On240031093928OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.1CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On53479679AVX-512 Off216508705OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 28281 Joules per run.2. AVX-512 Off: Approximate power consumption of 30159 Joules per run.
MinAvgMaxAVX-512 On38.649.457.0AVX-512 Off41.851.560.0OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Temperature Monitor1632486480

OSPRay Studio

Intel OSPRay Studio is an open-source, interactive visualization and ray-tracing software package. OSPRay Studio makes use of Intel OSPRay, a portable ray-tracing engine for high-performance, high-fidelity visualizations. OSPRay builds off Intel's Embree and Intel SPMD Program Compiler (ISPC) components as part of the oneAPI rendering toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 3 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path TracerAVX-512 OnAVX-512 Off50100150200250SE +/- 0.00, N = 3SE +/- 0.58, N = 3177214-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off270530013951AVX-512 On240029313701OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On38597682AVX-512 Off223621696OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 70072 Joules per run.2. AVX-512 Off: Approximate power consumption of 80066 Joules per run.
MinAvgMaxAVX-512 On44.158.462.3AVX-512 Off46.660.364.5OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 3 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path TracerAVX-512 OnAVX-512 Off6K12K18K24K30KSE +/- 21.63, N = 3SE +/- 15.62, N = 32218426785-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off272729733711AVX-512 On240029433701OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On57590687AVX-512 Off224620702OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 61507 Joules per run.2. AVX-512 Off: Approximate power consumption of 75640 Joules per run.
MinAvgMaxAVX-512 On44.657.962.6AVX-512 Off47.060.865.4OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

Embree

Intel Embree is a collection of high-performance ray-tracing kernels for execution on CPUs and supporting instruction sets such as SSE, AVX, AVX2, and AVX-512. Embree also supports making use of the Intel SPMD Program Compiler (ISPC). Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFrames Per Second, More Is BetterEmbree 3.13Binary: Pathtracer ISPC - Model: Asian DragonAVX-512 OnAVX-512 Off50100150200250SE +/- 0.22, N = 9SE +/- 0.21, N = 8212.94176.88MIN: 207.72 / MAX: 227.11MIN: 170.19 / MAX: 190
OpenBenchmarking.orgFrames Per Second Per Watt, More Is BetterEmbree 3.13Binary: Pathtracer ISPC - Model: Asian DragonAVX-512 OnAVX-512 Off0.15910.31820.47730.63640.79550.7070.431
MinAvgMaxAVX-512 Off289935084437AVX-512 On240029743695OpenBenchmarking.orgMegahertz, More Is BetterEmbree 3.13CPU Peak Freq (Highest CPU Core Frequency) Monitor12002400360048006000
MinAvgMaxAVX-512 On50301635AVX-512 Off132410637OpenBenchmarking.orgWatts, Fewer Is BetterEmbree 3.13CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 2241 Joules per run.2. AVX-512 Off: Approximate power consumption of 3385 Joules per run.
MinAvgMaxAVX-512 On35.540.447.0AVX-512 Off41.845.851.1OpenBenchmarking.orgCelsius, Fewer Is BetterEmbree 3.13CPU Temperature Monitor1530456075

OpenBenchmarking.orgFrames Per Second, More Is BetterEmbree 3.13Binary: Pathtracer ISPC - Model: Asian Dragon ObjAVX-512 OnAVX-512 Off4080120160200SE +/- 0.62, N = 4SE +/- 0.36, N = 4184.47153.45MIN: 178.83 / MAX: 196.34MIN: 130.31 / MAX: 166.17
OpenBenchmarking.orgFrames Per Second Per Watt, More Is BetterEmbree 3.13Binary: Pathtracer ISPC - Model: Asian Dragon ObjAVX-512 OnAVX-512 Off0.16310.32620.48930.65240.81550.7250.481
MinAvgMaxAVX-512 Off288935924020AVX-512 On240033723699OpenBenchmarking.orgMegahertz, More Is BetterEmbree 3.13CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On56254632AVX-512 Off217319632OpenBenchmarking.orgWatts, Fewer Is BetterEmbree 3.13CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 4453 Joules per run.2. AVX-512 Off: Approximate power consumption of 5980 Joules per run.
MinAvgMaxAVX-512 On34.638.446.6AVX-512 Off39.942.449.3OpenBenchmarking.orgCelsius, Fewer Is BetterEmbree 3.13CPU Temperature Monitor1428425670

Neural Magic DeepSparse

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: CV Detection,YOLOv5s COCO - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off4080120160200SE +/- 0.22, N = 3SE +/- 0.17, N = 3189.56157.82

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: CV Detection,YOLOv5s COCO - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off246810SE +/- 0.0061, N = 3SE +/- 0.0068, N = 35.27126.3313
MinAvgMaxAVX-512 Off306836414426AVX-512 On240035273821OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.1CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On52.3341.3408.7AVX-512 Off202.0364.8414.2OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Power Consumption Monitor1102203304405501. AVX-512 On: Approximate power consumption of 15131 Joules per run.2. AVX-512 Off: Approximate power consumption of 16296 Joules per run.
MinAvgMaxAVX-512 On39.850.154.6AVX-512 Off44.452.155.6OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Temperature Monitor1530456075

Embree

Intel Embree is a collection of high-performance ray-tracing kernels for execution on CPUs and supporting instruction sets such as SSE, AVX, AVX2, and AVX-512. Embree also supports making use of the Intel SPMD Program Compiler (ISPC). Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFrames Per Second, More Is BetterEmbree 3.13Binary: Pathtracer ISPC - Model: CrownAVX-512 OnAVX-512 Off4080120160200SE +/- 0.62, N = 8SE +/- 0.25, N = 7180.93151.06MIN: 124.41 / MAX: 209.74MIN: 114.4 / MAX: 176.43
OpenBenchmarking.orgFrames Per Second Per Watt, More Is BetterEmbree 3.13Binary: Pathtracer ISPC - Model: CrownAVX-512 OnAVX-512 Off0.11790.23580.35370.47160.58950.5240.344
MinAvgMaxAVX-512 Off272634465379AVX-512 On240029283696OpenBenchmarking.orgMegahertz, More Is BetterEmbree 3.13CPU Peak Freq (Highest CPU Core Frequency) Monitor14002800420056007000
MinAvgMaxAVX-512 On54346646AVX-512 Off174439660OpenBenchmarking.orgWatts, Fewer Is BetterEmbree 3.13CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 2894 Joules per run.2. AVX-512 Off: Approximate power consumption of 3947 Joules per run.
MinAvgMaxAVX-512 On33.340.548.0AVX-512 Off39.545.852.0OpenBenchmarking.orgCelsius, Fewer Is BetterEmbree 3.13CPU Temperature Monitor1530456075

OSPRay Studio

Intel OSPRay Studio is an open-source, interactive visualization and ray-tracing software package. OSPRay Studio makes use of Intel OSPRay, a portable ray-tracing engine for high-performance, high-fidelity visualizations. OSPRay builds off Intel's Embree and Intel SPMD Program Compiler (ISPC) components as part of the oneAPI rendering toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 2 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path TracerAVX-512 OnAVX-512 Off160320480640800SE +/- 1.00, N = 3SE +/- 0.67, N = 3604722-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off271230043940AVX-512 On240028683857OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On55578672AVX-512 Off223609696OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 57980 Joules per run.2. AVX-512 Off: Approximate power consumption of 65164 Joules per run.
MinAvgMaxAVX-512 On44.457.061.4AVX-512 Off46.659.964.0OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 2 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path TracerAVX-512 OnAVX-512 Off4080120160200SE +/- 0.00, N = 3SE +/- 0.33, N = 3154184-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl
MinAvgMaxAVX-512 Off270229953808AVX-512 On240028593697OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 0.11CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On59591670AVX-512 Off203613697OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 0.11CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 67805 Joules per run.2. AVX-512 Off: Approximate power consumption of 72937 Joules per run.
MinAvgMaxAVX-512 On44.957.761.8AVX-512 Off47.059.964.0OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 0.11CPU Temperature Monitor20406080100

Neural Magic DeepSparse

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off20406080100SE +/- 0.14, N = 3SE +/- 0.05, N = 380.0895.33
MinAvgMaxAVX-512 Off262329874137AVX-512 On240028683737OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.1CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On49517625AVX-512 Off217552659OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 23254 Joules per run.2. AVX-512 Off: Approximate power consumption of 25208 Joules per run.
MinAvgMaxAVX-512 On39.951.255.5AVX-512 Off43.954.459.1OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Temperature Monitor1632486480

OpenFOAM

OpenFOAM is the leading free, open-source software for computational fluid dynamics (CFD). This test profile currently uses the drivaerFastback test case for analyzing automotive aerodynamics or alternatively the older motorBike input. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterOpenFOAM 10Input: drivaerFastback, Medium Mesh Size - Execution TimeAVX-512 OnAVX-512 Off306090120150113.64135.221. (CXX) g++ options: -std=c++14 -m64 -O3 -ftemplate-depth-100 -fPIC -fuse-ld=bfd -Xlinker --add-needed --no-as-needed -lfiniteVolume -lmeshTools -lparallel -llagrangian -lregionModels -lgenericPatchFields -lOpenFOAM -ldl -lm
MinAvgMaxAVX-512 Off291732653714AVX-512 On240031863701OpenBenchmarking.orgMegahertz, More Is BetterOpenFOAM 10CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On57662731AVX-512 Off220674735OpenBenchmarking.orgWatts, Fewer Is BetterOpenFOAM 10CPU Power Consumption Monitor2004006008001000
MinAvgMaxAVX-512 On41.362.668.5AVX-512 Off47.064.970.3OpenBenchmarking.orgCelsius, Fewer Is BetterOpenFOAM 10CPU Temperature Monitor20406080100

Neural Magic DeepSparse

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off30060090012001500SE +/- 2.39, N = 3SE +/- 0.78, N = 31195.941005.16

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off20406080100SE +/- 0.45, N = 3SE +/- 0.27, N = 385.0073.54

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off3691215SE +/- 0.06, N = 3SE +/- 0.05, N = 311.7613.59
MinAvgMaxAVX-512 Off350736983967AVX-512 On240035593801OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.1CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On50.9306.4407.9AVX-512 Off111.5321.8409.9OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Power Consumption Monitor1102203304405501. AVX-512 On: Approximate power consumption of 19306 Joules per run.2. AVX-512 Off: Approximate power consumption of 20490 Joules per run.
MinAvgMaxAVX-512 On32.443.751.5AVX-512 Off36.845.252.4OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Temperature Monitor1530456075

OpenVKL

OpenVKL is the Intel Open Volume Kernel Library that offers high-performance volume computation kernels and part of the Intel oneAPI rendering toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgItems / Sec, More Is BetterOpenVKL 1.3.1Benchmark: vklBenchmark ISPCAVX-512 OnAVX-512 Off30060090012001500SE +/- 13.58, N = 3SE +/- 8.11, N = 313321155MIN: 329 / MAX: 4770MIN: 251 / MAX: 5181
OpenBenchmarking.orgItems / Sec Per Watt, More Is BetterOpenVKL 1.3.1Benchmark: vklBenchmark ISPCAVX-512 OnAVX-512 Off0.64191.28381.92572.56763.20952.8532.355
OpenBenchmarking.orgMegahertz, More Is BetterOpenVKL 1.3.1CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 OnAVX-512 Off7001400210028003500Min: 2400 / Avg: 3408.84 / Max: 3732Min: 2621 / Avg: 3373.13 / Max: 4085
OpenBenchmarking.orgWatts, Fewer Is BetterOpenVKL 1.3.1CPU Power Consumption MonitorAVX-512 OnAVX-512 Off120240360480600Min: 55.92 / Avg: 466.92 / Max: 669.54Min: 113.56 / Avg: 490.53 / Max: 691.711. AVX-512 On: Approximate power consumption of 193926 Joules per run.2. AVX-512 Off: Approximate power consumption of 196050 Joules per run.
OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVKL 1.3.1CPU Temperature MonitorAVX-512 OnAVX-512 Off1224364860Min: 41.25 / Avg: 49.91 / Max: 58.25Min: 42.63 / Avg: 52.24 / Max: 60.88

OpenVINO

This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUAVX-512 OnAVX-512 Off40K80K120K160K200KSE +/- 127.59, N = 3SE +/- 1455.37, N = 3170652.71151239.60-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

Neural Magic DeepSparse

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off816243240SE +/- 0.03, N = 3SE +/- 0.03, N = 334.5230.61

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off816243240SE +/- 0.03, N = 3SE +/- 0.04, N = 328.9632.66
MinAvgMaxAVX-512 Off368637114893AVX-512 On240035513868OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.1CPU Peak Freq (Highest CPU Core Frequency) Monitor12002400360048006000
MinAvgMaxAVX-512 On49.8321.8419.4AVX-512 Off110.4350.8422.9OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Power Consumption Monitor1102203304405501. AVX-512 On: Approximate power consumption of 18127 Joules per run.2. AVX-512 Off: Approximate power consumption of 19058 Joules per run.
MinAvgMaxAVX-512 On38.947.853.8AVX-512 Off43.950.755.5OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Temperature Monitor1530456075

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off816243240SE +/- 0.08, N = 3SE +/- 0.10, N = 334.2530.53

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off816243240SE +/- 0.07, N = 3SE +/- 0.11, N = 329.1932.75
MinAvgMaxAVX-512 Off308936794336AVX-512 On240035333703OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.1CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On30.2328.4418.5AVX-512 Off210.0351.1423.9OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Power Consumption Monitor1102203304405501. AVX-512 On: Approximate power consumption of 18062 Joules per run.2. AVX-512 Off: Approximate power consumption of 19075 Joules per run.
MinAvgMaxAVX-512 On40.648.754.3AVX-512 Off43.550.756.9OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Temperature Monitor1632486480

NCNN

NCNN is a high performance neural network inference framework optimized for mobile and other platforms developed by Tencent. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: blazefaceAVX-512 OnAVX-512 Off714212835SE +/- 0.18, N = 8SE +/- 0.54, N = 325.8829.00-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 24.45 / MAX: 112.37-mno-avx512f - MIN: 26.03 / MAX: 144.121. (CXX) g++ options: -O3 -march=native -rdynamic -lgomp -lpthread

LeelaChessZero

LeelaChessZero (lc0 / lczero) is a chess engine automated vian neural networks. This test profile can be used for OpenCL, CUDA + cuDNN, and BLAS (CPU-based) benchmarking. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgNodes Per Second, More Is BetterLeelaChessZero 0.28Backend: EigenAVX-512 OnAVX-512 Off2K4K6K8K10KSE +/- 45.37, N = 3SE +/- 43.59, N = 390968162-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -flto -O3 -march=native -pthread
OpenBenchmarking.orgNodes Per Second Per Watt, More Is BetterLeelaChessZero 0.28Backend: EigenAVX-512 OnAVX-512 Off61218243024.8921.29
MinAvgMaxAVX-512 Off369436994052AVX-512 On240036854074OpenBenchmarking.orgMegahertz, More Is BetterLeelaChessZero 0.28CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On56.3365.4500.1AVX-512 Off112.5383.3519.6OpenBenchmarking.orgWatts, Fewer Is BetterLeelaChessZero 0.28CPU Power Consumption Monitor1302603905206501. AVX-512 On: Approximate power consumption of 135210 Joules per run.2. AVX-512 Off: Approximate power consumption of 141836 Joules per run.
MinAvgMaxAVX-512 Off37.339.956.9AVX-512 On32.453.461.0OpenBenchmarking.orgCelsius, Fewer Is BetterLeelaChessZero 0.28CPU Temperature Monitor20406080100

NCNN

NCNN is a high performance neural network inference framework optimized for mobile and other platforms developed by Tencent. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: regnety_400mAVX-512 OnAVX-512 Off60120180240300SE +/- 2.25, N = 8SE +/- 3.75, N = 3247.32270.12-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 232.61 / MAX: 506.55-mno-avx512f - MIN: 245.47 / MAX: 498.81. (CXX) g++ options: -O3 -march=native -rdynamic -lgomp -lpthread

SMHasher

SMHasher is a hash function tester supporting various algorithms and able to make use of AVX and other modern CPU instruction set extensions. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMiB/sec, More Is BetterSMHasher 2022-08-22Hash: FarmHash32 x86_64 AVXAVX-512 OnAVX-512 Off9K18K27K36K45KSE +/- 0.46, N = 5SE +/- 28.78, N = 539794.4436483.70-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -flto=auto -fno-fat-lto-objects
OpenBenchmarking.orgcycles/hash, Fewer Is BetterSMHasher 2022-08-22Hash: FarmHash32 x86_64 AVXAVX-512 OnAVX-512 Off612182430SE +/- 0.00, N = 5SE +/- 0.00, N = 526.3926.41-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -flto=auto -fno-fat-lto-objects

NCNN

NCNN is a high performance neural network inference framework optimized for mobile and other platforms developed by Tencent. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: efficientnet-b0AVX-512 OnAVX-512 Off1428425670SE +/- 0.35, N = 8SE +/- 0.31, N = 357.7162.86-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 54.52 / MAX: 522.74-mno-avx512f - MIN: 59.46 / MAX: 154.821. (CXX) g++ options: -O3 -march=native -rdynamic -lgomp -lpthread

ONNX Runtime

ONNX Runtime is developed by Microsoft and partners as a open-source, cross-platform, high performance machine learning inferencing and training accelerator. This test profile runs the ONNX Runtime with various models available from the ONNX Zoo. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgInferences Per Minute, More Is BetterONNX Runtime 1.11Model: fcn-resnet101-11 - Device: CPU - Executor: StandardAVX-512 OnAVX-512 Off60120180240300SE +/- 0.44, N = 3SE +/- 2.36, N = 12271250-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt
OpenBenchmarking.orgInferences Per Minute Per Watt, More Is BetterONNX Runtime 1.11Model: fcn-resnet101-11 - Device: CPU - Executor: StandardAVX-512 OnAVX-512 Off0.12330.24660.36990.49320.61650.5480.497
OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.11CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 OffAVX-512 On7001400210028003500Min: 3047 / Avg: 3611.53 / Max: 4291Min: 2400 / Avg: 3577.02 / Max: 3696
MinAvgMaxAVX-512 On51.7494.2532.0AVX-512 Off211.3503.0528.2OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.11CPU Power Consumption Monitor1402804205607001. AVX-512 On: Approximate power consumption of 60947 Joules per run.2. AVX-512 Off: Approximate power consumption of 61864 Joules per run.
MinAvgMaxAVX-512 On37.851.157.5AVX-512 Off43.053.459.3OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.11CPU Temperature Monitor1632486480

LeelaChessZero

LeelaChessZero (lc0 / lczero) is a chess engine automated vian neural networks. This test profile can be used for OpenCL, CUDA + cuDNN, and BLAS (CPU-based) benchmarking. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgNodes Per Second, More Is BetterLeelaChessZero 0.28Backend: BLASAVX-512 OnAVX-512 Off2K4K6K8K10KSE +/- 103.04, N = 4SE +/- 21.53, N = 390778423-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -flto -O3 -march=native -pthread
OpenBenchmarking.orgNodes Per Second Per Watt, More Is BetterLeelaChessZero 0.28Backend: BLASAVX-512 OnAVX-512 Off61218243023.7821.93
MinAvgMaxAVX-512 Off338337004961AVX-512 On240036843952OpenBenchmarking.orgMegahertz, More Is BetterLeelaChessZero 0.28CPU Peak Freq (Highest CPU Core Frequency) Monitor13002600390052006500
OpenBenchmarking.orgWatts, Fewer Is BetterLeelaChessZero 0.28CPU Power Consumption MonitorAVX-512 OnAVX-512 Off90180270360450Min: 55.6 / Avg: 381.67 / Max: 513.97Min: 153.23 / Avg: 384.04 / Max: 522.791. AVX-512 On: Approximate power consumption of 140360 Joules per run.2. AVX-512 Off: Approximate power consumption of 140944 Joules per run.
OpenBenchmarking.orgCelsius, Fewer Is BetterLeelaChessZero 0.28CPU Temperature MonitorAVX-512 OnAVX-512 Off1326395265Min: 36.75 / Avg: 51.98 / Max: 63.13Min: 47.5 / Avg: 57.42 / Max: 64.5

CP2K Molecular Dynamics

CP2K is an open-source molecular dynamics software package focused on quantum chemistry and solid-state physics. This test profile currently uses the SSMP (OpenMP) version of cp2k. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterCP2K Molecular Dynamics 8.2Input: Fayalite-FISTAVX-512 OnAVX-512 Off300600900120015001122.731198.41
MinAvgMaxAVX-512 Off356936653808AVX-512 On240036613700OpenBenchmarking.orgMegahertz, More Is BetterCP2K Molecular Dynamics 8.2CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On58.1358.0402.6AVX-512 Off118.3361.5403.0OpenBenchmarking.orgWatts, Fewer Is BetterCP2K Molecular Dynamics 8.2CPU Power Consumption Monitor110220330440550
MinAvgMaxAVX-512 Off42.144.456.9AVX-512 On42.144.553.5OpenBenchmarking.orgCelsius, Fewer Is BetterCP2K Molecular Dynamics 8.2CPU Temperature Monitor1632486480

Mobile Neural Network

MNN is the Mobile Neural Network as a highly efficient, lightweight deep learning framework developed by Alibaba. This MNN test profile is building the OpenMP / CPU threaded version for processor benchmarking and not any GPU-accelerated test. MNN does allow making use of AVX-512 extensions. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 2.1Model: SqueezeNetV1.0AVX-512 OnAVX-512 Off3691215SE +/- 0.147, N = 9SE +/- 0.091, N = 88.5799.146-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 6.67 / MAX: 21.5-mno-avx512f - MIN: 7.72 / MAX: 19.031. (CXX) g++ options: -O3 -march=native -std=c++11 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl

OpenFOAM

OpenFOAM is the leading free, open-source software for computational fluid dynamics (CFD). This test profile currently uses the drivaerFastback test case for analyzing automotive aerodynamics or alternatively the older motorBike input. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterOpenFOAM 10Input: drivaerFastback, Medium Mesh Size - Mesh TimeAVX-512 OnAVX-512 Off306090120150135.77144.351. (CXX) g++ options: -std=c++14 -m64 -O3 -ftemplate-depth-100 -fPIC -fuse-ld=bfd -Xlinker --add-needed --no-as-needed -lfiniteVolume -lmeshTools -lparallel -llagrangian -lregionModels -lgenericPatchFields -lOpenFOAM -ldl -lm

Neural Magic DeepSparse

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off4080120160200SE +/- 0.11, N = 3SE +/- 0.29, N = 3187.08177.54

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off1.26632.53263.79895.06526.3315SE +/- 0.0031, N = 3SE +/- 0.0091, N = 35.34135.6282
MinAvgMaxAVX-512 Off369237124154AVX-512 On240035413702OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.1CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On37.8324.6394.5AVX-512 Off197.4352.2401.6OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Power Consumption Monitor1102203304405501. AVX-512 On: Approximate power consumption of 15039 Joules per run.2. AVX-512 Off: Approximate power consumption of 16554 Joules per run.
MinAvgMaxAVX-512 On39.048.052.4AVX-512 Off43.050.554.3OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.1CPU Temperature Monitor1530456075

Numpy Benchmark

This is a test to obtain the general Numpy performance. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgScore, More Is BetterNumpy BenchmarkAVX-512 OnAVX-512 Off120240360480600SE +/- 2.18, N = 3SE +/- 0.59, N = 3575.66548.83
OpenBenchmarking.orgScore Per Watt, More Is BetterNumpy BenchmarkAVX-512 OnAVX-512 Off0.60461.20921.81382.41843.0232.6872.474
MinAvgMaxAVX-512 Off369437024024AVX-512 On240036733705OpenBenchmarking.orgMegahertz, More Is BetterNumpy BenchmarkCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On54.1214.2222.9AVX-512 Off110.9221.9228.1OpenBenchmarking.orgWatts, Fewer Is BetterNumpy BenchmarkCPU Power Consumption Monitor601201802403001. AVX-512 On: Approximate power consumption of 33987 Joules per run.2. AVX-512 Off: Approximate power consumption of 37052 Joules per run.
MinAvgMaxAVX-512 On32.836.340.4AVX-512 Off36.437.943.5OpenBenchmarking.orgCelsius, Fewer Is BetterNumpy BenchmarkCPU Temperature Monitor1224364860

JPEG XL libjxl

The JPEG XL Image Coding System is designed to provide next-generation JPEG image capabilities with JPEG XL offering better image quality and compression over legacy JPEG. This test profile is currently focused on the multi-threaded JPEG XL image encode performance using the reference libjxl library. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMP/s, More Is BetterJPEG XL libjxl 0.7Input: JPEG - Quality: 90AVX-512 OnAVX-512 Off3691215SE +/- 0.04, N = 3SE +/- 0.03, N = 39.559.13-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -fno-rtti -funwind-tables -O2 -fPIE -pie -lm -latomic
OpenBenchmarking.orgMP/s Per Watt, More Is BetterJPEG XL libjxl 0.7Input: JPEG - Quality: 90AVX-512 OnAVX-512 Off0.00970.01940.02910.03880.04850.0430.039
MinAvgMaxAVX-512 Off366937013814AVX-512 On240036573712OpenBenchmarking.orgMegahertz, More Is BetterJPEG XL libjxl 0.7CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On54.8223.4308.1AVX-512 Off142.6233.9323.3OpenBenchmarking.orgWatts, Fewer Is BetterJPEG XL libjxl 0.7CPU Power Consumption Monitor801602403204001. AVX-512 On: Approximate power consumption of 23308 Joules per run.2. AVX-512 Off: Approximate power consumption of 25342 Joules per run.
MinAvgMaxAVX-512 Off36.839.448.9AVX-512 On35.039.746.6OpenBenchmarking.orgCelsius, Fewer Is BetterJPEG XL libjxl 0.7CPU Temperature Monitor1428425670

NCNN

NCNN is a high performance neural network inference framework optimized for mobile and other platforms developed by Tencent. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: googlenetAVX-512 OnAVX-512 Off20406080100SE +/- 0.76, N = 8SE +/- 1.35, N = 372.8076.13-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 67.13 / MAX: 388.52-mno-avx512f - MIN: 70.13 / MAX: 155.611. (CXX) g++ options: -O3 -march=native -rdynamic -lgomp -lpthread

JPEG XL libjxl

The JPEG XL Image Coding System is designed to provide next-generation JPEG image capabilities with JPEG XL offering better image quality and compression over legacy JPEG. This test profile is currently focused on the multi-threaded JPEG XL image encode performance using the reference libjxl library. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMP/s, More Is BetterJPEG XL libjxl 0.7Input: PNG - Quality: 90AVX-512 OnAVX-512 Off3691215SE +/- 0.02, N = 3SE +/- 0.02, N = 39.959.52-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -fno-rtti -funwind-tables -O2 -fPIE -pie -lm -latomic
OpenBenchmarking.orgMP/s Per Watt, More Is BetterJPEG XL libjxl 0.7Input: PNG - Quality: 90AVX-512 OnAVX-512 Off0.01010.02020.03030.04040.05050.0450.041
MinAvgMaxAVX-512 Off243936963828AVX-512 On240036433712OpenBenchmarking.orgMegahertz, More Is BetterJPEG XL libjxl 0.7CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On56.1222.2318.2AVX-512 Off134.0232.4317.2OpenBenchmarking.orgWatts, Fewer Is BetterJPEG XL libjxl 0.7CPU Power Consumption Monitor801602403204001. AVX-512 On: Approximate power consumption of 22223 Joules per run.2. AVX-512 Off: Approximate power consumption of 24322 Joules per run.
MinAvgMaxAVX-512 Off35.936.744.9AVX-512 On32.437.049.3OpenBenchmarking.orgCelsius, Fewer Is BetterJPEG XL libjxl 0.7CPU Temperature Monitor1428425670

NCNN

NCNN is a high performance neural network inference framework optimized for mobile and other platforms developed by Tencent. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: mnasnetAVX-512 OnAVX-512 Off1020304050SE +/- 0.26, N = 8SE +/- 0.49, N = 342.3044.19-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 39.64 / MAX: 571.78-mno-avx512f - MIN: 41.56 / MAX: 148.981. (CXX) g++ options: -O3 -march=native -rdynamic -lgomp -lpthread

Numenta Anomaly Benchmark

Numenta Anomaly Benchmark (NAB) is a benchmark for evaluating algorithms for anomaly detection in streaming, real-time applications. It is comprised of over 50 labeled real-world and artificial time-series data files plus a novel scoring mechanism designed for real-time applications. This test profile currently measures the time to run various detectors. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterNumenta Anomaly Benchmark 1.1Detector: Bayesian ChangepointAVX-512 OnAVX-512 Off48121620SE +/- 0.21, N = 4SE +/- 0.19, N = 516.6817.39
MinAvgMaxAVX-512 Off369437074079AVX-512 On240035323699OpenBenchmarking.orgMegahertz, More Is BetterNumenta Anomaly Benchmark 1.1CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On55.1216.0378.8AVX-512 Off137.1251.1391.0OpenBenchmarking.orgWatts, Fewer Is BetterNumenta Anomaly Benchmark 1.1CPU Power Consumption Monitor1002003004005001. AVX-512 On: Approximate power consumption of 4428 Joules per run.2. AVX-512 Off: Approximate power consumption of 5375 Joules per run.
MinAvgMaxAVX-512 On32.440.450.1AVX-512 Off38.643.249.9OpenBenchmarking.orgCelsius, Fewer Is BetterNumenta Anomaly Benchmark 1.1CPU Temperature Monitor1428425670

oneDNN

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPUAVX-512 OnAVX-512 Off0.1210.2420.3630.4840.605SE +/- 0.004067, N = 7SE +/- 0.004200, N = 150.5161670.537682-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 0.42-mno-avx512f - MIN: 0.421. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxAVX-512 Off286033483869AVX-512 On240031823695OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 2.7CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On54.1327.3548.8AVX-512 Off216.4379.5540.9OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 2.7CPU Power Consumption Monitor1402804205607001. AVX-512 On: Approximate power consumption of 3366 Joules per run.2. AVX-512 Off: Approximate power consumption of 3871 Joules per run.
MinAvgMaxAVX-512 On40.946.051.1AVX-512 Off43.547.753.8OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 2.7CPU Temperature Monitor1530456075

Intel Open Image Denoise

Open Image Denoise is a denoising library for ray-tracing and part of the oneAPI rendering toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgImages / Sec, More Is BetterIntel Open Image Denoise 1.4.0Run: RTLightmap.hdr.4096x4096AVX-512 OffAVX-512 On0.3870.7741.1611.5481.935SE +/- 0.01, N = 3SE +/- 0.01, N = 31.721.66
OpenBenchmarking.orgImages / Sec Per Watt, More Is BetterIntel Open Image Denoise 1.4.0Run: RTLightmap.hdr.4096x4096AVX-512 OnAVX-512 Off0.00110.00220.00330.00440.00550.0050.004
MinAvgMaxAVX-512 Off301736364215AVX-512 On240032163699OpenBenchmarking.orgMegahertz, More Is BetterIntel Open Image Denoise 1.4.0CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On52.8325.6574.0AVX-512 Off207.2399.0572.6OpenBenchmarking.orgWatts, Fewer Is BetterIntel Open Image Denoise 1.4.0CPU Power Consumption Monitor1402804205607001. AVX-512 On: Approximate power consumption of 7164 Joules per run.2. AVX-512 Off: Approximate power consumption of 8511 Joules per run.
MinAvgMaxAVX-512 On32.839.846.4AVX-512 Off39.944.749.8OpenBenchmarking.orgCelsius, Fewer Is BetterIntel Open Image Denoise 1.4.0CPU Temperature Monitor1428425670

ONNX Runtime

ONNX Runtime is developed by Microsoft and partners as a open-source, cross-platform, high performance machine learning inferencing and training accelerator. This test profile runs the ONNX Runtime with various models available from the ONNX Zoo. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgInferences Per Minute, More Is BetterONNX Runtime 1.11Model: bertsquad-12 - Device: CPU - Executor: StandardAVX-512 OnAVX-512 Off110220330440550SE +/- 1.86, N = 3SE +/- 3.11, N = 3516499-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt
OpenBenchmarking.orgInferences Per Minute Per Watt, More Is BetterONNX Runtime 1.11Model: bertsquad-12 - Device: CPU - Executor: StandardAVX-512 OnAVX-512 Off0.31340.62680.94021.25361.5671.3931.315
MinAvgMaxAVX-512 Off369437013978AVX-512 On240036633709OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.11CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On50.3370.5385.5AVX-512 Off193.6379.6388.9OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.11CPU Power Consumption Monitor1002003004005001. AVX-512 On: Approximate power consumption of 45821 Joules per run.2. AVX-512 Off: Approximate power consumption of 46813 Joules per run.
MinAvgMaxAVX-512 On34.646.548.9AVX-512 Off39.048.452.0OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.11CPU Temperature Monitor1530456075

Numenta Anomaly Benchmark

Numenta Anomaly Benchmark (NAB) is a benchmark for evaluating algorithms for anomaly detection in streaming, real-time applications. It is comprised of over 50 labeled real-world and artificial time-series data files plus a novel scoring mechanism designed for real-time applications. This test profile currently measures the time to run various detectors. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterNumenta Anomaly Benchmark 1.1Detector: Windowed GaussianAVX-512 OnAVX-512 Off1.09852.1973.29554.3945.4925SE +/- 0.032, N = 15SE +/- 0.044, N = 74.7274.882
MinAvgMaxAVX-512 Off369437183948AVX-512 On240032373715OpenBenchmarking.orgMegahertz, More Is BetterNumenta Anomaly Benchmark 1.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On53.3184.7388.1AVX-512 Off212.5259.2394.2OpenBenchmarking.orgWatts, Fewer Is BetterNumenta Anomaly Benchmark 1.1CPU Power Consumption Monitor1102203304405501. AVX-512 On: Approximate power consumption of 1613 Joules per run.2. AVX-512 Off: Approximate power consumption of 2259 Joules per run.
MinAvgMaxAVX-512 On32.439.253.3AVX-512 Off37.842.860.4OpenBenchmarking.orgCelsius, Fewer Is BetterNumenta Anomaly Benchmark 1.1CPU Temperature Monitor20406080100

JPEG XL libjxl

The JPEG XL Image Coding System is designed to provide next-generation JPEG image capabilities with JPEG XL offering better image quality and compression over legacy JPEG. This test profile is currently focused on the multi-threaded JPEG XL image encode performance using the reference libjxl library. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMP/s, More Is BetterJPEG XL libjxl 0.7Input: JPEG - Quality: 100AVX-512 OnAVX-512 Off0.16650.3330.49950.6660.8325SE +/- 0.01, N = 9SE +/- 0.01, N = 90.740.72-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -fno-rtti -funwind-tables -O2 -fPIE -pie -lm -latomic
OpenBenchmarking.orgMP/s Per Watt, More Is BetterJPEG XL libjxl 0.7Input: JPEG - Quality: 100AVX-512 OffAVX-512 On0.00070.00140.00210.00280.00350.0030.003
OpenBenchmarking.orgMegahertz, More Is BetterJPEG XL libjxl 0.7CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 OffAVX-512 On8001600240032004000Min: 3414 / Avg: 3701.38 / Max: 4381Min: 2400 / Avg: 3682.16 / Max: 4023
OpenBenchmarking.orgWatts, Fewer Is BetterJPEG XL libjxl 0.7CPU Power Consumption MonitorAVX-512 OnAVX-512 Off50100150200250Min: 28.84 / Avg: 217.33 / Max: 256.47Min: 110.34 / Avg: 222.62 / Max: 261.21. AVX-512 On: Approximate power consumption of 72104 Joules per run.2. AVX-512 Off: Approximate power consumption of 75890 Joules per run.
OpenBenchmarking.orgCelsius, Fewer Is BetterJPEG XL libjxl 0.7CPU Temperature MonitorAVX-512 OnAVX-512 Off1020304050Min: 32.75 / Avg: 36.48 / Max: 49.75Min: 35.5 / Avg: 36.56 / Max: 46.63

Darmstadt Automotive Parallel Heterogeneous Suite

DAPHNE is the Darmstadt Automotive Parallel HeterogeNEous Benchmark Suite with OpenCL / CUDA / OpenMP test cases for these automotive benchmarks for evaluating programming models in context to vehicle autonomous driving capabilities. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgTest Cases Per Minute, More Is BetterDarmstadt Automotive Parallel Heterogeneous SuiteBackend: OpenMP - Kernel: NDT MappingAVX-512 OnAVX-512 Off30060090012001500SE +/- 5.94, N = 3SE +/- 8.66, N = 31371.181335.631. (CXX) g++ options: -O3 -std=c++11 -fopenmp
OpenBenchmarking.orgTest Cases Per Minute Per Watt, More Is BetterDarmstadt Automotive Parallel Heterogeneous SuiteBackend: OpenMP - Kernel: NDT MappingAVX-512 OnAVX-512 Off2468106.4165.455
MinAvgMaxAVX-512 Off369437194426AVX-512 On240035163702OpenBenchmarking.orgMegahertz, More Is BetterDarmstadt Automotive Parallel Heterogeneous SuiteCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On37.8213.7282.2AVX-512 Off215.0244.8282.7OpenBenchmarking.orgWatts, Fewer Is BetterDarmstadt Automotive Parallel Heterogeneous SuiteCPU Power Consumption Monitor701402102803501. AVX-512 On: Approximate power consumption of 5200 Joules per run.2. AVX-512 Off: Approximate power consumption of 5876 Joules per run.
MinAvgMaxAVX-512 On32.435.436.4AVX-512 Off37.839.449.5OpenBenchmarking.orgCelsius, Fewer Is BetterDarmstadt Automotive Parallel Heterogeneous SuiteCPU Temperature Monitor1428425670

NCNN

NCNN is a high performance neural network inference framework optimized for mobile and other platforms developed by Tencent. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: resnet50AVX-512 OnAVX-512 Off1530456075SE +/- 0.50, N = 8SE +/- 0.89, N = 366.3467.48-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 62.92 / MAX: 194.74-mno-avx512f - MIN: 63.4 / MAX: 170.141. (CXX) g++ options: -O3 -march=native -rdynamic -lgomp -lpthread

GROMACS

The GROMACS (GROningen MAchine for Chemical Simulations) molecular dynamics package testing with the water_GMX50 data. This test profile allows selecting between CPU and GPU-based GROMACS builds. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgNs Per Day, More Is BetterGROMACS 2022.1Implementation: MPI CPU - Input: water_GMX50_bareAVX-512 OnAVX-512 Off510152025SE +/- 0.24, N = 3SE +/- 0.10, N = 318.7618.47-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native
OpenBenchmarking.orgNs Per Day Per Watt, More Is BetterGROMACS 2022.1Implementation: MPI CPU - Input: water_GMX50_bareAVX-512 OnAVX-512 Off0.00790.01580.02370.03160.03950.0350.033
MinAvgMaxAVX-512 Off287533584145AVX-512 On240031683725OpenBenchmarking.orgMegahertz, More Is BetterGROMACS 2022.1CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On58535698AVX-512 Off216553700OpenBenchmarking.orgWatts, Fewer Is BetterGROMACS 2022.1CPU Power Consumption Monitor20040060080010001. AVX-512 On: Approximate power consumption of 14442 Joules per run.2. AVX-512 Off: Approximate power consumption of 14926 Joules per run.
MinAvgMaxAVX-512 Off43.054.864.5AVX-512 On43.356.162.6OpenBenchmarking.orgCelsius, Fewer Is BetterGROMACS 2022.1CPU Temperature Monitor20406080100

Numenta Anomaly Benchmark

Numenta Anomaly Benchmark (NAB) is a benchmark for evaluating algorithms for anomaly detection in streaming, real-time applications. It is comprised of over 50 labeled real-world and artificial time-series data files plus a novel scoring mechanism designed for real-time applications. This test profile currently measures the time to run various detectors. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterNumenta Anomaly Benchmark 1.1Detector: Relative EntropyAVX-512 OnAVX-512 Off3691215SE +/- 0.089, N = 5SE +/- 0.082, N = 59.89810.035
MinAvgMaxAVX-512 Off369437023768AVX-512 On240034373911OpenBenchmarking.orgMegahertz, More Is BetterNumenta Anomaly Benchmark 1.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On53.6202.8367.2AVX-512 Off213.2250.8339.2OpenBenchmarking.orgWatts, Fewer Is BetterNumenta Anomaly Benchmark 1.1CPU Power Consumption Monitor1002003004005001. AVX-512 On: Approximate power consumption of 2799 Joules per run.2. AVX-512 Off: Approximate power consumption of 3512 Joules per run.
MinAvgMaxAVX-512 On31.538.847.0AVX-512 Off37.841.550.4OpenBenchmarking.orgCelsius, Fewer Is BetterNumenta Anomaly Benchmark 1.1CPU Temperature Monitor1428425670

oneDNN

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Deconvolution Batch shapes_1d - Data Type: f32 - Engine: CPUAVX-512 OnAVX-512 Off510152025SE +/- 0.16, N = 3SE +/- 0.06, N = 322.5922.84-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 19.6-mno-avx512f - MIN: 20.221. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxAVX-512 Off293335263802AVX-512 On240033943729OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 2.7CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On52.1376.1492.5AVX-512 Off215.5392.5509.7OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 2.7CPU Power Consumption Monitor1302603905206501. AVX-512 On: Approximate power consumption of 9401 Joules per run.2. AVX-512 Off: Approximate power consumption of 9812 Joules per run.
MinAvgMaxAVX-512 On37.343.546.1AVX-512 Off39.543.847.0OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 2.7CPU Temperature Monitor1428425670

Intel Open Image Denoise

Open Image Denoise is a denoising library for ray-tracing and part of the oneAPI rendering toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgImages / Sec, More Is BetterIntel Open Image Denoise 1.4.0Run: RT.hdr_alb_nrm.3840x2160AVX-512 OnAVX-512 Off0.78981.57962.36943.15923.949SE +/- 0.01, N = 5SE +/- 0.03, N = 53.513.50
OpenBenchmarking.orgImages / Sec Per Watt, More Is BetterIntel Open Image Denoise 1.4.0Run: RT.hdr_alb_nrm.3840x2160AVX-512 OnAVX-512 Off0.00270.00540.00810.01080.01350.0120.010
MinAvgMaxAVX-512 Off271836264090AVX-512 On240031243699OpenBenchmarking.orgMegahertz, More Is BetterIntel Open Image Denoise 1.4.0CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On48.8287.6572.9AVX-512 Off206.5366.7574.9OpenBenchmarking.orgWatts, Fewer Is BetterIntel Open Image Denoise 1.4.0CPU Power Consumption Monitor1402804205607001. AVX-512 On: Approximate power consumption of 3739 Joules per run.2. AVX-512 Off: Approximate power consumption of 4840 Joules per run.
MinAvgMaxAVX-512 On32.438.144.9AVX-512 Off36.841.647.0OpenBenchmarking.orgCelsius, Fewer Is BetterIntel Open Image Denoise 1.4.0CPU Temperature Monitor1428425670

OpenBenchmarking.orgImages / Sec, More Is BetterIntel Open Image Denoise 1.4.0Run: RT.ldr_alb_nrm.3840x2160AVX-512 OffAVX-512 On0.78981.57962.36943.15923.949SE +/- 0.01, N = 5SE +/- 0.01, N = 53.513.51
OpenBenchmarking.orgImages / Sec Per Watt, More Is BetterIntel Open Image Denoise 1.4.0Run: RT.ldr_alb_nrm.3840x2160AVX-512 OnAVX-512 Off0.00270.00540.00810.01080.01350.0120.010
MinAvgMaxAVX-512 Off285436103919AVX-512 On240032123702OpenBenchmarking.orgMegahertz, More Is BetterIntel Open Image Denoise 1.4.0CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX-512 On52.1288.6574.3AVX-512 Off208.7365.6577.3OpenBenchmarking.orgWatts, Fewer Is BetterIntel Open Image Denoise 1.4.0CPU Power Consumption Monitor1402804205607001. AVX-512 On: Approximate power consumption of 3752 Joules per run.2. AVX-512 Off: Approximate power consumption of 4752 Joules per run.
MinAvgMaxAVX-512 On32.838.244.1AVX-512 Off39.543.348.4OpenBenchmarking.orgCelsius, Fewer Is BetterIntel Open Image Denoise 1.4.0CPU Temperature Monitor1428425670

CPU Temperature Monitor

OpenBenchmarking.orgCelsiusCPU Temperature MonitorPhoronix Test Suite System MonitoringAVX-512 OnAVX-512 Off1428425670Min: 30.13 / Avg: 49.97 / Max: 73.38Min: 35.5 / Avg: 51.26 / Max: 73.75

CPU Power Consumption Monitor

OpenBenchmarking.orgWattsCPU Power Consumption MonitorPhoronix Test Suite System MonitoringAVX-512 OnAVX-512 Off130260390520650Min: 26.37 / Avg: 434.8 / Max: 766.01Min: 106.95 / Avg: 449.58 / Max: 735.32

Darmstadt Automotive Parallel Heterogeneous Suite

OpenBenchmarking.orgCelsius, Fewer Is BetterDarmstadt Automotive Parallel Heterogeneous SuiteCPU Temperature MonitorAVX-512 OnAVX-512 Off1020304050Min: 31.25 / Avg: 38.03 / Max: 47.5Min: 36.75 / Avg: 39.2 / Max: 49.75

OpenBenchmarking.orgWatts, Fewer Is BetterDarmstadt Automotive Parallel Heterogeneous SuiteCPU Power Consumption MonitorAVX-512 OnAVX-512 Off50100150200250Min: 49.36 / Avg: 258.94 / Max: 283.12Min: 136.19 / Avg: 273.1 / Max: 289.31

OpenBenchmarking.orgMegahertz, More Is BetterDarmstadt Automotive Parallel Heterogeneous SuiteCPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 OffAVX-512 On8001600240032004000Min: 3694 / Avg: 3698.64 / Max: 4768Min: 2400 / Avg: 3647.5 / Max: 3715

OpenBenchmarking.orgTest Cases Per Minute Per Watt, More Is BetterDarmstadt Automotive Parallel Heterogeneous SuiteBackend: OpenMP - Kernel: Points2ImageAVX-512 OnAVX-512 Off122436486052.8242.19

OpenBenchmarking.orgTest Cases Per Minute, More Is BetterDarmstadt Automotive Parallel Heterogeneous SuiteBackend: OpenMP - Kernel: Points2ImageAVX-512 OnAVX-512 Off3K6K9K12K15KSE +/- 126.17, N = 15SE +/- 269.82, N = 1513677.7611521.381. (CXX) g++ options: -O3 -std=c++11 -fopenmp
OpenBenchmarking.orgTest Cases Per Minute Per Watt, More Is BetterDarmstadt Automotive Parallel Heterogeneous SuiteBackend: OpenMP - Kernel: Points2ImageAVX-512 OnAVX-512 Off122436486052.8242.19
OpenBenchmarking.orgMegahertz, More Is BetterDarmstadt Automotive Parallel Heterogeneous SuiteCPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 OffAVX-512 On8001600240032004000Min: 3694 / Avg: 3698.64 / Max: 4768Min: 2400 / Avg: 3647.5 / Max: 3715
OpenBenchmarking.orgWatts, Fewer Is BetterDarmstadt Automotive Parallel Heterogeneous SuiteCPU Power Consumption MonitorAVX-512 OnAVX-512 Off50100150200250Min: 49.36 / Avg: 258.94 / Max: 283.12Min: 136.19 / Avg: 273.1 / Max: 289.311. AVX-512 On: Approximate power consumption of 21595 Joules per run.2. AVX-512 Off: Approximate power consumption of 22940 Joules per run.
OpenBenchmarking.orgCelsius, Fewer Is BetterDarmstadt Automotive Parallel Heterogeneous SuiteCPU Temperature MonitorAVX-512 OnAVX-512 Off1020304050Min: 31.25 / Avg: 38.03 / Max: 47.5Min: 36.75 / Avg: 39.2 / Max: 49.75

SVT-AV1

MinAvgMaxAVX-512 On30.634.643.0AVX-512 Off37.840.548.1OpenBenchmarking.orgCelsius, Fewer Is BetterSVT-AV1 1.4CPU Temperature Monitor1428425670

MinAvgMaxAVX-512 On52.7175.4338.1AVX-512 Off127.1241.8338.9OpenBenchmarking.orgWatts, Fewer Is BetterSVT-AV1 1.4CPU Power Consumption Monitor80160240320400

MinAvgMaxAVX-512 Off369437183985AVX-512 On240032743770OpenBenchmarking.orgMegahertz, More Is BetterSVT-AV1 1.4CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500

OpenBenchmarking.orgFrames Per Second Per Watt, More Is BetterSVT-AV1 1.4Encoder Mode: Preset 12 - Input: Bosphorus 4KAVX-512 OnAVX-512 Off0.3150.630.9451.261.5751.4000.968

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 1.4Encoder Mode: Preset 12 - Input: Bosphorus 4KAVX-512 OnAVX-512 Off50100150200250SE +/- 4.78, N = 15SE +/- 3.57, N = 15245.56233.94
OpenBenchmarking.orgFrames Per Second Per Watt, More Is BetterSVT-AV1 1.4Encoder Mode: Preset 12 - Input: Bosphorus 4KAVX-512 OnAVX-512 Off0.3150.630.9451.261.5751.4000.968
MinAvgMaxAVX-512 Off369437183985AVX-512 On240032743770OpenBenchmarking.orgMegahertz, More Is BetterSVT-AV1 1.4CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On52.7175.4338.1AVX-512 Off127.1241.8338.9OpenBenchmarking.orgWatts, Fewer Is BetterSVT-AV1 1.4CPU Power Consumption Monitor801602403204001. AVX-512 On: Approximate power consumption of 1719 Joules per run.2. AVX-512 Off: Approximate power consumption of 2434 Joules per run.
MinAvgMaxAVX-512 On30.634.643.0AVX-512 Off37.840.548.1OpenBenchmarking.orgCelsius, Fewer Is BetterSVT-AV1 1.4CPU Temperature Monitor1428425670

MinAvgMaxAVX-512 On32.436.849.3AVX-512 Off39.042.449.3OpenBenchmarking.orgCelsius, Fewer Is BetterSVT-AV1 1.4CPU Temperature Monitor1428425670

MinAvgMaxAVX-512 On52.7162.9323.1AVX-512 Off108.8243.3325.0OpenBenchmarking.orgWatts, Fewer Is BetterSVT-AV1 1.4CPU Power Consumption Monitor80160240320400

MinAvgMaxAVX-512 Off369437284349AVX-512 On240032023756OpenBenchmarking.orgMegahertz, More Is BetterSVT-AV1 1.4CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500

OpenBenchmarking.orgFrames Per Second Per Watt, More Is BetterSVT-AV1 1.4Encoder Mode: Preset 13 - Input: Bosphorus 4KAVX-512 OnAVX-512 Off0.34040.68081.02121.36161.7021.5130.986

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 1.4Encoder Mode: Preset 13 - Input: Bosphorus 4KAVX-512 OnAVX-512 Off50100150200250SE +/- 4.74, N = 15SE +/- 4.01, N = 15246.57239.88
OpenBenchmarking.orgFrames Per Second Per Watt, More Is BetterSVT-AV1 1.4Encoder Mode: Preset 13 - Input: Bosphorus 4KAVX-512 OnAVX-512 Off0.34040.68081.02121.36161.7021.5130.986
MinAvgMaxAVX-512 Off369437284349AVX-512 On240032023756OpenBenchmarking.orgMegahertz, More Is BetterSVT-AV1 1.4CPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX-512 On52.7162.9323.1AVX-512 Off108.8243.3325.0OpenBenchmarking.orgWatts, Fewer Is BetterSVT-AV1 1.4CPU Power Consumption Monitor801602403204001. AVX-512 On: Approximate power consumption of 1293 Joules per run.2. AVX-512 Off: Approximate power consumption of 1962 Joules per run.
MinAvgMaxAVX-512 On32.436.849.3AVX-512 Off39.042.449.3OpenBenchmarking.orgCelsius, Fewer Is BetterSVT-AV1 1.4CPU Temperature Monitor1428425670

ONNX Runtime

OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.11CPU Temperature MonitorAVX-512 OnAVX-512 Off1122334455Min: 32.75 / Avg: 48.03 / Max: 50.63Min: 38.13 / Avg: 50.46 / Max: 57.25

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.11CPU Power Consumption MonitorAVX-512 OnAVX-512 Off80160240320400Min: 51.25 / Avg: 441.77 / Max: 464.82Min: 107.66 / Avg: 447.6 / Max: 464.67

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.11CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 OffAVX-512 On7001400210028003500Min: 3593 / Avg: 3607.6 / Max: 4285Min: 2400 / Avg: 3571.99 / Max: 3770

OpenBenchmarking.orgInferences Per Minute Per Watt, More Is BetterONNX Runtime 1.11Model: ArcFace ResNet-100 - Device: CPU - Executor: StandardAVX-512 OnAVX-512 Off0.53531.07061.60592.14122.67652.3792.051

OpenBenchmarking.orgInferences Per Minute, More Is BetterONNX Runtime 1.11Model: ArcFace ResNet-100 - Device: CPU - Executor: StandardAVX-512 OnAVX-512 Off2004006008001000SE +/- 39.92, N = 12SE +/- 20.83, N = 121051918-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt
OpenBenchmarking.orgInferences Per Minute Per Watt, More Is BetterONNX Runtime 1.11Model: ArcFace ResNet-100 - Device: CPU - Executor: StandardAVX-512 OnAVX-512 Off0.53531.07061.60592.14122.67652.3792.051
OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.11CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 OffAVX-512 On7001400210028003500Min: 3593 / Avg: 3607.6 / Max: 4285Min: 2400 / Avg: 3571.99 / Max: 3770
OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.11CPU Power Consumption MonitorAVX-512 OnAVX-512 Off80160240320400Min: 51.25 / Avg: 441.77 / Max: 464.82Min: 107.66 / Avg: 447.6 / Max: 464.671. AVX-512 On: Approximate power consumption of 54448 Joules per run.2. AVX-512 Off: Approximate power consumption of 55203 Joules per run.
OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.11CPU Temperature MonitorAVX-512 OnAVX-512 Off1122334455Min: 32.75 / Avg: 48.03 / Max: 50.63Min: 38.13 / Avg: 50.46 / Max: 57.25

MinAvgMaxAVX-512 On36.439.942.6AVX-512 Off39.543.253.0OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.11CPU Temperature Monitor1530456075

MinAvgMaxAVX-512 On53.2259.3268.4AVX-512 Off132.5266.8278.0OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.11CPU Power Consumption Monitor70140210280350

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.11CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 OffAVX-512 On8001600240032004000Min: 3693 / Avg: 3701.93 / Max: 4397Min: 2400 / Avg: 3664.92 / Max: 3710

OpenBenchmarking.orgInferences Per Minute Per Watt, More Is BetterONNX Runtime 1.11Model: super-resolution-10 - Device: CPU - Executor: StandardAVX-512 OnAVX-512 Off71421283528.5423.57

OpenBenchmarking.orgInferences Per Minute, More Is BetterONNX Runtime 1.11Model: super-resolution-10 - Device: CPU - Executor: StandardAVX-512 OnAVX-512 Off16003200480064008000SE +/- 5.46, N = 3SE +/- 260.54, N = 1274016288-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt
OpenBenchmarking.orgInferences Per Minute Per Watt, More Is BetterONNX Runtime 1.11Model: super-resolution-10 - Device: CPU - Executor: StandardAVX-512 OnAVX-512 Off71421283528.5423.57
OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.11CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 OffAVX-512 On8001600240032004000Min: 3693 / Avg: 3701.93 / Max: 4397Min: 2400 / Avg: 3664.92 / Max: 3710
MinAvgMaxAVX-512 On53.2259.3268.4AVX-512 Off132.5266.8278.0OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.11CPU Power Consumption Monitor701402102803501. AVX-512 On: Approximate power consumption of 31809 Joules per run.2. AVX-512 Off: Approximate power consumption of 32839 Joules per run.
MinAvgMaxAVX-512 On36.439.942.6AVX-512 Off39.543.253.0OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.11CPU Temperature Monitor1530456075

NCNN

OpenBenchmarking.orgCelsius, Fewer Is BetterNCNN 20220729CPU Temperature MonitorAVX-512 OnAVX-512 Off1020304050Min: 38.63 / Avg: 44.45 / Max: 48Min: 43 / Avg: 46.1 / Max: 50.13

OpenBenchmarking.orgWatts, Fewer Is BetterNCNN 20220729CPU Power Consumption MonitorAVX-512 OnAVX-512 Off80160240320400Min: 54.96 / Avg: 396.44 / Max: 451.5Min: 199.1 / Avg: 400.13 / Max: 450.41

OpenBenchmarking.orgMegahertz, More Is BetterNCNN 20220729CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 OffAVX-512 On7001400210028003500Min: 3563 / Avg: 3574.38 / Max: 3815Min: 2400 / Avg: 3565.67 / Max: 3699

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: FastestDetAVX-512 OnAVX-512 Off1428425670SE +/- 0.49, N = 8SE +/- 2.90, N = 358.9560.73-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 55.11 / MAX: 282.72-mno-avx512f - MIN: 52.89 / MAX: 236.851. (CXX) g++ options: -O3 -march=native -rdynamic -lgomp -lpthread
OpenBenchmarking.orgMegahertz, More Is BetterNCNN 20220729CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 OffAVX-512 On7001400210028003500Min: 3563 / Avg: 3574.38 / Max: 3815Min: 2400 / Avg: 3565.67 / Max: 3699
OpenBenchmarking.orgWatts, Fewer Is BetterNCNN 20220729CPU Power Consumption MonitorAVX-512 OnAVX-512 Off80160240320400Min: 54.96 / Avg: 396.44 / Max: 451.5Min: 199.1 / Avg: 400.13 / Max: 450.411. AVX-512 On: Approximate power consumption of 173692 Joules per run.2. AVX-512 Off: Approximate power consumption of 185927 Joules per run.
OpenBenchmarking.orgCelsius, Fewer Is BetterNCNN 20220729CPU Temperature MonitorAVX-512 OnAVX-512 Off1020304050Min: 38.63 / Avg: 44.45 / Max: 48Min: 43 / Avg: 46.1 / Max: 50.13

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: vision_transformerAVX-512 OnAVX-512 Off20406080100SE +/- 1.59, N = 8SE +/- 5.82, N = 374.9386.16-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 65.04 / MAX: 2154.62-mno-avx512f - MIN: 73.72 / MAX: 1760.741. (CXX) g++ options: -O3 -march=native -rdynamic -lgomp -lpthread

TensorFlow

MinAvgMaxAVX-512 Off38.640.550.4AVX-512 On34.641.748.9OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.10CPU Temperature Monitor1428425670

MinAvgMaxAVX-512 Off156.1298.9329.3AVX-512 On53.1308.0379.5OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.10CPU Power Consumption Monitor100200300400500

MinAvgMaxAVX-512 Off369437274474AVX-512 On240035864161OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.10CPU Peak Freq (Highest CPU Core Frequency) Monitor12002400360048006000

OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.10Device: CPU - Batch Size: 16 - Model: GoogLeNetAVX-512 OnAVX-512 Off0.04390.08780.13170.17560.21950.1950.104

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.10Device: CPU - Batch Size: 16 - Model: GoogLeNetAVX-512 OnAVX-512 Off1326395265SE +/- 1.01, N = 15SE +/- 0.25, N = 360.1731.12
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.10Device: CPU - Batch Size: 16 - Model: GoogLeNetAVX-512 OnAVX-512 Off0.04390.08780.13170.17560.21950.1950.104
MinAvgMaxAVX-512 Off369437274474AVX-512 On240035864161OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.10CPU Peak Freq (Highest CPU Core Frequency) Monitor12002400360048006000
MinAvgMaxAVX-512 Off156.1298.9329.3AVX-512 On53.1308.0379.5OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.10CPU Power Consumption Monitor1002003004005001. AVX-512 Off: Approximate power consumption of 18931 Joules per run.2. AVX-512 On: Approximate power consumption of 11110 Joules per run.
MinAvgMaxAVX-512 Off38.640.550.4AVX-512 On34.641.748.9OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.10CPU Temperature Monitor1428425670

163 Results Shown

TensorFlow
oneDNN
AI Benchmark Alpha
oneDNN
OpenVINO:
  Weld Porosity Detection FP16 - CPU:
    ms
    FPS
  Face Detection FP16 - CPU:
    FPS
    ms
  Age Gender Recognition Retail 0013 FP16 - CPU:
    ms
Cpuminer-Opt
AI Benchmark Alpha
OpenVINO:
  Machine Translation EN To DE FP16 - CPU:
    ms
    FPS
  Weld Porosity Detection FP16-INT8 - CPU:
    FPS
    ms
  Face Detection FP16-INT8 - CPU:
    FPS
    ms
  Vehicle Detection FP16 - CPU:
    FPS
    ms
AI Benchmark Alpha
OpenVINO:
  Vehicle Detection FP16-INT8 - CPU:
    FPS
    ms
oneDNN
OpenVINO:
  Person Vehicle Bike Detection FP16 - CPU:
    FPS
    ms
TensorFlow
OSPRay
Cpuminer-Opt
OpenVINO:
  Person Detection FP16 - CPU:
    FPS
    ms
  Person Detection FP32 - CPU:
    FPS
    ms
OSPRay
Cpuminer-Opt
OpenVINO
Mobile Neural Network
Neural Magic DeepSparse:
  NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream:
    items/sec
    ms/batch
Mobile Neural Network
Cpuminer-Opt
miniBUDE:
  OpenMP - BM1:
    GFInst/s
    Billion Interactions/s
Neural Magic DeepSparse:
  NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Stream:
    items/sec
    ms/batch
  CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream:
    ms/batch
    items/sec
oneDNN
OpenVINO
Cpuminer-Opt
miniBUDE:
  OpenMP - BM2:
    Billion Interactions/s
    GFInst/s
OSPRay
Cpuminer-Opt
simdjson:
  DistinctUserID
  PartialTweets
Neural Magic DeepSparse:
  CV Detection,YOLOv5s COCO - Asynchronous Multi-Stream:
    items/sec
    ms/batch
simdjson
OSPRay Studio:
  2 - 1080p - 32 - Path Tracer
  2 - 4K - 16 - Path Tracer
  2 - 1080p - 16 - Path Tracer
  1 - 1080p - 32 - Path Tracer
  1 - 1080p - 1 - Path Tracer
  2 - 4K - 32 - Path Tracer
  1 - 4K - 1 - Path Tracer
  1 - 4K - 32 - Path Tracer
  1 - 4K - 16 - Path Tracer
  1 - 1080p - 16 - Path Tracer
  3 - 1080p - 16 - Path Tracer
  3 - 1080p - 32 - Path Tracer
simdjson
OSPRay Studio
simdjson
Neural Magic DeepSparse
OSPRay Studio
Neural Magic DeepSparse
OSPRay Studio:
  3 - 1080p - 1 - Path Tracer
  3 - 4K - 32 - Path Tracer
Embree:
  Pathtracer ISPC - Asian Dragon
  Pathtracer ISPC - Asian Dragon Obj
Neural Magic DeepSparse:
  CV Detection,YOLOv5s COCO - Synchronous Single-Stream:
    items/sec
    ms/batch
Embree
OSPRay Studio:
  2 - 4K - 1 - Path Tracer
  2 - 1080p - 1 - Path Tracer
Neural Magic DeepSparse
OpenFOAM
Neural Magic DeepSparse:
  NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream
  NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Stream
  NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Stream
OpenVKL
OpenVINO
Neural Magic DeepSparse:
  NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream:
    items/sec
    ms/batch
  NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream:
    items/sec
    ms/batch
NCNN
LeelaChessZero
NCNN
SMHasher
NCNN
ONNX Runtime
LeelaChessZero
CP2K Molecular Dynamics
Mobile Neural Network
OpenFOAM
Neural Magic DeepSparse:
  NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream:
    items/sec
    ms/batch
Numpy Benchmark
JPEG XL libjxl
NCNN
JPEG XL libjxl
NCNN
Numenta Anomaly Benchmark
oneDNN
Intel Open Image Denoise
ONNX Runtime
Numenta Anomaly Benchmark
JPEG XL libjxl
Darmstadt Automotive Parallel Heterogeneous Suite
NCNN
GROMACS
Numenta Anomaly Benchmark
oneDNN
Intel Open Image Denoise:
  RT.hdr_alb_nrm.3840x2160
  RT.ldr_alb_nrm.3840x2160
CPU Temperature Monitor:
  Phoronix Test Suite System Monitoring:
    Celsius
    Watts
  CPU Temp Monitor:
    Celsius
  CPU Power Consumption Monitor:
    Watts
  CPU Peak Freq (Highest CPU Core Frequency) Monitor:
    Megahertz
  OpenMP - Points2Image:
    Test Cases Per Minute Per Watt
Darmstadt Automotive Parallel Heterogeneous Suite
SVT-AV1:
  CPU Temp Monitor
  CPU Power Consumption Monitor
  CPU Peak Freq (Highest CPU Core Frequency) Monitor
  Preset 12 - Bosphorus 4K
SVT-AV1
SVT-AV1:
  CPU Temp Monitor
  CPU Power Consumption Monitor
  CPU Peak Freq (Highest CPU Core Frequency) Monitor
  Preset 13 - Bosphorus 4K
SVT-AV1
ONNX Runtime:
  CPU Temp Monitor
  CPU Power Consumption Monitor
  CPU Peak Freq (Highest CPU Core Frequency) Monitor
  ArcFace ResNet-100 - CPU - Standard
ONNX Runtime
ONNX Runtime:
  CPU Temp Monitor
  CPU Power Consumption Monitor
  CPU Peak Freq (Highest CPU Core Frequency) Monitor
  super-resolution-10 - CPU - Standard
ONNX Runtime
NCNN:
  CPU Temp Monitor
  CPU Power Consumption Monitor
  CPU Peak Freq (Highest CPU Core Frequency) Monitor
NCNN:
  CPU - FastestDet
  CPU - vision_transformer
TensorFlow:
  CPU Temp Monitor
  CPU Power Consumption Monitor
  CPU Peak Freq (Highest CPU Core Frequency) Monitor
  CPU - 16 - GoogLeNet
TensorFlow

AVX-512 On

Kernel Notes: Transparent Huge Pages: madvise
Environment Notes: CXXFLAGS="-O3 -march=native -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi -mprefer-vector-width=512" CFLAGS="-O3 -march=native -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi -mprefer-vector-width=512"
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-U8K4Qv/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-U8K4Qv/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa10110d
Python Notes: Python 3.10.7
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 18 December 2022 08:13 by user phoronix.

AVX-512 Off

Processor: 2 x AMD EPYC 9654 96-Core @ 2.40GHz (192 Cores / 384 Threads), Motherboard: AMD Titanite_4G (RTI1002E BIOS), Chipset: AMD Device 14a4, Memory: 1520GB, Disk: 800GB INTEL SSDPF21Q800GB, Graphics: ASPEED, Monitor: VGA HDMI, Network: Broadcom NetXtreme BCM5720 PCIe

OS: Ubuntu 22.10, Kernel: 6.1.0-phx (x86_64), Desktop: GNOME Shell 43.0, Display Server: X Server 1.21.1.4, Vulkan: 1.3.224, Compiler: GCC 12.2.0 + Clang 15.0.2-1, File-System: ext4, Screen Resolution: 1920x1080

Kernel Notes: Transparent Huge Pages: madvise
Environment Notes: CXXFLAGS="-O3 -march=native -mno-avx512f" CFLAGS="-O3 -march=native -mno-avx512f"
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-U8K4Qv/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-U8K4Qv/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa10110d
Python Notes: Python 3.10.7
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 18 December 2022 08:13 by user phoronix.