nov

Intel Core i7-1280P testing with a MSI Prestige 14Evo A12M MS-14C6 (E14C6IMS.115 BIOS) and MSI Intel ADL GT2 8GB on Ubuntu 24.10 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2411271-NE-NOV20712723&sor&grr.

novProcessorMotherboardChipsetMemoryDiskGraphicsAudioNetworkOSKernelDesktopDisplay ServerOpenGLCompilerFile-SystemScreen ResolutionabbbcIntel Core i7-1280P @ 4.70GHz (14 Cores / 20 Threads)MSI Prestige 14Evo A12M MS-14C6 (E14C6IMS.115 BIOS)Intel Alder Lake PCH8 x 2GB LPDDR4-4267MT/s SK Hynix H9HCNNNCPMMLXR-1024GB Micron_3400_MTFDKBA1T0TFHMSI Intel ADL GT2 8GBRealtek ALC274Intel Alder Lake-P PCH CNVi WiFiUbuntu 24.106.11.0-rc6-phx (x86_64)GNOME Shell 47.0X Server + Wayland4.6 Mesa 24.2.3-1ubuntu1GCC 14.2.0ext41920x1080OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2,rust --enable-libphobos-checking=release --enable-libstdcxx-backtrace --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-14-zdkDXv/gcc-14-14.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-14-zdkDXv/gcc-14-14.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0x434 - Thermald 2.5.8 Java Details- OpenJDK Runtime Environment (build 21.0.5-ea+8-Ubuntu-1)Python Details- Python 3.12.7Security Details- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Mitigation of Clear Register File + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS; IBPB: conditional; RSB filling; PBRSB-eIBRS: SW sequence; BHI: BHI_DIS_S + srbds: Not affected + tsx_async_abort: Not affected

novllama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 2048llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 2048blender: Pabellon Barcelona - CPU-Onlyblender: Classroom - CPU-Onlyrustls: handshake-resume - TLS13_CHACHA20_POLY1305_SHA256llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 1024rustls: handshake-ticket - TLS13_CHACHA20_POLY1305_SHA256rustls: handshake-ticket - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384llama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 1024rustls: handshake-resume - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384blender: Fishy Cat - CPU-Onlyblender: Junkshop - CPU-Onlyopenvino-genai: Gemma-7b-int4-ov - CPU - Time Per Output Tokenopenvino-genai: Gemma-7b-int4-ov - CPU - Time To First Tokenmt-dgemm: Sustained Floating-Point Raterustls: handshake - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 512blender: BMW27 - CPU-Onlyllama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 2048llama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 512palabos: 100openvino-genai: Gemma-7b-int4-ov - CPUrustls: handshake - TLS13_CHACHA20_POLY1305_SHA256rustls: handshake - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256build-eigen: Time To Compilepalabos: 400llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Text Generation 128vkpeak: int16-vec4vkpeak: int16-scalarvkpeak: int32-vec4vkpeak: int32-scalarvkpeak: fp16-vec4vkpeak: fp16-scalarvkpeak: fp32-vec4vkpeak: fp32-scalaropenvino-genai: Falcon-7b-instruct-int4-ov - CPU - Time Per Output Tokenopenvino-genai: Falcon-7b-instruct-int4-ov - CPU - Time To First Tokenopenvino-genai: Falcon-7b-instruct-int4-ov - CPUllama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 1024rustls: handshake-ticket - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256renaissance: ALS Movie Lensrustls: handshake-resume - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256llama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Text Generation 128openvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPU - Time Per Output Tokenopenvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPU - Time To First Tokenopenvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPUrenaissance: In-Memory Database Shootoutrenaissance: Apache Spark PageRankrenaissance: Akka Unbalanced Cobwebbed Treeopenvino: Face Detection FP16 - CPUopenvino: Face Detection FP16 - CPUrenaissance: Savina Reactors.IOrenaissance: Gaussian Mixture Modelrenaissance: Finagle HTTP Requestsrenaissance: Apache Spark Bayesopenvino: Face Detection FP16-INT8 - CPUopenvino: Face Detection FP16-INT8 - CPUrenaissance: Rand Forestrenaissance: Genetic Algorithm Using Jenetics + Futuresrenaissance: Scala Dottyopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Handwritten English Recognition FP16 - CPUopenvino: Handwritten English Recognition FP16 - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16 - CPUopenvino: Road Segmentation ADAS FP16 - CPUopenvino: Noise Suppression Poconet-Like FP16 - CPUopenvino: Noise Suppression Poconet-Like FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Person Re-Identification Retail FP16 - CPUopenvino: Person Re-Identification Retail FP16 - CPUopenvino: Vehicle Detection FP16 - CPUopenvino: Vehicle Detection FP16 - CPUopenvino: Face Detection Retail FP16-INT8 - CPUopenvino: Face Detection Retail FP16-INT8 - CPUopenvino: Face Detection Retail FP16 - CPUopenvino: Face Detection Retail FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU - Time Per Output Tokenopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU - Time To First Tokenopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPUsupertuxkart: 1920 x 1080 - High - Vulkan - Mostly Stable FPSsupertuxkart: 1920 x 1080 - High - Vulkan - Steady FPSsupertuxkart: 1920 x 1080 - Ultimate - Vulkan - Mostly Stable FPSsupertuxkart: 1920 x 1080 - Ultimate - Vulkan - Steady FPSllama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 512supertuxkart: 1920 x 1080 - Low - Vulkan - Mostly Stable FPSsupertuxkart: 1920 x 1080 - Low - Vulkan - Steady FPSsupertuxkart: 1920 x 1080 - Medium - Vulkan - Mostly Stable FPSsupertuxkart: 1920 x 1080 - Medium - Vulkan - Steady FPSsupertuxkart: 1920 x 1080 - High - OpenGL - Mostly Stable FPSsupertuxkart: 1920 x 1080 - High - OpenGL - Steady FPSsupertuxkart: 1920 x 1080 - Low - OpenGL - Mostly Stable FPSsupertuxkart: 1920 x 1080 - Low - OpenGL - Steady FPSsupertuxkart: 1920 x 1080 - Ultimate - OpenGL - Mostly Stable FPSsupertuxkart: 1920 x 1080 - Ultimate - OpenGL - Steady FPSsupertuxkart: 1920 x 1080 - Medium - OpenGL - Mostly Stable FPSsupertuxkart: 1920 x 1080 - Medium - OpenGL - Steady FPSllama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Text Generation 128renaissance: Apache Spark ALSabbbc13.3412.48916.89755.48105546.1812.77106054.04356237.6813.7415559.3403.53368.84179.4640292044.913.09271.3752.1814.0743.31049355.199526.43147.80374.92556.181919.651013.63550.78473.563551.692503.321652.721868.4494.57191.3310.5757.88809762.8114459.41082805.96.698.98345.810.15302.83063.29185.65004.251.175338.74643.44385.9638.21402.54.27570.11244.5550.0439.6113.38392.4815.25259.9661.0565.6891.1238.6666.62117.550.7629.11205.0234.21174.7832.52183.9462.0196.288.78678.8417.793351.668888.154.013750.5744.1280.6422.6629724529424561.5132125031424765591351136055867628.1313.4801.74687.92111914.7613.76112736.73379460.5113.79445443.44349.6322.47181.14305797635.314.11235.5514.8240.838710102.4210169.47148.02571.13476.421919.431013.52550.7473.513551.192502.951652.471868.13864815.5515204.01154350.716086.22809.09157.14489.061.315229.64581.13884.6745.21285.294.61600.61222.8561.4411.3314.53350.7417.05246.5664.7159.62100.47217.8673.28114.3252.3626.71223.6230.68195.0229.42203.3959.37100.768.15731.8716.04372.291.539570.873.634146.332962482862363181513022476157135465651867313.3913.38801.96753.66105168.6713.76104991.4355887.3213.77416122.79399.23366.21222.88943.04181.33411291665.5314.13270.0952.3114.126.5294.499387.989571.62147.95550.72956.421609.56939.02542.62473.4935512502.821652.351868.0194.78191.7410.5557.7808705.1614475.61079421.746.6100.11335.049.995054.42922.28669.34504.411.315360.84616.04297.2716.91281.774.62624.21230.5596.6410.9114.57352.8116.96240.7166.3359.42100.83210.7575.74114.6552.2426.59224.6630.48196.3729.28204.459.8799.988.09737.3115.99373.561.539615.643.624149.8644.2873.0322.5828022926821161.423032482862366255131595650817028.189.369.381054.24864.22105132.057.17105650.04379788.69.59448873.55450.68355.04278.261158.18180.51615192487.827.21237.1444.059.7241.18773.599367.739594148.03971.40354.151919.41013.51550.72473.53551.162502.981652.461868.28104.88238.249.5348.15811877.8614302.41083026.435.09121.95473.058.24797.62815.18570.65415.461.085250.64534.23929.9789.71568.683.73601.91251.9573.8454.2513.01432.5513.75273.6658.0671.1383.92241.5665.86119.5350.0733.81175.6737.93157.3838157.1463.1594.6510.48565.2119.39306.992.087143.564.793171.5448.8188.2420.4929824428623651.63202513072526357135475852877523.68OpenBenchmarking.org

Llama.cpp

Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048bbac369121513.3913.349.361. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Llama.cpp

Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048bbbac369121513.4013.3812.489.381. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Blender

Blend File: Pabellon Barcelona - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Pabellon Barcelona - Compute: CPU-Onlybbbac2004006008001000801.74801.96916.891054.24

Blender

Blend File: Classroom - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Classroom - Compute: CPU-Onlybbbac2004006008001000687.92753.66755.48864.22

Rustls

Benchmark: handshake-resume - Suite: TLS13_CHACHA20_POLY1305_SHA256

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake-resume - Suite: TLS13_CHACHA20_POLY1305_SHA256babbc20K40K60K80K100K111914.76105546.18105168.67105132.051. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

Llama.cpp

Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024bbbac4812162013.7613.7612.777.171. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Rustls

Benchmark: handshake-ticket - Suite: TLS13_CHACHA20_POLY1305_SHA256

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake-ticket - Suite: TLS13_CHACHA20_POLY1305_SHA256bacbb20K40K60K80K100K112736.73106054.04105650.04104991.401. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

Rustls

Benchmark: handshake-ticket - Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake-ticket - Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384cbabb80K160K240K320K400K379788.60379460.51356237.68355887.321. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

Llama.cpp

Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024bbbac4812162013.7913.7713.709.591. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Rustls

Benchmark: handshake-resume - Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake-resume - Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384cbbba100K200K300K400K500K448873.55445443.44416122.79415559.301. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

Blender

Blend File: Fishy Cat - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Fishy Cat - Compute: CPU-Onlybbbac100200300400500349.60399.23403.53450.68

Blender

Blend File: Junkshop - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Junkshop - Compute: CPU-Onlybcbba80160240320400322.47355.04366.21368.84

OpenVINO GenAI

Model: Gemma-7b-int4-ov - Device: CPU - Time Per Output Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: Gemma-7b-int4-ov - Device: CPU - Time Per Output Tokenbbc60120180240300222.88278.26

OpenVINO GenAI

Model: Gemma-7b-int4-ov - Device: CPU - Time To First Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: Gemma-7b-int4-ov - Device: CPU - Time To First Tokenbbc2004006008001000943.041158.18

ACES DGEMM

Sustained Floating-Point Rate

OpenBenchmarking.orgGFLOP/s, More Is BetterACES DGEMM 1.0Sustained Floating-Point Ratebbbca4080120160200181.33181.14180.52179.461. (CC) gcc options: -ffast-math -mavx2 -O3 -fopenmp -lopenblas

Rustls

Benchmark: handshake - Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake - Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384bcabb20K40K60K80K100K97635.3092487.8292044.9091665.531. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

Llama.cpp

Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512bbbac4812162014.1314.1113.097.211. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Blender

Blend File: BMW27 - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: BMW27 - Compute: CPU-Onlybcbba60120180240300235.55237.14270.09271.37

Llama.cpp

Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 2048bbac122436486052.3152.1844.051. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Llama.cpp

Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512bbbac4812162014.8214.1014.079.721. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Palabos

Grid Size: 100

OpenBenchmarking.orgMega Site Updates Per Second, More Is BetterPalabos 2.3Grid Size: 100acbbb102030405043.3141.1940.8426.531. (CXX) g++ options: -std=c++17 -pedantic -O3 -rdynamic -lcrypto -lcurl -lsz -lz -ldl -lm

OpenVINO GenAI

Model: Gemma-7b-int4-ov - Device: CPU

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: Gemma-7b-int4-ov - Device: CPUbbc1.01032.02063.03094.04125.05154.493.59

Rustls

Benchmark: handshake - Suite: TLS13_CHACHA20_POLY1305_SHA256

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake - Suite: TLS13_CHACHA20_POLY1305_SHA256bbbca2K4K6K8K10K10102.429387.989367.739355.191. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

Rustls

Benchmark: handshake - Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake - Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256bcbba2K4K6K8K10K10169.479594.009571.629526.431. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

Timed Eigen Compilation

Time To Compile

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Eigen Compilation 3.4.0Time To Compileabbbc306090120150147.80147.96148.03148.04

Palabos

Grid Size: 400

OpenBenchmarking.orgMega Site Updates Per Second, More Is BetterPalabos 2.3Grid Size: 400acbbb2040608010074.9371.4071.1350.731. (CXX) g++ options: -std=c++17 -pedantic -O3 -rdynamic -lcrypto -lcurl -lsz -lz -ldl -lm

Llama.cpp

Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Text Generation 128

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Text Generation 128bbbac2468106.426.426.184.151. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

vkpeak

int16-vec4

OpenBenchmarking.orgGIOPS, More Is Bettervkpeak 20240505int16-vec4abcbb4008001200160020001919.651919.431919.401609.56

vkpeak

int16-scalar

OpenBenchmarking.orgGIOPS, More Is Bettervkpeak 20240505int16-scalarabcbb20040060080010001013.631013.521013.51939.02

vkpeak

int32-vec4

OpenBenchmarking.orgGIOPS, More Is Bettervkpeak 20240505int32-vec4acbbb120240360480600550.78550.72550.70542.62

vkpeak

int32-scalar

OpenBenchmarking.orgGIOPS, More Is Bettervkpeak 20240505int32-scalarabcbb100200300400500473.56473.51473.50473.49

vkpeak

fp16-vec4

OpenBenchmarking.orgGFLOPS, More Is Bettervkpeak 20240505fp16-vec4abcbb80016002400320040003551.693551.193551.163551.00

vkpeak

fp16-scalar

OpenBenchmarking.orgGFLOPS, More Is Bettervkpeak 20240505fp16-scalaracbbb50010001500200025002503.322502.982502.952502.82

vkpeak

fp32-vec4

OpenBenchmarking.orgGFLOPS, More Is Bettervkpeak 20240505fp32-vec4abcbb4008001200160020001652.721652.471652.461652.35

vkpeak

fp32-scalar

OpenBenchmarking.orgGFLOPS, More Is Bettervkpeak 20240505fp32-scalaracbbb4008001200160020001868.441868.281868.131868.01

OpenVINO GenAI

Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time Per Output Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time Per Output Tokenabbc2040608010094.5794.78104.88

OpenVINO GenAI

Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time To First Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time To First Tokenabbc50100150200250191.33191.74238.24

OpenVINO GenAI

Model: Falcon-7b-instruct-int4-ov - Device: CPU

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: Falcon-7b-instruct-int4-ov - Device: CPUabbc369121510.5710.559.53

Llama.cpp

Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 1024abbc132639526557.8857.7048.151. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Rustls

Benchmark: handshake-ticket - Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake-ticket - Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256bcabb200K400K600K800K1000K864815.55811877.86809762.81808705.161. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

Renaissance

Test: ALS Movie Lens

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: ALS Movie Lenscabbb3K6K9K12K15K14302.414459.414475.615204.0MIN: 13229.02 / MAX: 14790.02MIN: 13527.85 / MAX: 14809.99MIN: 13363.8 / MAX: 14850.22MIN: 14604.93 / MAX: 15315.73

Rustls

Benchmark: handshake-resume - Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake-resume - Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256bcabb200K400K600K800K1000K1154350.711083026.431082805.901079421.741. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

Llama.cpp

Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Text Generation 128

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Text Generation 128bbac2468106.606.605.091. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

OpenVINO GenAI

Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time Per Output Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time Per Output Tokenabbc30609012015098.98100.11121.95

OpenVINO GenAI

Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time To First Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time To First Tokenbbac100200300400500335.04345.80473.05

OpenVINO GenAI

Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPUabbc369121510.109.998.20

Renaissance

Test: In-Memory Database Shootout

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: In-Memory Database Shootoutcbbab130026003900520065004797.65054.45302.86086.2MIN: 4238.93 / MAX: 5345.86MIN: 4319.98 / MAX: 5749.04MIN: 4531.82 / MAX: 5427.64MIN: 5581.49 / MAX: 6632.91

Renaissance

Test: Apache Spark PageRank

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Apache Spark PageRankbcbba70014002100280035002809.02815.12922.23063.2MIN: 2658.75MIN: 2628.83 / MAX: 2815.13MIN: 2752.32 / MAX: 3030.18MIN: 2879.98 / MAX: 3230.31

Renaissance

Test: Akka Unbalanced Cobwebbed Tree

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Akka Unbalanced Cobwebbed Treecbbba2K4K6K8K10K8570.68669.39157.19185.6MIN: 8210.22 / MAX: 9186.21MIN: 8535.62 / MAX: 9221.73MAX: 9628.95MIN: 9185.56 / MAX: 9829.42

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Face Detection FP16 - Device: CPUbbbac120024003600480060004489.064504.415004.255415.46MIN: 2543.37 / MAX: 7801.22MIN: 2560.7 / MAX: 7228.61MIN: 2565.77 / MAX: 11954.25MIN: 2567.66 / MAX: 21634.631. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Face Detection FP16 - Device: CPUbbbac0.29480.58960.88441.17921.4741.311.311.171.081. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Renaissance

Test: Savina Reactors.IO

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Savina Reactors.IObcabb110022003300440055005229.65250.65338.75360.8MAX: 7901.15MIN: 5250.57 / MAX: 7316.25MAX: 7566.83MIN: 5360.76 / MAX: 7595.61

Renaissance

Test: Gaussian Mixture Model

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Gaussian Mixture Modelcbbba100020003000400050004534.24581.14616.04643.4MIN: 4412.76 / MAX: 4943.27MIN: 4423.98 / MAX: 5050.76MAX: 5157.94MIN: 4643.38 / MAX: 5248.85

Renaissance

Test: Finagle HTTP Requests

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Finagle HTTP Requestsbcbba90018002700360045003884.63929.94297.24385.9MIN: 3335.79 / MAX: 4399.52MIN: 3248.37 / MAX: 4520.84MAX: 4817.4MAX: 5421.32

Renaissance

Test: Apache Spark Bayes

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Apache Spark Bayesabbbc2004006008001000638.2716.9745.2789.7MIN: 580.8 / MAX: 828.74MIN: 591.68 / MAX: 844.76MIN: 609.15 / MAX: 891.9MIN: 640.59 / MAX: 897.21

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Face Detection FP16-INT8 - Device: CPUbbbac300600900120015001281.771285.291402.501568.68MIN: 663.22 / MAX: 2901.61MIN: 661.66 / MAX: 2931.41MIN: 660.35 / MAX: 5619.27MIN: 666.6 / MAX: 9839.791. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Face Detection FP16-INT8 - Device: CPUbbbac1.03952.0793.11854.1585.19754.624.614.273.731. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Renaissance

Test: Random Forest

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Random Forestabcbb130260390520650570.1600.6601.9624.2MIN: 534.15 / MAX: 651.47MIN: 560.53 / MAX: 656.17MIN: 554.87 / MAX: 660.27MIN: 564.93 / MAX: 776.27

Renaissance

Test: Genetic Algorithm Using Jenetics + Futures

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Genetic Algorithm Using Jenetics + Futuresbbbac300600900120015001222.81230.51244.51251.9MIN: 1123.42 / MAX: 1307.54MIN: 1114.96 / MAX: 1356.94MIN: 1130.89 / MAX: 1359.65MIN: 1146.96 / MAX: 1404.63

Renaissance

Test: Scala Dotty

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Scala Dottyabcbb130260390520650550.0561.4573.8596.6MIN: 463.24 / MAX: 928.24MIN: 478.02 / MAX: 968.67MIN: 477.81 / MAX: 939.92MIN: 486.85 / MAX: 1078.3

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Person Detection FP16 - Device: CPUbbbac100200300400500410.91411.33439.61454.25MIN: 222.81 / MAX: 845.42MIN: 224.97 / MAX: 963.04MIN: 232.29 / MAX: 1903.58MIN: 192.15 / MAX: 1904.481. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Person Detection FP16 - Device: CPUbbbac4812162014.5714.5313.3813.011. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Machine Translation EN To DE FP16 - Device: CPUbbbac90180270360450350.74352.81392.48432.55MIN: 169.89 / MAX: 756.81MIN: 183.5 / MAX: 837.79MIN: 168.33 / MAX: 1771.01MIN: 164.06 / MAX: 1770.861. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Machine Translation EN To DE FP16 - Device: CPUbbbac4812162017.0516.9615.2513.751. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Handwritten English Recognition FP16 - Device: CPUbbbac60120180240300240.71246.56259.96273.66MIN: 129.63 / MAX: 979.78MIN: 128.61 / MAX: 661.55MIN: 130.23 / MAX: 1000.64MIN: 131.85 / MAX: 999.971. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Handwritten English Recognition FP16 - Device: CPUbbbac153045607566.3364.7161.0558.061. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Road Segmentation ADAS FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Road Segmentation ADAS FP16-INT8 - Device: CPUbbbac163248648059.4259.6265.6871.13MIN: 29.99 / MAX: 117.72MIN: 31.69 / MAX: 109.78MIN: 30.63 / MAX: 319.44MIN: 30.21 / MAX: 364.451. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Road Segmentation ADAS FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Road Segmentation ADAS FP16-INT8 - Device: CPUbbbac20406080100100.83100.4791.1083.921. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Handwritten English Recognition FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Handwritten English Recognition FP16-INT8 - Device: CPUbbbac50100150200250210.75217.86238.66241.56MIN: 103.37 / MAX: 548.96MIN: 103.9 / MAX: 635.93MIN: 101.28 / MAX: 947.08MIN: 104.83 / MAX: 911.971. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Handwritten English Recognition FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Handwritten English Recognition FP16-INT8 - Device: CPUbbbac2040608010075.7473.2866.6265.861. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Road Segmentation ADAS FP16 - Device: CPUbbbac306090120150114.32114.65117.50119.53MIN: 48.61 / MAX: 405.31MIN: 76.78 / MAX: 342.17MIN: 45.67 / MAX: 445.31MIN: 72.87 / MAX: 434.21. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Road Segmentation ADAS FP16 - Device: CPUbbbac122436486052.3652.2450.7650.071. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Noise Suppression Poconet-Like FP16 - Device: CPUbbbac81624324026.5926.7129.1133.81MIN: 13.16 / MAX: 47.31MIN: 13.32 / MAX: 44.78MIN: 13.53 / MAX: 79.9MIN: 13.51 / MAX: 143.781. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Noise Suppression Poconet-Like FP16 - Device: CPUbbbac50100150200250224.66223.62205.02175.671. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Person Vehicle Bike Detection FP16 - Device: CPUbbbac91827364530.4830.6834.2137.93MIN: 13.66 / MAX: 56.89MIN: 16.13 / MAX: 55.32MIN: 14.72 / MAX: 152.84MIN: 15.71 / MAX: 152.861. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Person Vehicle Bike Detection FP16 - Device: CPUbbbac4080120160200196.37195.02174.78157.381. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Person Re-Identification Retail FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Person Re-Identification Retail FP16 - Device: CPUbbbac91827364529.2829.4232.5238.00MIN: 13.23 / MAX: 56.48MIN: 13.3 / MAX: 50.36MIN: 13.27 / MAX: 98.34MIN: 13.58 / MAX: 150.481. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Person Re-Identification Retail FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Person Re-Identification Retail FP16 - Device: CPUbbbac4080120160200204.40203.39183.94157.141. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Vehicle Detection FP16 - Device: CPUbbbac142842567059.3759.8762.0163.15MIN: 31.87 / MAX: 215.48MIN: 26.15 / MAX: 156.54MIN: 27.87 / MAX: 273.2MIN: 32.01 / MAX: 273.441. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Vehicle Detection FP16 - Device: CPUbbbac20406080100100.7699.9896.2894.651. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Face Detection Retail FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Face Detection Retail FP16-INT8 - Device: CPUbbbac36912158.098.158.7810.48MIN: 3.69 / MAX: 17.48MIN: 3.56 / MAX: 16.97MIN: 3.74 / MAX: 28.34MIN: 3.94 / MAX: 72.261. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Face Detection Retail FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Face Detection Retail FP16-INT8 - Device: CPUbbbac160320480640800737.31731.87678.84565.211. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Face Detection Retail FP16 - Device: CPUbbbac51015202515.9916.0417.7919.39MIN: 7.49 / MAX: 39.2MIN: 7.67 / MAX: 35.36MIN: 6.93 / MAX: 78.65MIN: 8.01 / MAX: 78.861. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Face Detection Retail FP16 - Device: CPUbbbac80160240320400373.56372.29335.00306.991. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUbbbac0.4680.9361.4041.8722.341.531.531.662.08MIN: 0.48 / MAX: 5.15MIN: 0.48 / MAX: 4.89MIN: 0.45 / MAX: 6.8MIN: 0.46 / MAX: 11.031. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUbbbac2K4K6K8K10K9615.649570.878888.157143.561. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUbbbac1.07782.15563.23344.31125.3893.623.634.014.79MIN: 1.25 / MAX: 11.17MIN: 1.26 / MAX: 10.8MIN: 1.23 / MAX: 11.41MIN: 1.1 / MAX: 20.471. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUbbbac90018002700360045004149.864146.333750.573171.541. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO GenAI

Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time Per Output Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time Per Output Tokenabbc112233445544.1244.2848.81

OpenVINO GenAI

Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time To First Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time To First Tokenbbac2040608010073.0380.6488.24

OpenVINO GenAI

Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPUabbc51015202522.6622.5820.49

SuperTuxKart

Resolution: 1920 x 1080 - Graphics Effects: High - Renderer: Vulkan - Mostly Stable FPS

OpenBenchmarking.orgFrames Per Second, More Is BetterSuperTuxKart 1.5 Beta 1Resolution: 1920 x 1080 - Graphics Effects: High - Renderer: Vulkan - Mostly Stable FPScabbb60120180240300298297296280

SuperTuxKart

Resolution: 1920 x 1080 - Graphics Effects: High - Renderer: Vulkan - Steady FPS

OpenBenchmarking.orgFrames Per Second, More Is BetterSuperTuxKart 1.5 Beta 1Resolution: 1920 x 1080 - Graphics Effects: High - Renderer: Vulkan - Steady FPSbacbb50100150200250248245244229

SuperTuxKart

Resolution: 1920 x 1080 - Graphics Effects: Ultimate - Renderer: Vulkan - Mostly Stable FPS

OpenBenchmarking.orgFrames Per Second, More Is BetterSuperTuxKart 1.5 Beta 1Resolution: 1920 x 1080 - Graphics Effects: Ultimate - Renderer: Vulkan - Mostly Stable FPSacbbb60120180240300294286286268

SuperTuxKart

Resolution: 1920 x 1080 - Graphics Effects: Ultimate - Renderer: Vulkan - Steady FPS

OpenBenchmarking.orgFrames Per Second, More Is BetterSuperTuxKart 1.5 Beta 1Resolution: 1920 x 1080 - Graphics Effects: Ultimate - Renderer: Vulkan - Steady FPSacbbb50100150200250245236236211

Llama.cpp

Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512abbc142842567061.5161.4251.601. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

SuperTuxKart

Resolution: 1920 x 1080 - Graphics Effects: Low - Renderer: Vulkan - Mostly Stable FPS

OpenBenchmarking.orgFrames Per Second, More Is BetterSuperTuxKart 1.5 Beta 1Resolution: 1920 x 1080 - Graphics Effects: Low - Renderer: Vulkan - Mostly Stable FPSacbbb70140210280350321320318303

SuperTuxKart

Resolution: 1920 x 1080 - Graphics Effects: Low - Renderer: Vulkan - Steady FPS

OpenBenchmarking.orgFrames Per Second, More Is BetterSuperTuxKart 1.5 Beta 1Resolution: 1920 x 1080 - Graphics Effects: Low - Renderer: Vulkan - Steady FPScabbb50100150200250251250248151

SuperTuxKart

Resolution: 1920 x 1080 - Graphics Effects: Medium - Renderer: Vulkan - Mostly Stable FPS

OpenBenchmarking.orgFrames Per Second, More Is BetterSuperTuxKart 1.5 Beta 1Resolution: 1920 x 1080 - Graphics Effects: Medium - Renderer: Vulkan - Mostly Stable FPSacbbb70140210280350314307302286

SuperTuxKart

Resolution: 1920 x 1080 - Graphics Effects: Medium - Renderer: Vulkan - Steady FPS

OpenBenchmarking.orgFrames Per Second, More Is BetterSuperTuxKart 1.5 Beta 1Resolution: 1920 x 1080 - Graphics Effects: Medium - Renderer: Vulkan - Steady FPScbabb60120180240300252247247236

SuperTuxKart

Resolution: 1920 x 1080 - Graphics Effects: High - Renderer: OpenGL - Mostly Stable FPS

OpenBenchmarking.orgFrames Per Second, More Is BetterSuperTuxKart 1.5 Beta 1Resolution: 1920 x 1080 - Graphics Effects: High - Renderer: OpenGL - Mostly Stable FPSacbbb153045607565636261

SuperTuxKart

Resolution: 1920 x 1080 - Graphics Effects: High - Renderer: OpenGL - Steady FPS

OpenBenchmarking.orgFrames Per Second, More Is BetterSuperTuxKart 1.5 Beta 1Resolution: 1920 x 1080 - Graphics Effects: High - Renderer: OpenGL - Steady FPSacbbb132639526559575755

SuperTuxKart

Resolution: 1920 x 1080 - Graphics Effects: Low - Renderer: OpenGL - Mostly Stable FPS

OpenBenchmarking.orgFrames Per Second, More Is BetterSuperTuxKart 1.5 Beta 1Resolution: 1920 x 1080 - Graphics Effects: Low - Renderer: OpenGL - Mostly Stable FPScbabb306090120150135135135131

SuperTuxKart

Resolution: 1920 x 1080 - Graphics Effects: Low - Renderer: OpenGL - Steady FPS

OpenBenchmarking.orgFrames Per Second, More Is BetterSuperTuxKart 1.5 Beta 1Resolution: 1920 x 1080 - Graphics Effects: Low - Renderer: OpenGL - Steady FPSabbcb306090120150113594746

SuperTuxKart

Resolution: 1920 x 1080 - Graphics Effects: Ultimate - Renderer: OpenGL - Mostly Stable FPS

OpenBenchmarking.orgFrames Per Second, More Is BetterSuperTuxKart 1.5 Beta 1Resolution: 1920 x 1080 - Graphics Effects: Ultimate - Renderer: OpenGL - Mostly Stable FPSacbbb132639526560585656

SuperTuxKart

Resolution: 1920 x 1080 - Graphics Effects: Ultimate - Renderer: OpenGL - Steady FPS

OpenBenchmarking.orgFrames Per Second, More Is BetterSuperTuxKart 1.5 Beta 1Resolution: 1920 x 1080 - Graphics Effects: Ultimate - Renderer: OpenGL - Steady FPSacbbb122436486055525150

SuperTuxKart

Resolution: 1920 x 1080 - Graphics Effects: Medium - Renderer: OpenGL - Mostly Stable FPS

OpenBenchmarking.orgFrames Per Second, More Is BetterSuperTuxKart 1.5 Beta 1Resolution: 1920 x 1080 - Graphics Effects: Medium - Renderer: OpenGL - Mostly Stable FPScbabb2040608010087868681

SuperTuxKart

Resolution: 1920 x 1080 - Graphics Effects: Medium - Renderer: OpenGL - Steady FPS

OpenBenchmarking.orgFrames Per Second, More Is BetterSuperTuxKart 1.5 Beta 1Resolution: 1920 x 1080 - Graphics Effects: Medium - Renderer: OpenGL - Steady FPSacbbb2040608010076757370

Llama.cpp

Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Text Generation 128

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Text Generation 128bbac71421283528.1828.1323.681. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas


Phoronix Test Suite v10.8.5