llamacpp-mistral-07-03-prompt_processing_2048

AMD Ryzen 5 5600X 6-Core testing with a ASUS CROSSHAIR VI HERO (8801 BIOS) and Gigabyte NVIDIA GeForce RTX 4070 Ti SUPER 16GB on Ubuntu 24.10 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2501078-NE-LLAMACPPM09.

llamacpp-mistral-07-03-prompt_processing_2048ProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerDisplay DriverOpenGLCompilerFile-SystemScreen Resolutionllamacpp-mistral-07-03-prompt_processing_2048AMD Ryzen 5 5600X 6-Core @ 4.65GHz (6 Cores / 12 Threads)ASUS CROSSHAIR VI HERO (8801 BIOS)AMD Starship/Matisse32GB1000GB Samsung SSD 980 PRO with Heatsink 1TB + 4001GB Samsung SSD 870 + 1000GB Samsung SSD 860 + 500GB Samsung SSD 850 + 250GB Samsung SSD 840 + 8002GB ASM1156-PM + 4001GB ASM1156-PMGigabyte NVIDIA GeForce RTX 4070 Ti SUPER 16GBNVIDIA GA104 HD AudioEV2460 + AW3225QFIntel I211Ubuntu 24.106.11.0-13-generic (x86_64)Xfce 4.18X Server 1.21.1.13NVIDIA 560.35.034.6.0GCC 14.2.0 + CUDA 12.1ext44224x1296OpenBenchmarking.org- Transparent Huge Pages: madvise- NVM_CD_FLAGS=- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2,rust --enable-libphobos-checking=release --enable-libstdcxx-backtrace --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-14-zdkDXv/gcc-14-14.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-14-zdkDXv/gcc-14-14.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: amd-pstate-epp performance (Boost: Enabled EPP: performance) - CPU Microcode: 0xa201210- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of Safe RET + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines; IBPB: conditional; IBRS_FW; STIBP: always-on; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Not affected + tsx_async_abort: Not affected

llamacpp-mistral-07-03-prompt_processing_2048llama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 2048Standard ErrorStandard Deviationllamacpp-mistral-07-03-prompt_processing_204830.870.372.07%OpenBenchmarking.org

Llama.cpp

Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048llamacpp-mistral-07-03-prompt_processing_2048714212835SE +/- 0.37, N = 330.871. (CXX) g++ options: -O3

Phoronix Test Suite v10.8.5