llama1

Intel Core i7-8550U testing with a LENOVO ThinkPad T480 20L5000AMZ (N24ET76W 1.51 BIOS) and Intel UHD 620 on Ubuntu 24.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2407197-NE-LLAMA185233.

llama1ProcessorMotherboardChipsetMemoryDiskGraphicsAudioNetworkOSKernelDesktopDisplay ServerCompilerFile-SystemScreen ResolutionIntel Core i7-8550U - Intel UHD 620 - LENOVO ThinkPadIntel Core i7-8550U @ 4.00GHz (4 Cores / 8 Threads)LENOVO ThinkPad T480 20L5000AMZ (N24ET76W 1.51 BIOS)Intel Xeon E3-1200 v6/7th2 x 16GB DDR4-2400MT/s CT16G4SFD824A.M16FR2048GB Lexar SSD NM790 2TBIntel UHD 620Realtek ALC257Intel I219-V + Intel 8265 / 8275Ubuntu 24.046.8.0-38-generic (x86_64)GNOME Shell 46.0X Server + WaylandGCC 13.2.0ext41920x1080OpenBenchmarking.org- Transparent Huge Pages: madvise- NVM_CD_FLAGS=-q- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-backtrace --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-uJ7kn6/gcc-13-13.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-uJ7kn6/gcc-13-13.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: intel_pstate powersave (EPP: performance) - CPU Microcode: 0xf4 - Thermald 2.5.6 - gather_data_sampling: Mitigation of Microcode + itlb_multihit: KVM: Mitigation of VMX unsupported + l1tf: Mitigation of PTE Inversion + mds: Mitigation of Clear buffers; SMT vulnerable + meltdown: Mitigation of PTI + mmio_stale_data: Mitigation of Clear buffers; SMT vulnerable + reg_file_data_sampling: Not affected + retbleed: Mitigation of IBRS + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of IBRS; IBPB: conditional; STIBP: conditional; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Mitigation of Microcode + tsx_async_abort: Not affected

llama1llama-cpp: Meta-Llama-3-8B-Instruct-Q8_0.ggufStandard ErrorStandard DeviationIntel Core i7-8550U - Intel UHD 620 - LENOVO ThinkPad3.110.033.18%OpenBenchmarking.org

Llama.cpp

Model: Meta-Llama-3-8B-Instruct-Q8_0.gguf

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b3067Model: Meta-Llama-3-8B-Instruct-Q8_0.ggufIntel Core i7-8550U - Intel UHD 620 - LENOVO ThinkPad0.69981.39962.09942.79923.499SE +/- 0.03, N = 123.111. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -march=native -mtune=native -lopenblas

Phoronix Test Suite v10.8.5