OV - exec - Loops

▶Scalability Runs Description

Run 1x1	Number processes: 1Number nodes: 1Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings Dataset: Run Directory: /home/eoseret/Tools/QaaS/qaas_runs/ip-172-31-47-249.ec2.internal/176-131-3962/llama.cpp/run/oneview_runs/multicore/armclang/oneview_run_1761315400OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1
Run 1x2	Number processes: 1Number nodes: 1Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings Dataset: Run Directory: /home/eoseret/Tools/QaaS/qaas_runs/ip-172-31-47-249.ec2.internal/176-131-3962/llama.cpp/run/oneview_runs/multicore/armclang/oneview_run_1761315400OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads
Run 1x4	Number processes: 1Number nodes: 1Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings Dataset: Run Directory: /home/eoseret/Tools/QaaS/qaas_runs/ip-172-31-47-249.ec2.internal/176-131-3962/llama.cpp/run/oneview_runs/multicore/armclang/oneview_run_1761315400OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads
Run 1x8	Number processes: 1Number nodes: 1Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings Dataset: Run Directory: /home/eoseret/Tools/QaaS/qaas_runs/ip-172-31-47-249.ec2.internal/176-131-3962/llama.cpp/run/oneview_runs/multicore/armclang/oneview_run_1761315400OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads
Run 1x16	Number processes: 1Number nodes: 1Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings Dataset: Run Directory: /home/eoseret/Tools/QaaS/qaas_runs/ip-172-31-47-249.ec2.internal/176-131-3962/llama.cpp/run/oneview_runs/multicore/armclang/oneview_run_1761315400OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads
Run 1x24	Number processes: 1Number nodes: 1Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings Dataset: Run Directory: /home/eoseret/Tools/QaaS/qaas_runs/ip-172-31-47-249.ec2.internal/176-131-3962/llama.cpp/run/oneview_runs/multicore/armclang/oneview_run_1761315400OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads
Run 1x32	Number processes: 1Number nodes: 1Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings Dataset: Run Directory: /home/eoseret/Tools/QaaS/qaas_runs/ip-172-31-47-249.ec2.internal/176-131-3962/llama.cpp/run/oneview_runs/multicore/armclang/oneview_run_1761315400OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads
Run 1x40	Number processes: 1Number nodes: 1Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings Dataset: Run Directory: /home/eoseret/Tools/QaaS/qaas_runs/ip-172-31-47-249.ec2.internal/176-131-3962/llama.cpp/run/oneview_runs/multicore/armclang/oneview_run_1761315400OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads
Run 1x48	Number processes: 1Number nodes: 1Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings Dataset: Run Directory: /home/eoseret/Tools/QaaS/qaas_runs/ip-172-31-47-249.ec2.internal/176-131-3962/llama.cpp/run/oneview_runs/multicore/armclang/oneview_run_1761315400OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads
Run 1x56	Number processes: 1Number nodes: 1Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings Dataset: Run Directory: /home/eoseret/Tools/QaaS/qaas_runs/ip-172-31-47-249.ec2.internal/176-131-3962/llama.cpp/run/oneview_runs/multicore/armclang/oneview_run_1761315400OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads
Run 1x64	Number processes: 1Number nodes: 1Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings Dataset: Run Directory: /home/eoseret/Tools/QaaS/qaas_runs/ip-172-31-47-249.ec2.internal/176-131-3962/llama.cpp/run/oneview_runs/multicore/armclang/oneview_run_1761315400OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads
Run 1x72	Number processes: 1Number nodes: 1Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings Dataset: Run Directory: /home/eoseret/Tools/QaaS/qaas_runs/ip-172-31-47-249.ec2.internal/176-131-3962/llama.cpp/run/oneview_runs/multicore/armclang/oneview_run_1761315400OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads
Run 1x80	Number processes: 1Number nodes: 1Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings Dataset: Run Directory: /home/eoseret/Tools/QaaS/qaas_runs/ip-172-31-47-249.ec2.internal/176-131-3962/llama.cpp/run/oneview_runs/multicore/armclang/oneview_run_1761315400OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads
Run 1x88	Number processes: 1Number nodes: 1Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings Dataset: Run Directory: /home/eoseret/Tools/QaaS/qaas_runs/ip-172-31-47-249.ec2.internal/176-131-3962/llama.cpp/run/oneview_runs/multicore/armclang/oneview_run_1761315400OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads
Run 1x96	Number processes: 1Number nodes: 1Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings Dataset: Run Directory: /home/eoseret/Tools/QaaS/qaas_runs/ip-172-31-47-249.ec2.internal/176-131-3962/llama.cpp/run/oneview_runs/multicore/armclang/oneview_run_1761315400OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

Loop id	Source Location	Source Function	Level	Max Thread Time / Walltime 1x1 (%)	Max Thread Time / Walltime 1x2 (%)	Max Thread Time / Walltime 1x4 (%)	Max Thread Time / Walltime 1x8 (%)	Max Thread Time / Walltime 1x16 (%)	Max Thread Time / Walltime 1x24 (%)	Max Thread Time / Walltime 1x32 (%)	Max Thread Time / Walltime 1x40 (%)	Max Thread Time / Walltime 1x48 (%)	Max Thread Time / Walltime 1x56 (%)	Max Thread Time / Walltime 1x64 (%)	Max Thread Time / Walltime 1x72 (%)	Max Thread Time / Walltime 1x80 (%)	Max Thread Time / Walltime 1x88 (%)	Max Thread Time / Walltime 1x96 (%)	Exclusive Coverage 1x1 (%)	Exclusive Coverage 1x2 (%)	Exclusive Coverage 1x4 (%)	Exclusive Coverage 1x8 (%)	Exclusive Coverage 1x16 (%)	Exclusive Coverage 1x24 (%)	Exclusive Coverage 1x32 (%)	Exclusive Coverage 1x40 (%)	Exclusive Coverage 1x48 (%)	Exclusive Coverage 1x56 (%)	Exclusive Coverage 1x64 (%)	Exclusive Coverage 1x72 (%)	Exclusive Coverage 1x80 (%)	Exclusive Coverage 1x88 (%)	Exclusive Coverage 1x96 (%)	Inclusive Coverage 1x1 (%)	Inclusive Coverage 1x2 (%)	Inclusive Coverage 1x4 (%)	Inclusive Coverage 1x8 (%)	Inclusive Coverage 1x16 (%)	Inclusive Coverage 1x24 (%)	Inclusive Coverage 1x32 (%)	Inclusive Coverage 1x40 (%)	Inclusive Coverage 1x48 (%)	Inclusive Coverage 1x56 (%)	Inclusive Coverage 1x64 (%)	Inclusive Coverage 1x72 (%)	Inclusive Coverage 1x80 (%)	Inclusive Coverage 1x88 (%)	Inclusive Coverage 1x96 (%)	Max Exclusive Time Over Threads 1x1 (s)	Max Exclusive Time Over Threads 1x2 (s)	Max Exclusive Time Over Threads 1x4 (s)	Max Exclusive Time Over Threads 1x8 (s)	Max Exclusive Time Over Threads 1x16 (s)	Max Exclusive Time Over Threads 1x24 (s)	Max Exclusive Time Over Threads 1x32 (s)	Max Exclusive Time Over Threads 1x40 (s)	Max Exclusive Time Over Threads 1x48 (s)	Max Exclusive Time Over Threads 1x56 (s)	Max Exclusive Time Over Threads 1x64 (s)	Max Exclusive Time Over Threads 1x72 (s)	Max Exclusive Time Over Threads 1x80 (s)	Max Exclusive Time Over Threads 1x88 (s)	Max Exclusive Time Over Threads 1x96 (s)	Max Inclusive Time Over Threads 1x1 (s)	Max Inclusive Time Over Threads 1x2 (s)	Max Inclusive Time Over Threads 1x4 (s)	Max Inclusive Time Over Threads 1x8 (s)	Max Inclusive Time Over Threads 1x16 (s)	Max Inclusive Time Over Threads 1x24 (s)	Max Inclusive Time Over Threads 1x32 (s)	Max Inclusive Time Over Threads 1x40 (s)	Max Inclusive Time Over Threads 1x48 (s)	Max Inclusive Time Over Threads 1x56 (s)	Max Inclusive Time Over Threads 1x64 (s)	Max Inclusive Time Over Threads 1x72 (s)	Max Inclusive Time Over Threads 1x80 (s)	Max Inclusive Time Over Threads 1x88 (s)	Max Inclusive Time Over Threads 1x96 (s)	Exclusive Time w.r.t. Wall Time 1x1 (s)	Exclusive Time w.r.t. Wall Time 1x2 (s)	Exclusive Time w.r.t. Wall Time 1x4 (s)	Exclusive Time w.r.t. Wall Time 1x8 (s)	Exclusive Time w.r.t. Wall Time 1x16 (s)	Exclusive Time w.r.t. Wall Time 1x24 (s)	Exclusive Time w.r.t. Wall Time 1x32 (s)	Exclusive Time w.r.t. Wall Time 1x40 (s)	Exclusive Time w.r.t. Wall Time 1x48 (s)	Exclusive Time w.r.t. Wall Time 1x56 (s)	Exclusive Time w.r.t. Wall Time 1x64 (s)	Exclusive Time w.r.t. Wall Time 1x72 (s)	Exclusive Time w.r.t. Wall Time 1x80 (s)	Exclusive Time w.r.t. Wall Time 1x88 (s)	Exclusive Time w.r.t. Wall Time 1x96 (s)	Inclusive Time w.r.t. Wall Time 1x1 (s)	Inclusive Time w.r.t. Wall Time 1x2 (s)	Inclusive Time w.r.t. Wall Time 1x4 (s)	Inclusive Time w.r.t. Wall Time 1x8 (s)	Inclusive Time w.r.t. Wall Time 1x16 (s)	Inclusive Time w.r.t. Wall Time 1x24 (s)	Inclusive Time w.r.t. Wall Time 1x32 (s)	Inclusive Time w.r.t. Wall Time 1x40 (s)	Inclusive Time w.r.t. Wall Time 1x48 (s)	Inclusive Time w.r.t. Wall Time 1x56 (s)	Inclusive Time w.r.t. Wall Time 1x64 (s)	Inclusive Time w.r.t. Wall Time 1x72 (s)	Inclusive Time w.r.t. Wall Time 1x80 (s)	Inclusive Time w.r.t. Wall Time 1x88 (s)	Inclusive Time w.r.t. Wall Time 1x96 (s)	Nb Threads 1x1	Nb Threads 1x2	Nb Threads 1x4	Nb Threads 1x8	Nb Threads 1x16	Nb Threads 1x24	Nb Threads 1x32	Nb Threads 1x40	Nb Threads 1x48	Nb Threads 1x56	Nb Threads 1x64	Nb Threads 1x72	Nb Threads 1x80	Nb Threads 1x88	Nb Threads 1x96	Vectorization Ratio (%)	Vector Length Use (%)	Speedup If No Scalar Integer	Speedup If FP Vectorized	Speedup If Fully Vectorized	Speedup If Perfect Load Balancing 1x1	Speedup If Perfect Load Balancing 1x2	Speedup If Perfect Load Balancing 1x4	Speedup If Perfect Load Balancing 1x8	Speedup If Perfect Load Balancing 1x16	Speedup If Perfect Load Balancing 1x24	Speedup If Perfect Load Balancing 1x32	Speedup If Perfect Load Balancing 1x40	Speedup If Perfect Load Balancing 1x48	Speedup If Perfect Load Balancing 1x56	Speedup If Perfect Load Balancing 1x64	Speedup If Perfect Load Balancing 1x72	Speedup If Perfect Load Balancing 1x80	Speedup If Perfect Load Balancing 1x88	Speedup If Perfect Load Balancing 1x96	Stride 0	Stride 1	Stride n	Stride Unknown	Stride Indirect	Array Access Efficiency	(1x1) Efficiency	(1x1) Potential Speed-Up (%)	(1x2) Efficiency	(1x2) Potential Speed-Up (%)	(1x4) Efficiency	(1x4) Potential Speed-Up (%)	(1x8) Efficiency	(1x8) Potential Speed-Up (%)	(1x16) Efficiency	(1x16) Potential Speed-Up (%)	(1x24) Efficiency	(1x24) Potential Speed-Up (%)	(1x32) Efficiency	(1x32) Potential Speed-Up (%)	(1x40) Efficiency	(1x40) Potential Speed-Up (%)	(1x48) Efficiency	(1x48) Potential Speed-Up (%)	(1x56) Efficiency	(1x56) Potential Speed-Up (%)	(1x64) Efficiency	(1x64) Potential Speed-Up (%)	(1x72) Efficiency	(1x72) Potential Speed-Up (%)	(1x80) Efficiency	(1x80) Potential Speed-Up (%)	(1x88) Efficiency	(1x88) Potential Speed-Up (%)	(1x96) Efficiency	(1x96) Potential Speed-Up (%)
2354	libggml-cpu.so - kai_matmul_clamp_f32_qsi8d32p4x8_qsi4c32p4x8_16x4_neon_i8mm.c:131-131	kai_run_matmul_clamp_f32_qsi8d32p4x8_qsi4c32p4x8_16x4_neon_i8mm	Innermost	90.08	88.39	85.18	79.85	70.57	65.18	60.45	56.72	53.40	51.23	47.07	45.57	42.87	41.39	38.82	91.19	90.89	90.46	89.78	88.19	86.60	86.43	84.73	82.75	82.69	82.64	79.32	78.74	78.01	75.59	91.19	90.89	90.46	89.78	88.19	86.60	86.43	84.73	82.75	82.69	82.64	79.32	78.74	78.01	75.59	115.13	57.49	28.68	14.43	7.24	5.30	4.18	3.51	3.03	2.71	2.33	2.18	1.96	1.83	1.70	115.13	57.49	28.68	14.43	7.24	5.30	4.18	3.51	3.03	2.71	2.33	2.18	1.96	1.83	1.70	115.13	57.73	29.07	14.79	7.64	5.68	4.63	3.91	3.46	3.06	2.75	2.53	2.29	2.11	2.01	115.13	57.73	29.07	14.79	7.64	5.68	4.63	3.91	3.46	3.06	2.75	2.53	2.29	2.11	2.01	1	2	4	8	16	24	32	40	48	56	64	72	80	88	95	28.14	55.37	1	1	1.85	1	1	1	1	1.01	1.02	1.01	1.02	1.04	1.05	1.02	1.06	1.05	1.05	1.05	0	0	5	0	0	75.00	1	0	1	0.25	0.99	0.9	0.97	2.4	0.94	5.15	0.84	13.47	0.78	19.24	0.74	22.36	0.69	25.37	0.67	27.2	0.65	28.54	0.63	29.27	0.63	29.28	0.62	29.55	0.6	30.59
886	libggml-cpu.so - vec.cpp:385-387 [...]	ggml_vec_swiglu_f32	Single	2.76	2.68	2.69	2.66	2.24	2.15	1.73	1.61	1.67	1.60	1.61	1.46	1.31	1.47	1.60	2.79	2.74	2.75	2.70	2.65	2.62	2.30	2.01	2.15	1.65	1.92	1.89	1.87	1.82	1.79	2.79	2.74	2.75	2.70	2.65	2.62	2.30	2.01	2.15	1.65	1.92	1.89	1.87	1.82	1.79	3.52	1.74	0.90	0.48	0.23	0.17	0.12	0.10	0.09	0.09	0.08	0.07	0.06	0.06	0.07	3.52	1.74	0.90	0.48	0.23	0.17	0.12	0.10	0.09	0.09	0.08	0.07	0.06	0.06	0.07	3.52	1.74	0.88	0.44	0.23	0.17	0.12	0.09	0.09	0.06	0.06	0.06	0.05	0.05	0.05	3.52	1.74	0.88	0.44	0.23	0.17	0.12	0.09	0.09	0.06	0.06	0.06	0.05	0.05	0.05	1	2	4	8	16	24	32	40	47	52	64	64	74	86	86	70.45	83.38	1	1	1.21	1	1	1.04	1.11	1.06	1.11	1.09	1.23	1.23	1.52	1.51	1.27	1.25	1.56	1.65	0	0	0	0	0	0.00	1	0	1.01	0	1	0.01	0.99	0.02	0.96	0.11	0.86	0.38	0.89	0.25	0.95	0.1	0.82	0.39	1.03	0	0.86	0.26	0.81	0.35	0.81	0.35	0.81	0.34	0.77	0.41
1710	libggml-cpu.so - vec.h:282-662 [...]	ggml_compute_forward_flash_attn_ext	InBetween	0.93	0.95	0.91	0.89	1.07	0.86	0.79	0.89	0.88	0.85	0.81	0.94	0.65	0.68	0.80	0.94	0.94	0.90	0.79	1.00	0.82	0.67	0.76	0.72	0.59	0.71	0.67	0.61	0.59	0.59	1.26	1.30	1.27	1.20	1.34	1.11	1.01	0.98	1.05	0.84	0.96	0.94	0.83	0.80	0.83	1.19	0.62	0.31	0.16	0.11	0.07	0.06	0.05	0.05	0.05	0.04	0.05	0.03	0.03	0.04	1.58	0.86	0.44	0.24	0.14	0.09	0.08	0.06	0.06	0.05	0.05	0.05	0.04	0.04	0.05	1.19	0.60	0.29	0.13	0.09	0.05	0.04	0.04	0.03	0.02	0.02	0.02	0.02	0.02	0.02	1.58	0.83	0.41	0.20	0.12	0.07	0.05	0.05	0.04	0.03	0.03	0.03	0.02	0.02	0.02	1	2	4	8	16	24	32	40	48	54	64	70	72	84	88	16.83	53.17	3.04	1.03	1.39	1	1.03	1.07	1.27	1.35	1.42	1.7	1.78	1.96	2.34	2.03	2.51	1.87	2.18	2.56	NA	NA	NA	NA	NA	0.00	1	0	1	0	1.03	0	1.14	0	0.86	0.14	0.93	0.06	1.03	0	0.85	0.12	0.82	0.13	0.97	0.02	0.79	0.15	0.77	0.15	0.84	0.1	0.85	0.09	0.79	0.13
1713	libggml-cpu.so - vec.h:282-725 [...]	ggml_compute_forward_flash_attn_ext	Outermost	0.55	0.58	0.65	0.55	0.68	0.68	0.65	0.48	0.62	0.47	0.71	0.63	0.44	0.45	0.46	0.56	0.55	0.52	0.47	0.44	0.46	0.45	0.38	0.47	0.34	0.44	0.37	0.33	0.32	0.32	2.21	2.17	2.12	2.03	2.07	1.84	1.77	1.66	1.79	1.38	1.65	1.55	1.38	1.35	1.37	0.70	0.38	0.22	0.10	0.07	0.05	0.04	0.03	0.04	0.03	0.04	0.03	0.02	0.02	0.02	2.79	1.42	0.75	0.35	0.21	0.15	0.11	0.09	0.09	0.07	0.08	0.06	0.05	0.05	0.06	0.70	0.35	0.17	0.08	0.04	0.03	0.02	0.02	0.02	0.01	0.01	0.01	0.01	0.01	0.01	2.79	1.38	0.68	0.34	0.18	0.12	0.09	0.08	0.07	0.05	0.05	0.05	0.04	0.04	0.04	1	2	4	8	16	24	32	40	47	51	56	60	63	68	69	16.49	49.06	5.71	1	1.45	1	1.1	1.33	1.32	1.93	1.98	2.09	1.95	2.08	2.14	2.5	2.61	2	2.18	2.14	NA	NA	NA	NA	NA	0.00	1	0	1.02	0	1.05	0	1.13	0	1.15	0	0.97	0.01	0.91	0.04	1.01	0	0.75	0.12	1	0	0.75	0.11	0.83	0.06	0.91	0.03	0.93	0.02	0.87	0.04
2355	libggml-cpu.so - kai_matmul_clamp_f32_qsi8d32p4x8_qsi4c32p4x8_16x4_neon_i8mm.c:131-131	kai_run_matmul_clamp_f32_qsi8d32p4x8_qsi4c32p4x8_16x4_neon_i8mm	InBetween	0.38	0.38	0.52	0.58	0.58	0.61	0.94	0.65	0.79	0.75	0.60	0.73	0.76	0.68	0.69	0.39	0.38	0.49	0.43	0.47	0.42	0.56	0.44	0.52	0.47	0.52	0.43	0.42	0.49	0.43	91.58	91.27	90.94	90.21	88.67	87.02	86.99	85.17	83.27	83.16	83.16	79.75	79.16	78.50	76.02	0.49	0.25	0.17	0.11	0.06	0.05	0.07	0.04	0.05	0.04	0.03	0.04	0.04	0.03	0.03	115.62	57.74	28.85	14.50	7.28	5.31	4.20	3.52	3.04	2.72	2.35	2.18	1.97	1.84	1.71	0.49	0.24	0.16	0.07	0.04	0.03	0.03	0.02	0.02	0.02	0.02	0.01	0.01	0.01	0.01	115.62	57.97	29.23	14.86	7.68	5.71	4.66	3.93	3.48	3.08	2.76	2.55	2.30	2.12	2.03	1	2	4	8	16	24	32	37	48	53	58	66	73	77	79	27.59	55.71	1	1	1.62	1	1.04	1.14	1.51	1.55	1.98	2.42	2.08	2.45	2.59	1.9	2.85	3.23	2.41	2.69	NA	NA	NA	NA	NA	0.00	1	0	1.02	0	0.79	0.1	0.86	0.06	0.74	0.12	0.74	0.11	0.51	0.27	0.61	0.17	0.47	0.28	0.5	0.23	0.45	0.29	0.49	0.22	0.51	0.21	0.42	0.28	0.44	0.24
1718	libggml-cpu.so - vec.h:411-458	ggml_compute_forward_flash_attn_ext	Innermost	0.38	0.33	0.34	0.53	0.34	0.43	0.43	0.48	0.35	0.38	0.40	0.42	0.55	0.45	0.46	0.39	0.31	0.33	0.36	0.29	0.27	0.30	0.30	0.27	0.20	0.23	0.24	0.22	0.23	0.21	0.39	0.31	0.33	0.36	0.29	0.27	0.30	0.30	0.27	0.20	0.23	0.24	0.22	0.23	0.21	0.49	0.22	0.11	0.09	0.04	0.04	0.03	0.03	0.02	0.02	0.02	0.02	0.03	0.02	0.02	0.49	0.22	0.11	0.09	0.04	0.04	0.03	0.03	0.02	0.02	0.02	0.02	0.03	0.02	0.02	0.49	0.20	0.11	0.06	0.03	0.02	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.49	0.20	0.11	0.06	0.03	0.02	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	1	2	4	8	16	23	30	39	42	40	46	54	50	55	58	100	100	1	1	1	1	1.1	1.11	1.67	1.45	2.04	1.96	2.44	1.85	2.25	2.22	2.4	3.01	2.47	2.67	0	0	0	2	0	50.00	1	0	1.25	0	1.16	0	1.05	0	1.2	0	1.14	0	0.95	0.01	0.9	0.03	0.91	0.02	1.17	0	0.98	0	0.89	0.03	0.96	0.01	0.91	0.02	0.9	0.02
2317	libggml-cpu.so - kai_lhs_quant_pack_qsi8d32p4x8sb_f32_neon.c:96-258 [...]	kai_run_lhs_quant_pack_qsi8d32p4x8sb_f32_neon	Innermost	0.38	0.53	0.46	0.66	0.68	0.74	0.72	0.81	0.70	1.23	0.81	0.84	1.09	1.24	1.26	0.38	0.53	0.45	0.58	0.53	0.53	0.52	0.48	0.58	0.62	0.55	0.62	0.63	0.77	0.76	0.38	0.53	0.45	0.58	0.53	0.53	0.52	0.48	0.58	0.62	0.55	0.62	0.63	0.77	0.76	0.48	0.34	0.16	0.12	0.07	0.06	0.05	0.05	0.04	0.06	0.04	0.04	0.05	0.05	0.06	0.48	0.34	0.16	0.12	0.07	0.06	0.05	0.05	0.04	0.06	0.04	0.04	0.05	0.05	0.06	0.48	0.33	0.14	0.10	0.05	0.03	0.03	0.02	0.02	0.02	0.02	0.02	0.02	0.02	0.02	0.48	0.33	0.14	0.10	0.05	0.03	0.03	0.02	0.02	0.02	0.02	0.02	0.02	0.02	0.02	1	2	4	8	16	22	31	32	43	43	63	63	64	64	64	77.23	96.44	1	1	1.03	1	1.04	1.1	1.29	1.61	1.73	1.95	2.05	1.74	2.59	2.6	2.16	2.66	2.34	2.28	1	0	1	3	0	65.00	1	0	0.73	0.14	0.85	0.07	0.63	0.21	0.66	0.18	0.58	0.22	0.55	0.24	0.55	0.22	0.41	0.34	0.38	0.38	0.42	0.32	0.34	0.41	0.33	0.43	0.27	0.56	0.25	0.57
879	libggml-cpu.so - vec.cpp:231-262	ggml_vec_dot_f16	Single	0.32	0.34	0.36	0.36	0.44	0.55	0.36	0.48	0.44	0.47	0.50	0.52	0.55	0.56	0.46	0.33	0.33	0.34	0.30	0.32	0.29	0.31	0.23	0.25	0.23	0.28	0.22	0.26	0.25	0.24	0.33	0.33	0.34	0.30	0.32	0.29	0.31	0.23	0.25	0.23	0.28	0.22	0.26	0.25	0.24	0.42	0.22	0.12	0.07	0.05	0.05	0.03	0.03	0.02	0.03	0.02	0.03	0.03	0.02	0.02	0.42	0.22	0.12	0.07	0.05	0.05	0.03	0.03	0.02	0.03	0.02	0.03	0.03	0.02	0.02	0.42	0.21	0.11	0.05	0.03	0.02	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.42	0.21	0.11	0.05	0.03	0.02	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	1	2	4	8	16	24	32	32	38	42	55	48	57	58	61	100	100	1	1	1	1	1.05	1.12	1.37	1.73	2.57	1.7	2.56	2.21	2.59	2.81	2.93	2.88	2.93	2.52	0	0	0	14	0	50.00	1	0	0.98	0.01	0.95	0.02	1.06	0	0.94	0.02	0.91	0.03	0.79	0.06	0.97	0.01	0.81	0.05	0.87	0.03	0.71	0.08	0.82	0.04	0.68	0.08	0.69	0.08	0.68	0.08
1249	libggml-cpu.so - ops.cpp:4325-4326	ggml_compute_forward_rms_norm	Innermost	0.32	0.32	0.36	0.55	0.49	0.49	0.51	0.48	0.62	0.75	0.81	0.63	0.65	0.68	1.14	0.32	0.29	0.32	0.40	0.39	0.43	0.38	0.36	0.43	0.51	0.39	0.41	0.47	0.39	0.48	0.32	0.29	0.32	0.40	0.39	0.43	0.38	0.36	0.43	0.51	0.39	0.41	0.47	0.39	0.48	0.41	0.20	0.12	0.10	0.05	0.04	0.04	0.03	0.04	0.04	0.04	0.03	0.03	0.03	0.05	0.41	0.20	0.12	0.10	0.05	0.04	0.04	0.03	0.04	0.04	0.04	0.03	0.03	0.03	0.05	0.41	0.19	0.10	0.07	0.03	0.03	0.02	0.02	0.02	0.02	0.01	0.01	0.01	0.01	0.01	0.41	0.19	0.10	0.07	0.03	0.03	0.02	0.02	0.02	0.02	0.01	0.01	0.01	0.01	0.01	1	2	4	8	16	24	31	38	45	52	57	63	68	72	80	94.12	97.06	1	1	1.03	1	1.11	1.2	1.55	1.55	1.54	1.87	1.93	2.19	2.31	3.3	2.44	2.29	2.84	4.12	0	0	0	0	0	0.00	1	0	1.1	0	1.01	0	0.77	0.09	0.75	0.1	0.6	0.17	0.63	0.14	0.61	0.14	0.48	0.22	0.38	0.32	0.49	0.2	0.43	0.24	0.38	0.29	0.45	0.21	0.34	0.32
1721	libggml-cpu.so - ops.cpp:8778-8920 [...]	ggml_compute_forward_flash_attn_ext	InBetween	0.29	0.38	0.36	0.50	0.34	0.37	0.36	0.40	0.44	0.28	0.30	0.42	0.33	0.34	0.34	0.29	0.34	0.33	0.34	0.30	0.25	0.28	0.20	0.27	0.21	0.20	0.22	0.17	0.19	0.20	0.31	0.36	0.37	0.41	0.34	0.29	0.33	0.22	0.32	0.25	0.25	0.26	0.22	0.21	0.24	0.37	0.25	0.12	0.09	0.04	0.03	0.03	0.03	0.03	0.02	0.02	0.02	0.02	0.02	0.02	0.40	0.25	0.13	0.10	0.04	0.03	0.03	0.03	0.03	0.02	0.02	0.02	0.02	0.02	0.02	0.37	0.22	0.11	0.06	0.03	0.02	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.39	0.23	0.12	0.07	0.03	0.02	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	1	2	4	8	16	22	31	29	43	45	48	51	47	50	57	0	47.79	80	1.14	1.6	1	1.14	1.14	1.66	1.44	1.83	1.78	2.23	2.34	1.85	2	2.52	2.14	2.03	2.14	NA	NA	NA	NA	NA	0.00	1	0	0.84	0.05	0.86	0.05	0.82	0.06	0.88	0.04	0.93	0.02	0.75	0.07	0.99	0	0.67	0.09	0.84	0.03	0.84	0.03	0.73	0.06	0.9	0.02	0.81	0.03	0.73	0.05
2236	libggml-cpu.so - quants.c:2506-2590 [...]	ggml_vec_dot_q6_K_q8_K	Innermost	0.25	0.25	0.25	0.25	0.24	0.25	0.22	0.16	0.18	0.19	0.20	0.21	0.22	0.11	0.11	0.25	0.26	0.26	0.25	0.25	0.24	0.20	0.23	0.26	0.27	0.17	0.18	0.19	0.21	0.19	0.25	0.26	0.26	0.25	0.25	0.24	0.20	0.23	0.26	0.27	0.17	0.18	0.19	0.21	0.19	0.32	0.17	0.09	0.05	0.03	0.02	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.32	0.17	0.09	0.05	0.03	0.02	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.32	0.16	0.08	0.04	0.02	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.32	0.16	0.08	0.04	0.02	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	1	2	4	8	16	24	32	39	48	55	55	62	70	81	79	50.56	65.8	1	1	1.47	1	1.02	1.01	1.13	1.23	1.39	1.55	1.05	1.09	1.18	1.86	1.88	1.92	1	1	2	0	1	19	0	55.68	1	0	0.98	0.01	0.94	0.02	0.97	0.01	0.93	0.02	0.85	0.04	0.92	0.02	0.76	0.06	0.61	0.1	0.58	0.11	0.9	0.02	0.79	0.04	0.72	0.05	0.65	0.07	0.64	0.07
2337	libggml-cpu.so - kai_rhs_pack_nxk_qsi4c32pscalef16_qsu4c32s16s0.c:127-139 [...]	kai_run_rhs_pack_nxk_qsi4c32pscalef16_qsu4c32s16s0	Innermost	0.25	0.42	0.67	1.49	2.68	2.40	4.48	4.28	4.85	5.09	4.64	5.96	7.53	6.20	6.51	0.25	0.21	0.18	0.21	0.21	0.13	0.20	0.16	0.16	0.15	0.13	0.15	0.18	0.14	0.14	0.25	0.21	0.18	0.21	0.21	0.13	0.20	0.16	0.16	0.15	0.13	0.15	0.18	0.14	0.14	0.31	0.27	0.22	0.27	0.28	0.20	0.31	0.27	0.27	0.27	0.23	0.28	0.35	0.28	0.28	0.31	0.27	0.22	0.27	0.28	0.20	0.31	0.27	0.27	0.27	0.23	0.28	0.35	0.28	0.28	0.31	0.14	0.06	0.03	0.02	0.01	0.01	0.01	0.01	0.01	0.00	0.00	0.01	0.00	0.00	0.31	0.14	0.06	0.03	0.02	0.01	0.01	0.01	0.01	0.01	0.00	0.00	0.01	0.00	0.00	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	0	50	1	1	2	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	0	0	0	1	50.00	1	0	1.16	0	1.38	0	1.13	0	1.08	0	1.48	0	0.91	0.02	1.04	0	0.97	0.01	0.99	0	1.14	0	0.9	0.02	0.75	0.05	0.95	0.01	0.88	0.02
0	libggml-cpu.so - ggml-cpu.c:3228-3229 [...]	ggml_cpu_fp32_to_fp16	Single	0.23	0.22	0.25	0.30	0.34	0.43	0.22	0.24	0.26	0.28	0.30	0.31	0.33	0.34	0.34	0.23	0.20	0.19	0.19	0.24	0.17	0.14	0.11	0.16	0.14	0.14	0.12	0.17	0.12	0.14	0.23	0.20	0.19	0.19	0.24	0.17	0.14	0.11	0.16	0.14	0.14	0.12	0.17	0.12	0.14	0.29	0.14	0.09	0.05	0.04	0.04	0.02	0.02	0.02	0.02	0.02	0.01	0.02	0.02	0.02	0.29	0.14	0.09	0.05	0.04	0.04	0.02	0.02	0.02	0.02	0.02	0.01	0.02	0.02	0.02	0.29	0.13	0.06	0.03	0.02	0.01	0.01	0.00	0.01	0.01	0.00	0.00	0.00	0.00	0.00	0.29	0.13	0.06	0.03	0.02	0.01	0.01	0.00	0.01	0.01	0.00	0.00	0.00	0.00	0.00	1	2	4	8	16	20	24	25	35	34	34	36	46	37	42	92.5	98.75	1	1	1	1	1.1	1.39	1.8	1.78	2.8	1.67	2.14	1.94	2.08	2.04	2.4	2.12	2.36	2.21	2	0	0	0	0	100.00	1	0	1.13	0	1.17	0	1.15	0	0.87	0.03	1.06	0	1.21	0	1.45	0	0.91	0.01	1	0	0.97	0	1.05	0	0.73	0.05	1.02	0	0.81	0.03
413	libggml-cpu.so - binary-ops.cpp:10-32 [...]	ggml_compute_forward_add_non_quantized	Innermost	0.19	0.18	0.15	0.11	0.24	0.31	0.29	0.40	0.44	0.47	0.50	0.42	0.55	0.34	0.46	0.19	0.17	0.14	0.06	0.18	0.24	0.18	0.22	0.24	0.19	0.23	0.24	0.27	0.22	0.25	0.19	0.17	0.14	0.06	0.18	0.24	0.18	0.22	0.24	0.19	0.23	0.24	0.27	0.22	0.25	0.25	0.11	0.05	0.02	0.03	0.03	0.02	0.03	0.03	0.02	0.02	0.02	0.03	0.02	0.02	0.25	0.11	0.05	0.02	0.03	0.03	0.02	0.03	0.03	0.02	0.02	0.02	0.03	0.02	0.02	0.25	0.11	0.04	0.01	0.02	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.25	0.11	0.04	0.01	0.02	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	1	2	4	6	16	24	28	32	39	34	42	52	51	58	59	0	25	1	1	5.33	1	1.07	1.14	1.6	1.74	1.74	2	2.22	2.38	2.62	2.56	2.29	2.52	1.98	2.29	0	0	0	3	0	50.00	1	0	1.13	0	1.38	0	3.17	0	1	0	0.65	0.08	0.78	0.04	0.6	0.09	0.5	0.12	0.64	0.07	0.5	0.12	0.44	0.14	0.4	0.16	0.46	0.12	0.38	0.16
520	libggml-cpu.so - binary-ops.cpp:18-32 [...]	ggml_compute_forward_mul	Innermost	0.19	0.15	0.21	0.36	0.29	0.31	0.65	0.24	0.44	0.38	0.71	0.52	0.33	0.34	0.46	0.19	0.13	0.18	0.22	0.21	0.19	0.46	0.15	0.23	0.22	0.25	0.21	0.21	0.19	0.20	0.19	0.13	0.18	0.22	0.21	0.19	0.46	0.15	0.23	0.22	0.25	0.21	0.21	0.19	0.20	0.24	0.10	0.07	0.06	0.03	0.02	0.04	0.02	0.03	0.02	0.04	0.03	0.02	0.02	0.02	0.24	0.10	0.07	0.06	0.03	0.02	0.04	0.02	0.03	0.02	0.04	0.03	0.02	0.02	0.02	0.24	0.09	0.06	0.04	0.02	0.01	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.24	0.09	0.06	0.04	0.02	0.01	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	1	2	4	8	16	22	32	31	42	41	51	47	53	49	55	0	23.68	1	1	5.33	1	1.12	1.22	1.82	1.71	2	2.04	1.94	2.73	2.13	4.01	3.01	2.01	1.99	2.68	0	0	0	3	0	50.00	1	0	1.41	0	1.03	0	0.82	0.04	0.81	0.04	0.8	0.04	0.3	0.32	0.88	0.02	0.53	0.11	0.53	0.1	0.45	0.14	0.5	0.1	0.5	0.1	0.54	0.09	0.47	0.11
1403	libggml-cpu.so - ops.cpp:6446-6456 [...]	ggml_compute_forward_rope_f32(ggml_compute_params const, ggml_tensor, bool)	Innermost	0.10	0.13	0.16	0.22	0.19	0.25	0.22	0.24	0.26	0.28	0.30	0.31	0.44	0.23	0.34	0.10	0.12	0.11	0.08	0.09	0.07	0.08	0.11	0.08	0.07	0.12	0.08	0.10	0.08	0.08	0.10	0.12	0.11	0.08	0.09	0.07	0.08	0.11	0.08	0.07	0.12	0.08	0.10	0.08	0.08	0.13	0.09	0.05	0.04	0.02	0.02	0.02	0.02	0.02	0.02	0.01	0.02	0.02	0.01	0.01	0.13	0.09	0.05	0.04	0.02	0.02	0.02	0.02	0.02	0.02	0.01	0.02	0.02	0.01	0.01	0.13	0.08	0.04	0.01	0.01	0.00	0.00	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.13	0.08	0.04	0.01	0.01	0.00	0.00	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	1	2	4	8	14	13	20	28	18	18	29	26	33	28	27	35.29	39.71	1.2	1.33	2.49	1	1.1	1.52	3.05	2.43	2.74	2.5	2.33	2	2.35	2.12	2.52	3.47	1.7	2.45	2	0	0	1	0	83.33	1	0	0.8	0.02	0.85	0.02	1.16	0	1.02	0	1.21	0	0.93	0.01	0.61	0.04	0.78	0.02	0.92	0.01	0.51	0.06	0.66	0.03	0.54	0.05	0.63	0.03	0.6	0.03
2338	libggml-cpu.so - kai_rhs_pack_nxk_qsi4c32pscalef16_qsu4c32s16s0.c:115-118	kai_run_rhs_pack_nxk_qsi4c32pscalef16_qsu4c32s16s0	Innermost	0.08	0.22	0.42	0.61	1.12	2.21	1.73	2.26	3.79	2.74	2.62	2.93	1.96	2.14	2.28	0.08	0.11	0.11	0.09	0.09	0.12	0.08	0.09	0.13	0.08	0.07	0.07	0.05	0.05	0.05	0.08	0.11	0.11	0.09	0.09	0.12	0.08	0.09	0.13	0.08	0.07	0.07	0.05	0.05	0.05	0.10	0.14	0.14	0.11	0.11	0.18	0.12	0.14	0.21	0.14	0.13	0.14	0.09	0.09	0.10	0.10	0.14	0.14	0.11	0.11	0.18	0.12	0.14	0.21	0.14	0.13	0.14	0.09	0.09	0.10	0.11	0.07	0.04	0.01	0.01	0.01	0.00	0.00	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.11	0.07	0.04	0.01	0.01	0.01	0.00	0.00	0.01	0.00	0.00	0.00	0.00	0.00	0.00	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	0	31.25	1	1	5.33	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	0	0	0	1	1	25.00	1	0	0.72	0.03	0.74	0.03	0.93	0.01	0.86	0.01	0.54	0.06	0.78	0.02	0.66	0.03	0.41	0.07	0.61	0.03	0.67	0.02	0.61	0.03	0.95	0	0.91	0	0.84	0.01
2336	libggml-cpu.so - kai_rhs_pack_nxk_qsi4c32pscalef16_qsu4c32s16s0.c:107-154 [...]	kai_run_rhs_pack_nxk_qsi4c32pscalef16_qsu4c32s16s0	Outermost	0.04	0.08	0.19	0.36	0.34	0.92	0.79	0.89	1.06	0.47	1.21	1.05	0.76	0.90	1.26	0.04	0.04	0.05	0.05	0.03	0.05	0.04	0.03	0.04	0.01	0.03	0.03	0.02	0.02	0.03	0.38	0.37	0.34	0.35	0.33	0.31	0.32	0.28	0.32	0.25	0.24	0.25	0.25	0.21	0.22	0.05	0.05	0.06	0.07	0.03	0.08	0.05	0.06	0.06	0.03	0.06	0.05	0.04	0.04	0.05	0.47	0.47	0.43	0.45	0.42	0.45	0.48	0.46	0.55	0.44	0.42	0.47	0.47	0.41	0.44	0.06	0.03	0.02	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.48	0.24	0.11	0.06	0.03	0.02	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	0	45.83	1	1	2	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	NA	NA	NA	NA	NA	0.00	1	0	1	0	0.83	0.01	0.82	0.01	1.48	0	0.67	0.02	0.9	0	0.88	0	0.77	0.01	1.86	0	0.76	0.01	0.89	0	1.28	0	1.14	0	0.8	0.01
1408	libggml-cpu.so - ops.cpp:6220-6245 [...]	ggml_compute_forward_rope_f32(ggml_compute_params const, ggml_tensor, bool)	Innermost	0.04	0.05	0.10	0.25	0.54	0.49	0.72	0.89	0.97	0.85	1.01	1.05	1.42	1.35	1.60	0.04	0.05	0.08	0.18	0.32	0.38	0.64	0.88	0.79	0.88	0.97	1.09	1.47	1.36	1.36	0.04	0.05	0.08	0.18	0.32	0.38	0.64	0.88	0.79	0.88	0.97	1.09	1.47	1.36	1.36	0.05	0.03	0.04	0.04	0.05	0.04	0.05	0.05	0.05	0.05	0.05	0.05	0.07	0.06	0.07	0.05	0.03	0.04	0.04	0.05	0.04	0.05	0.05	0.05	0.05	0.05	0.05	0.07	0.06	0.07	0.04	0.03	0.03	0.03	0.03	0.03	0.03	0.04	0.03	0.03	0.03	0.03	0.04	0.04	0.04	0.04	0.03	0.03	0.03	0.03	0.03	0.03	0.04	0.03	0.03	0.03	0.03	0.04	0.04	0.04	1	2	4	8	16	24	32	40	48	56	64	72	80	88	96	0	24.52	1	1.64	4.36	1	1.17	1.33	1.57	2.1	1.73	1.62	1.54	1.96	1.64	1.87	1.76	1.86	1.97	2.43	2	0	0	1	0	83.33	1	0	0.75	0.01	0.42	0.05	0.19	0.15	0.1	0.29	0.07	0.36	0.04	0.62	0.03	0.86	0.03	0.77	0.02	0.86	0.02	0.95	0.02	1.07	0.01	1.45	0.01	1.34	0.01	1.34
2237	libggml-cpu.so - quants.c:2492-2660 [...]	ggml_vec_dot_q6_K_q8_K	Outermost	0.03	0.03	0.03	0.06	0.15	0.06	0.07	0.08	0.09	0.19	0.10	0.21	0.11	0.11	0.11	0.03	0.02	0.02	0.02	0.05	0.01	0.03	0.01	0.02	0.04	0.03	0.03	0.02	0.02	0.03	0.29	0.28	0.28	0.27	0.29	0.25	0.23	0.24	0.28	0.30	0.19	0.21	0.22	0.22	0.23	0.04	0.02	0.01	0.01	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.00	0.01	0.36	0.18	0.09	0.05	0.03	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.00	0.01	0.04	0.02	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.36	0.18	0.09	0.05	0.03	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	1	2	3	5	9	3	9	3	7	12	10	10	9	6	14	64.15	74.65	1.08	1	1.08	1	1.33	1.5	1.67	2.25	1	1	1	1	1.85	1	1.82	1	1	1	0	0	0	22	0	50.00	1	0	1.33	0	1.97	0	1.3	0	0.63	0.02	2.45	0	0.8	0.01	2.34	0	0.96	0	0.52	0.02	0.67	0.01	0.59	0.01	0.73	0.01	1.1	0	0.45	0.02
1252	libggml-cpu.so - vec.h:646-653	ggml_compute_forward_rms_norm	Innermost	0.03	0.08	0.15	0.11	0.15	0.18	0.22	0.16	0.18	0.19	0.20	0.21	0.22	0.23	0.23	0.03	0.08	0.08	0.05	0.06	0.04	0.06	0.05	0.03	0.05	0.06	0.04	0.03	0.02	0.03	0.03	0.08	0.08	0.05	0.06	0.04	0.06	0.05	0.03	0.05	0.06	0.04	0.03	0.02	0.03	0.04	0.05	0.05	0.02	0.02	0.01	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.04	0.05	0.05	0.02	0.02	0.01	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.04	0.05	0.03	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.04	0.05	0.03	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	1	2	4	6	10	9	14	13	9	13	18	14	10	8	11	100	100	1	1	1	1	1.16	1.9	1.71	2	2.45	2.33	1.63	1.64	1.63	1.8	1.87	1.82	1.78	1.83	0	0	0	2	0	50.00	1	0	0.37	0.05	0.33	0.06	0.49	0.03	0.44	0.03	0.58	0.02	0.35	0.04	0.38	0.03	0.54	0.02	0.37	0.03	0.29	0.04	0.38	0.02	0.52	0.01	0.64	0.01	0.46	0.02
1709	libggml-cpu.so - vec.h:474-662 [...]	ggml_compute_forward_flash_attn_ext	InBetween	0.02	0.01	0.03	0.06	0.05	0.12	0.07	0.08	0.09	0.09	0.20	0.10	0.11	0.11	0.11	0.02	0.00	0.02	0.03	0.02	0.02	0.02	0.01	0.02	0.02	0.04	0.01	0.02	0.01	0.02	0.02	0.02	0.04	0.07	0.04	0.04	0.05	0.02	0.05	0.04	0.05	0.05	0.04	0.02	0.04	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.00	0.01	0.03	0.02	0.02	0.02	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.02	0.00	0.01	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.03	0.01	0.01	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	1	1	3	6	4	3	6	3	6	8	13	5	8	4	10	12.24	50.78	16.23	1.01	1.35	1	1	1.2	1.5	1	1.2	1	1	1	1	1.86	1	1	1	1	NA	NA	NA	NA	NA	0.00	1	0	3.98	0	0.79	0	0.49	0.02	0.94	0	0.73	0	0.6	0.01	1.17	0	0.56	0.01	0.42	0.01	0.24	0.03	0.65	0	0.41	0.01	0.83	0	0.32	0.02
2858	libllama.so - new_allocator.h:172-172 [...]	std::_Hashtable<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::pair<std::pai...	Single	0.01	0.02	0.04	0.08	0.15	0.25	0.07	0.16	0.18	0.19	0.20	0.21	0.11	0.11	0.23	0.01	0.01	0.01	0.01	0.01	0.01	0.00	0.01	0.01	0.01	0.01	0.01	0.00	0.00	0.00	0.01	0.01	0.01	0.01	0.01	0.01	0.00	0.01	0.01	0.01	0.01	0.01	0.00	0.00	0.00	0.01	0.01	0.01	0.02	0.01	0.02	0.00	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.02	0.01	0.02	0.00	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	0	45.44	1	1	2.71	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	0	0	0	2	0	50.00	1	0	1	0	0.66	0	0.65	0	0.63	0	0.46	0.01	1.79	0	0.88	0	0.84	0	0.84	0	0.83	0	0.81	0	1.63	0	1.65	0	0.8	0
1719	libggml-cpu.so - vec.h:710-717	ggml_compute_forward_flash_attn_ext	Innermost	0.01	0.02	0.01	0.03	0.00	0.06	0.07	0.08	0.09	0.00	0.10	0.10	0.11	0.23	0.11	0.01	0.01	0.00	0.01	0.00	0.00	0.01	0.00	0.01	0.00	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.00	0.01	0.00	0.00	0.01	0.00	0.01	0.00	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.00	0.00	0.00	0.00	0.01	0.00	0.01	0.00	0.01	0.00	0.00	0.01	0.00	0.01	0.01	0.00	0.00	0.00	0.00	0.01	0.00	0.01	0.00	0.01	0.00	0.00	0.01	0.00	0.01	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	1	2	1	2	0	1	3	1	3	0	2	2	2	3	4	96	98	1	1	1	1	1.33	1	1	0	1	1	1	1	0	1	1	1	1.5	1	0	0	0	1	0	50.00	1	0	0.66	0	1.97	0	0.97	0			1.84	0	0.6	0	1.75	0	0.56	0			0.83	0	0.81	0	0.82	0	0.41	0.01	0.4	0.01
1722	libggml-cpu.so - vec.h:646-653	ggml_compute_forward_flash_attn_ext	Innermost	0.01	0.02	0.03	0.08	0.10	0.06	0.07	0.00	0.09	0.09	0.10	0.10	0.11	0.11	0.11	0.01	0.01	0.01	0.02	0.02	0.01	0.02	0.00	0.02	0.01	0.01	0.01	0.02	0.01	0.01	0.01	0.01	0.01	0.02	0.02	0.01	0.02	0.00	0.02	0.01	0.01	0.01	0.02	0.01	0.01	0.01	0.01	0.01	0.02	0.01	0.00	0.01	0.00	0.01	0.00	0.00	0.01	0.01	0.01	0.00	0.01	0.01	0.01	0.02	0.01	0.00	0.01	0.00	0.01	0.00	0.00	0.01	0.01	0.01	0.00	0.01	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	1	1	2	3	3	4	5	0	6	4	2	4	6	4	4	100	100	1	1	1	1	1	1.33	1.8	1.5	1	1	0	1	1	1	1	1	1	1	0	0	0	2	0	50.00	1	0	1	0	0.66	0	0.39	0.01	0.47	0.01	0.46	0.01	0.36	0.01			0.28	0.01	0.42	0.01	0.83	0	0.41	0.01	0.27	0.01	0.41	0.01	0.4	0.01
305	libggml-base.so - ggml-quants.c:314-322 [...]	dequantize_row_q4_0	Single	0.00	0.01	0.00	0.03	0.00	0.06	0.00	0.08	0.00	0.00	0.10	0.00	0.00	0.00	0.11	0.00	0.01	0.00	0.00	0.00	0.01	0.00	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.00	0.00	0.00	0.01	0.00	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.00	0.00	0.00	0.01	0.00	0.01	0.00	0.00	0.01	0.00	0.00	0.00	0.01	0.00	0.01	0.00	0.00	0.00	0.01	0.00	0.01	0.00	0.00	0.01	0.00	0.00	0.00	0.01	0.00	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0	2	0	1	0	4	0	4	0	0	1	0	0	0	2	45.68	49.69	1.1	1.03	1.55	0	1	0	1	0	1	0	1	0	0	1	0	0	0	1	0	0	0	0	0	0.00			1	0			1	0			1	0			1	0					1	0							1	0
1400	libggml-cpu.so - ops.cpp:6210-6484 [...]	ggml_compute_forward_rope_f32(ggml_compute_params const, ggml_tensor, bool)	Outermost	0.00	0.02	0.01	0.06	0.05	0.06	0.14	0.16	0.09	0.09	0.20	0.21	0.22	0.34	0.23	0.00	0.01	0.00	0.01	0.02	0.02	0.03	0.04	0.00	0.01	0.03	0.04	0.04	0.05	0.04	0.00	0.18	0.22	0.27	0.47	0.48	0.79	1.07	0.94	1.01	1.19	1.32	1.74	1.58	1.57	0.00	0.01	0.00	0.01	0.01	0.01	0.01	0.01	0.00	0.00	0.01	0.01	0.01	0.01	0.01	0.00	0.13	0.07	0.06	0.06	0.04	0.05	0.06	0.05	0.05	0.05	0.05	0.07	0.06	0.07	0.00	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.12	0.07	0.05	0.04	0.03	0.04	0.05	0.04	0.04	0.04	0.04	0.05	0.04	0.04	0	2	1	1	4	5	7	12	1	4	8	15	14	15	15	7.69	42.15	5.05	1.52	3.73	0	1.33	1	1	1	1	1.75	1.85	1	1	1.78	1.87	1.87	2.5	1.87	NA	NA	NA	NA	NA	0.00			1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0
1399	libggml-cpu.so - ops.cpp:6210-6409 [...]	ggml_compute_forward_rope_f32(ggml_compute_params const, ggml_tensor, bool)	InBetween	0.00	0.00	0.04	0.03	0.19	0.06	0.14	0.24	0.18	0.28	0.20	0.21	0.22	0.34	0.46	0.00	0.00	0.02	0.00	0.05	0.02	0.04	0.04	0.07	0.05	0.08	0.10	0.13	0.09	0.10	0.00	0.00	0.10	0.18	0.37	0.40	0.68	0.92	0.86	0.93	1.05	1.20	1.60	1.45	1.45	0.00	0.00	0.02	0.00	0.02	0.01	0.01	0.01	0.01	0.02	0.01	0.01	0.01	0.01	0.02	0.00	0.00	0.05	0.04	0.06	0.04	0.05	0.06	0.04	0.05	0.05	0.06	0.07	0.05	0.06	0.00	0.00	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.03	0.03	0.03	0.03	0.04	0.04	0.04	0.03	0.03	0.04	0.05	0.04	0.04	0	0	2	1	9	5	10	10	22	14	24	32	41	28	29	0	41.88	5.47	1.38	2.88	0	0	1.5	1	2.77	1	1.67	2.31	1.91	2.21	1.71	1.64	1.64	2.33	2.9	NA	NA	NA	NA	NA	0.00					1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0
1724	libggml-cpu.so - ops.cpp:8885-8886 [...]	ggml_compute_forward_flash_attn_ext	Innermost	0.00	0.01	0.01	0.06	0.05	0.06	0.07	0.08	0.18	0.09	0.10	0.10	0.22	0.00	0.11	0.00	0.01	0.00	0.02	0.01	0.01	0.01	0.01	0.02	0.00	0.00	0.02	0.01	0.00	0.00	0.00	0.01	0.00	0.02	0.01	0.01	0.01	0.01	0.02	0.00	0.00	0.02	0.01	0.00	0.00	0.00	0.01	0.00	0.01	0.01	0.01	0.01	0.01	0.01	0.00	0.00	0.01	0.01	0.00	0.01	0.00	0.01	0.00	0.01	0.01	0.01	0.01	0.01	0.01	0.00	0.00	0.01	0.01	0.00	0.01	0.00	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0	2	1	3	2	3	4	2	5	1	1	9	2	0	2	0	50	1	1	1.6	0	1	1	1.2	1	1	1	1	1.67	1	1	1	1.33	0	1	1	0	1	0	0	87.50			1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0			1	0
365	libggml-cpu.so - traits.cpp:13-17 [...]	ggml_cpu_extra_compute_forward	Single	0.00	0.00	0.00	0.03	0.00	0.06	0.00	0.08	0.09	0.09	0.10	0.10	0.11	0.23	0.11	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.01	0.01	0.02	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.01	0.01	0.02	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.00	0.00	0.00	0.00	0.01	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.00	0.00	0.00	0.00	0.01	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0	0	0	1	0	1	0	1	1	1	3	4	2	7	2	0	50	1	1	2	0	0	0	1	0	1	0	1	1	1	1	1	1	1.75	1	0	0	1	2	0	62.19							1	0			1	0			1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0
73	libggml-cpu.so - ggml-cpu.c:533-2897 [...]	ggml_graph_compute_thread	Outermost	0.00	0.00	0.01	0.00	0.00	0.06	0.00	0.08	0.09	0.09	0.20	0.10	0.11	0.23	0.23	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.01	0.01	0.03	0.02	0.02	0.06	0.06	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.01	0.01	0.03	0.02	0.02	0.06	0.06	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.01	0.00	0.01	0.01	0.01	0.01	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.01	0.00	0.01	0.01	0.01	0.01	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0	0	1	0	0	1	0	3	2	2	8	8	7	23	21	0	38.84	1	1	5.33	0	0	1	0	0	1	0	1	1	1	1.6	1	1	1.92	1.75	NA	NA	NA	NA	NA	0.00					1	0					1	0			1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0

Report Configuration

Loops Index

▶Scalability Runs Description

▶Filters

▼Columns Filter