ProDeo
Would you like to react to this message? Create an account in a few clicks or log in to continue.
ProDeo

Computer Chess
 
HomeHome  CalendarCalendar  Latest imagesLatest images  FAQFAQ  SearchSearch  MemberlistMemberlist  UsergroupsUsergroups  RegisterRegister  Log in  

 

 Totally Meaningless Stockfish Benchmarks. And Being Drunk on NPS, and Ignoring Reality.

Go down 
AuthorMessage
mwyoung

mwyoung


Posts : 880
Join date : 2020-11-25
Location : USA

Totally Meaningless Stockfish Benchmarks. And Being Drunk on NPS, and Ignoring Reality. Empty
PostSubject: Totally Meaningless Stockfish Benchmarks. And Being Drunk on NPS, and Ignoring Reality.   Totally Meaningless Stockfish Benchmarks. And Being Drunk on NPS, and Ignoring Reality. EmptyMon Nov 28, 2022 4:48 am

People insist on believing nodes per second as a trust worthy benchmark for Stockfish, and other chess engines. But nothing can be further from the truth.

I will show that nodes per second is a meaningless stat in judging hardware speed for Stockfish. By showing my $1500 Laptop is far superior in running Stockfish then my Threadripper.


The test was run with the Stockfish bench mark command for Stockfish 15.

bench 1024 32 26 default depth nnue - Threadripper
bench 1024 8 26 default depth nnue - $1500 Laptop.


Theadripper Results.
NPS = 21,956,331

$1500 Laptop Results.
NPS = 13,699,948

The Threadripper is 1.6x faster in NPS. If your conclusion is the Threadripper is the faster computer running Stockfish. You would be dead wrong.

Solve time to complete the benchmark.

$1500 Laptop Total Solve Time Results.
Total Time (ms) 110,731

Threadripper Total Solve Time Results.
Total Time (ms) 199,936

The $1500 Laptop is 1.8x faster in completing the benchmark test.

How is this possible?

The answer is in how wildly inefficient Stockfish becomes as you add more and more threads.

Total Nodes searched for Threadripper.
Nodes Search 4,389,861,054

Total Nodes searched for $1500 Laptop.
Nodes Searched 1,517,009,043

The $1500 Laptop had to search 2.9x less nodes to complete the benchmark, and the Laptop cores are faster.

Totally Meaningless Stockfish Benchmarks. And Being Drunk on NPS, and Ignoring Reality. Bench11
Threadripper results.
Totally Meaningless Stockfish Benchmarks. And Being Drunk on NPS, and Ignoring Reality. Bench_11
$1500 Laptop results.
Totally Meaningless Stockfish Benchmarks. And Being Drunk on NPS, and Ignoring Reality. Bench_12
Back to top Go down
 
Totally Meaningless Stockfish Benchmarks. And Being Drunk on NPS, and Ignoring Reality.
Back to top 
Page 1 of 1
 Similar topics
-
» Drunk Magnus!
» Chess And Reality: The Great Reversal
» Testing the New Stockfish Net of 45 Mb, Stockfish 190521 vs Stockfish 13 (TC = 5m+5s) (32 Threads)
» Testing the New Stockfish Net of 45 Mb vs 20 Mb, Stockfish 190521 vs Stockfish 13 (TC = 1m+1s)
» Stockfish 110221 Vs. Stockfish 12, Ratings Gain Test, 5000 Games.

Permissions in this forum:You cannot reply to topics in this forum
ProDeo :: Computer Chess-
Jump to: