Stockfish 26/09/21 Compile Test. How much does it matter?

Stockfish 26/09/21 Compile Test. How much does it matter?

Hardware Threadripper 2950x, RTX 2080TI 64 Gb Ram, SSD 2 TB Evo 970 Plus.

Book Perfect Book 2021
7 man Table bases
TC = 3m+2s
1 Threads
1 GB Hash

What do you mean with - Compile Test. How much does it matter?

Admin wrote:

What do you mean with - Compile Test. How much does it matter?

I am running all the different versions of Stockfish. And putting a Elo number to them. In theory AVX2 should be the best on my CPU,  as it is the fastest. And some tout SF NNUE must be run in  AVX to get good results.

Results so far...

Rank Name                          Elo     +/-   Games    Wins  Losses   Draws   Points   Score    Draw
  1 Stockfish 26/09/21 Modern      72      11     725     151       3     571    436.5   60.2%   78.8%
  2 Stockfish 26/09/21 BMI2        71      11     726     150       4     572    436.0   60.1%   78.8%
  3 Stockfish 26/09/21 AVX2        68      11     727     145       4     578    434.0   59.7%   79.5%
  4 Stockfish 26/09/21 SSE         67      11     724     144       7     573    430.5   59.5%   79.1%
  5 Stockfish 26/09/21 64 Bit      66      11     725     144       7     574    431.0   59.4%   79.2%
  6 Stockfish 26/09/21 32 Bit    -743      81     729       0     709      20     10.0    1.4%    2.7%

2178 of 15000 games finished.

I see...

How much elo are these versions better than SF14, any estimate?

Results show most of the 64 bit versions of Stockfish perform about the same. Except for the slowest 64 bit compile. AVX2 seems to be way over hyped.

Rank Name                          Elo     +/-   Games    Wins  Losses   Draws   Points   Score    Draw
  1 Stockfish 26/09/21 Modern      71       8    1290     270       9    1011    775.5   60.1%   78.4%
  2 Stockfish 26/09/21 SSE         69       8    1290     265      12    1013    771.5   59.8%   78.5%
  3 Stockfish 26/09/21 AVX2        69       8    1290     261       9    1020    771.0   59.8%   79.1%
  4 Stockfish 26/09/21 BMI2        68       8    1290     264      13    1013    770.5   59.7%   78.5%
  5 Stockfish 26/09/21 64 Bit      65       8    1289     257      20    1012    763.0   59.2%   78.5%
  6 Stockfish 26/09/21 32 Bit    -726      56    1293       0    1254      39     19.5    1.5%    3.0%

mwyoung wrote:

Results show most of the 64 bit versions of Stockfish perform about the same. Except for the slowest 64 bit compile. AVX2 seems to be way over hyped.

Rank Name                          Elo     +/-   Games    Wins  Losses   Draws   Points   Score    Draw
  1 Stockfish 26/09/21 Modern      71       8    1290     270       9    1011    775.5   60.1%   78.4%
  2 Stockfish 26/09/21 SSE         69       8    1290     265      12    1013    771.5   59.8%   78.5%
  3 Stockfish 26/09/21 AVX2        69       8    1290     261       9    1020    771.0   59.8%   79.1%
  4 Stockfish 26/09/21 BMI2        68       8    1290     264      13    1013    770.5   59.7%   78.5%
  5 Stockfish 26/09/21 64 Bit      65       8    1289     257      20    1012    763.0   59.2%   78.5%
  6 Stockfish 26/09/21 32 Bit    -726      56    1293       0    1254      39     19.5    1.5%    3.0%

I believe that with a balanced book speed improvement of 100% does not give much elo for stockfish so I am not surprised

I guess that if you play stockfish at 6+4 against 3+2 single core you will get something like 10 elo improvement and maybe less than it.

More interesting may be 60+40 against 3+2 single core and even here my guess is that most of the games are going to be draws.

The only surprising result is the bad result of 32 bit and it seems that it has some bug with the 32 bit version.

I agree with Uri, draw rate is 78%.

Use some anti-draw opening set of openings, see for instance https://www.sp-cc.de/anti-draw-openings.htm

Or the gambit positions, look at - http://rebel13.nl/grl-40-2.html - SF14 draw perc only 37%.

Admin wrote:

I agree with Uri, draw rate is 78%.

Use some anti-draw opening set of openings, see for instance https://www.sp-cc.de/anti-draw-openings.htm

Or the gambit positions, look at - http://rebel13.nl/grl-40-2.html - SF14 draw perc only 37%.

Why?

Try !

I have, and when you setup a corrupted test, you only get garbage results.

Draw rate should not be defined by your desire. But as a result of the strength differences between the chess engines being tested, and the true nature of chess.

That is like people thinking in my current match with Zero books (chess 960) . The draw rate will be lower.

That is not the nature of chess, or why draws are made.

Chess 960 match

Score of Dragon 2.5  vs Stockfish 26/09/21: 0 - 0 - 14 [0.500]
...      Dragon 2.5  playing White: 0 - 0 - 7  [0.500] 7
...      Dragon 2.5  playing Black: 0 - 0 - 7  [0.500] 7
...      White vs Black: 0 - 0 - 14  [0.500] 14
Elo difference: 0.0 +/- 0.0, LOS: nan %, DrawRatio: 100.0 %
14 of 1000 games finished.

You said - AVX2 seems to be way over hyped.

You run it from book, how many doubles does that produce?

It's not the way to test compiles.

Most simple way is to run each compile from the command line.

uci
go depth 30

And post the time and NPS for each compile.

ok

You know programmers, they start hyperventilating when a compile produces 10% faster code, me included

I know, but we should not tell people and some have. That AVX is a must have to run NNUE to get good results. That is simply not true.

It's my impression also.

Admin wrote:

I agree with Uri, draw rate is 78%.

Use some anti-draw opening set of openings, see for instance https://www.sp-cc.de/anti-draw-openings.htm

Or the gambit positions, look at - http://rebel13.nl/grl-40-2.html - SF14 draw perc only 37%.

It is 78% only because of buggy stockfish 32 bits

If you ignore stockfish 32 bit it is close to 98% draws.

I guess that even with time handicap of 10:1 you will get majority of draws.

I tested Stockfish 5M nodes per move against stockfish 60+0.6 time control in 40 games with opening book of all the first possible legal moves
and got 37 draws out of 40 and 3 black wins for g4 f3 Na3(white did not win a single game)

total result was 20.5-19.5 for the 60+0.6 time control

Later I tested 10M nodes per move against 60+0.6 with the same conditions and 10M nodes per move won 22-18 but there was only a single win for white for the 10M nodes per move with 1.Nc3.
10M nodes per move won also 3 games with black against 1.g4 1.f3 1.Na3 and the rest of the games were draws.

I now test 5M nodes per move as white against 50M nodes per move as black with stockfish developement version.
We can expect 50M nodes to win but I am afraid I will see again draws in most games inspite of the fact that I think black has the advantage.

So far I have draws for the moves 1.a3 1.a4 1.b3

I stopped my match after 15 games.

only 1.f4 and 1.g4 lost for white when white used 5M nodes per move against 50M nodes per move
2048 mbytes with no tablebases for both sides.

games were from 1.a3 1.a4 1.b3 1.b4,...1.g3 1.g4 1.h3

disappointing result is 13 draws and only 2 wins for black when I guess that one of them after 1.g4 is a forced win for black.



I thought maybe something in the setting is wrong and maybe the 5M nodes learned something from the search of the 50M nodes
but I could reproduce some moves with the exact evaluations in arena gui