Stockfish FRC Correction from Classical to NNUE Testing Tournament.

Hardware Threadripper 2950x, RTX 2080 TI, SSD 2 TB Evo 970 Plus.

Time Control 3m + 2s.
50 Rounds
4 Core
Book Fischer Random.
1024 Mb Hash
7 man tablebases.

Add a specific FRC correction from classical to NNUE

our net currently is not trained on FRC games, and so doesn't know about the important pattern of a bishop that is cornered in FRC.
This patch introduces a term we have in the classical evaluation for this case, and adds it to the NNUE eval.

The new Stockfish FRC patch is just crushing the field. Against the best FRC engines currently out. A successful patch looks to be an understatement.

Yes, I'm running it as chess960 also, stunning performance so far. The +40 Elo is probably right.

Apologies to hijack your thread, but my testing has completed and I have the same assessment !

http://ccrl.chessdom.com/ccrl/404FRC/

I took a very quick look, there only appears to be a change made to the classical evaluation, the NNUE looks untouched. Ok it’s checks if B on corner and applies a rough penalty depending on blocking pawn and if the blocking pawn is blocked..

// An important Chess960 pattern: a cornered bishop blocked by a friendly
// pawn diagonally in front of it is a very serious problem, especially
// when that pawn is also blocked.
if ( pos.is_chess960()
&& (s == relative_square(Us, SQ_A1) || s == relative_square(Us, SQ_H1)))
{
Direction d = pawn_push(Us) + (file_of(s) == FILE_A ? EAST : WEST);
if (pos.piece_on(s + d) == make_piece(Us, PAWN))
score -= !pos.empty(s + d + pawn_push(Us)) ? CorneredBishop * 4
: pos.piece_on(s + d + d) == make_piece(Us, PAWN) ? CorneredBishop * 2
: CorneredBishop;
}

Ray wrote:

Apologies to hijack your thread, but my testing has completed and I have the same assessment !

http://ccrl.chessdom.com/ccrl/404FRC/

You are always welcome Ray. With more cores and time. The Patch Hack is down to +19 Elo +/- 27. But still very nice!

Code:

Rank Name                          Elo     +/-   Games    Wins  Losses   Draws   Points   Score    Draw
   1 Stockfish FRC correction       87      27     131      32       0      99     81.5   62.2%   75.6%
   2 Stockfish 13                   68      33     130      33       8      89     77.5   59.6%   68.5%
   3 Lc0 0.27.0                     38      35     130      30      16      84     72.0   55.4%   64.6%
   4 Stockfish 13 BN avx2           21      31     130      22      14      94     69.0   53.1%   72.3%
   5 Dragon 64bit avx2              -3      33     131      19      20      92     65.0   49.6%   70.2%
   6 Ethereal 12.75 x64 avx2      -241      52     130       4      82      44     26.0   20.0%   33.8%

391 of 750 games finished.

Chris Whittington wrote:

I took a very quick look, there only appears to be a change made to the classical evaluation, the NNUE looks untouched. Ok it’s checks if B on corner and applies a rough penalty depending on blocking pawn and if the blocking pawn is blocked..

               // An important Chess960 pattern: a cornered bishop blocked by a friendly
               // pawn diagonally in front of it is a very serious problem, especially
               // when that pawn is also blocked.
               if (   pos.is_chess960()
                   && (s == relative_square(Us, SQ_A1) || s == relative_square(Us, SQ_H1)))
               {
                   Direction d = pawn_push(Us) + (file_of(s) == FILE_A ? EAST : WEST);
                   if (pos.piece_on(s + d) == make_piece(Us, PAWN))
                       score -= !pos.empty(s + d + pawn_push(Us))                ? CorneredBishop * 4
                               : pos.piece_on(s + d + d) == make_piece(Us, PAWN) ? CorneredBishop * 2
                                                                                 : CorneredBishop;
               }

Yes this changes is only of very limited help with FRC. And it seems with more time, or cores in testing. Even the current NN are able to correct this issue with more search.

We need a FRC trained net. But I guess this is still very hard to do. As you must train the net in all of the 960 starting positions. As each opening position would have its own unique opening development issues. That the NN would need to understand.

Current results +13 Elo with more time and cores. Way under the +40 Elo with 1 core testing.

Stockfish. Why ?! Stockfish or crafty or fritz. Same procedure as every year.

End of test. +7 Elo

Code:

Rank Name                          Elo    +/-  Games    Wins  Losses  Draws  Points  Score    Draw
  1 Stockfish FRC correction      80      25    160      37      1    122    98.0  61.3%  76.3%
  2 Stockfish 13                  73      29    160      41      8    111    96.5  60.3%  69.4%
  3 Lc0 0.27.0                    33      31    160      34      19    107    87.5  54.7%  66.9%
  4 Stockfish 13 BN avx2          28      28    160      29      16    115    86.5  54.1%  71.9%
  5 Dragon 64bit avx2              -4      31    160      25      27    108    79.0  49.4%  67.5%
  6 Ethereal 12.75 x64 avx2      -237      45    160      4      99      57    32.5  20.3%  35.6%

480 games finished.

mwyoung wrote:

Chris Whittington wrote:

I took a very quick look, there only appears to be a change made to the classical evaluation, the NNUE looks untouched. Ok it’s checks if B on corner and applies a rough penalty depending on blocking pawn and if the blocking pawn is blocked..

               // An important Chess960 pattern: a cornered bishop blocked by a friendly
               // pawn diagonally in front of it is a very serious problem, especially
               // when that pawn is also blocked.
               if (   pos.is_chess960()
                   && (s == relative_square(Us, SQ_A1) || s == relative_square(Us, SQ_H1)))
               {
                   Direction d = pawn_push(Us) + (file_of(s) == FILE_A ? EAST : WEST);
                   if (pos.piece_on(s + d) == make_piece(Us, PAWN))
                       score -= !pos.empty(s + d + pawn_push(Us))                ? CorneredBishop * 4
                               : pos.piece_on(s + d + d) == make_piece(Us, PAWN) ? CorneredBishop * 2
                                                                                 : CorneredBishop;
               }

Yes this changes is only of very limited help with FRC. And it seems with more time, or cores in testing. Even the current NN are able to correct this issue with more search.

We need a FRC trained net. But I guess this is still very hard to do. As you must train the net in all of the 960 starting positions. As each opening position would have its own unique opening development issues. That the NN would need to understand.

Current results +13 Elo with more time and cores. Way under the +40 Elo with 1 core testing.

It ought not be too problematical to train for FRC, just have to more or less start again. A bore, but doable.