Testing the new Lc0

Lc0-v28 - SF14 for the GRL

200 games.

Live support via - http://rebel13.nl/pgn4web-3.05/live-test.html

Code:

Name               Gms  :  Win  : Draw  : Lose  :  Pts  
Stockfish 14       200  :  63+  : 121=  :  16-  : 123.5  
Lc0-v28            200  :  16+  : 121=  :  63-  :  76.5  

History

Code:

Lc0-v27 - Stockfish 13  :  32%
Lc0-v28 - Stockfish 14  :  38.5%

Next Lc0-v28 - Komodo Dragon 2

Code:

 7)  Lc0-v27             3532.5 :    800 (+307,=388,-105),  62.6 %
     vs.                        :  games (   +,   =,   -),   (%) :    Diff,    SD, CFS (%)
     Stockfish_13               :    100 (   3,  58,  39),  32.0 :  -139.6,  10.1,    0.0
     Komodo-Dragon              :    100 (  13,  67,  20),  46.5 :   -53.0,  11.4,    0.0
     Komodo_14                  :    100 (  43,  46,  11),  66.0 :  +108.1,  13.1,  100.0
     RubiChess_2.1              :    100 (  54,  40,   6),  74.0 :  +149.8,   9.1,  100.0
     SlowChess_2.5              :    100 (  51,  38,  11),  70.0 :  +164.4,   8.4,  100.0
     Igel_3.0.5                 :    100 (  42,  50,   8),  67.0 :  +171.0,  11.9,  100.0
     Ethereal_12.75             :    100 (  52,  42,   6),  73.0 :  +174.4,  11.8,  100.0
     Igel_3.0.0                 :    100 (  49,  47,   4),  72.5 :  +184.5,  12.0,  100.0

Looking at the (above) results of Lc0-v27 the new Lc0 looks a lot stronger, it scored 6% better against Stockfish and after 94 games it is in the clear lead against Komodo Dragon 2 while version 27 lost.

Code:

     Name             Gms  :  Win  : Draw  : Lose  :  Pts
Lc0-v28                94  :  26+  :  52=  :  16-  :  52.
Komodo-Dragon_2        94  :  16+  :  52=  :  26-  :  42.
Games played 94, games to go 106

http://rebel13.nl/pgn4web-3.05/live-test.html

Code:

Lc0-v28               200  :  50+  : 104=  :  46-  : 102.
Komodo-Dragon 2       200  :  46+  : 104=  :  50-  :  98.

Next run - Lc0-v28 - Ethereal 13.25 NNUE.

Previous run - Lc0-v27 - Ethereal 12.75 : 73%

Hello Ed, thanks you for your test with LC0. For your tests on CPU, I would be interested to know which networks were used here (703810 for v.0.27 and 752187 for v.0.28 ?). I have been testing Lc0 on CPU myself for years. Did you use the DNNL version (which is much faster) or the blas version for these
tests ?

For v27 and 28 I used the net that came with the downloads.

v27 - 703810
v28 - 744204

Using indeed the DNNL version.

Welcome BTW.

Thanks for your answer, of course 744204 as this is the new standard network for blas/dnnl. I am curious about your tests on GPU, so far there have been no tests of large networks on GTX 1650.

The current gauntlets run on a GTX 1060 6Gb, network 609992.pb (130 Mb).

Thanks Ed, very interesting, your results on CPU corresponds very well to my own
results, is it possible to post a png file here ?

[pgn][Event "Lc0-v28-et1325"]
[Site "EDDY-PC"]
[Date "2021.08.28"]
[Round "21"]
[White "Lc0-v28"]
[Black "Ethereal_13.25-NNUE"]
[Result "1-0"]
[BlackElo "2200"]
[ECO "C37"]
[Opening "KGA"]
[Time "22:01:25"]
[Variation "Sörensen Gambit"]
[WhiteElo "2200"]
[TimeControl "40/120:40/120:40/120"]
[Termination "adjudication"]
[PlyCount "42"]
[WhiteType "program"]
[BlackType "program"]

1. e4 e5 2. f4 exf4 3. Nf3 g5 {+0.40/20 6} 4. d4 {-0.24/19 3} g4 {+0.88/21
3} 5. Ne5 {-0.26/23 2} Qh4+ {+1.92/22 6} 6. g3 {-0.26/20 2} fxg3 {+1.79/25
7} 7. Qxg4 {-0.29/20 3} g2+ {+0.88/23 4} 8. Qxh4 {0.00/19 6} gxh1=Q
{0.00/24 7} 9. Nc3 {0.00/20 3} Bb4 {+0.20/21 2} 10. Kf2 {+0.40/10 6} h5
{-0.71/23 12} 11. Nd5 {+0.99/8 2} Be7 {-1.25/23 10} 12. Bg5 {+0.81/10 3} f6
{-1.86/23 9} 13. Qf4 {+1.05/14 0} Na6 {-0.73/24 5} 14. Qf5 {+3.90/8 2}
Qxh2+ {-8.99/20 5} 15. Bg2 {+5.71/10 1} fxe5 {-10.12/22 5} 16. Qg6+
{+6.43/12 4} Kd8 {-11.63/22 4} 17. Qg7 {+9.02/12 5} d6 {-11.98/22 3} 18.
Rh1 {+12.15/12 8} Qxg2+ {-8.97/22 4} 19. Kxg2 {+11.17/12 2} Bxg5 {-9.44/21
1} 20. Qxg5+ {+13.29/10 2} Kd7 {-9.95/22 2} 21. Qg7+ {+18.36/1 0} Kc6
{-9.73/22 3  Arena Adjudication} 1-0[/pgn]

Nice firework.

I am testing lc0 on a 2070 vs. Stockfish and other running 8 cores and permanent brain

Mclane wrote:

I am testing lc0 on a 2070 vs. Stockfish and other running 8 cores and permanent brain

Please post results.

BTW, I am toying with the idea to open a second list, top-10 engines that compete on 16 or 20 cores. It has never been done before but I can't do it alone.

Are you and Mark Young willing to participate?

Made a start, current top-10 GRL, 20 cores, hash 4Gb, games 9000 [!], each playing each the 200 gambit positions.

http://rebel13.nl/b/grl.htm

For the moment at bullet 40/10.

Somehow Lc0-v28 should move in but at a good GPU.

I have a 16 core machine.
because i am doing permanent brain, i chose 8 cores for each engine plus LC0 on 2070.
of course another test could be done with more cores but only if permanent brain is off then.

i am testing with arena so far. the advantage is that arena is doing all alone and i can watch the whole
tournament like a cinema-movie before my eyes.

it automatically ranks the engines. it automatically changes opponents. it automatically arranges the whole thing.

a new participant (e.g. a new version of LC0 in arena) means it only has to play the 300-400 games and ranks automatically in the big data pool. so the tournament is an elo list on its own.

my own 1min + 1sec "rating-list" on CPU (2 threads each) with Lc0 703810 and 74706

the Elo difference between 703810 and 744706 fits perfectly with your own tests on CPU.

Code:

Name                      Gms  :  Win  : Draw  : Lose  :
Lc0-v28                   200  :  86+  :  94=  :  20-  :
Ethereal_13.25-NNUE       200  :  20+  :  94=  :  86-  :

It wouldn't surprise me if Lc0 will take over the second place of the Dragon.

Next : Lc0-v28 vs Slowchess 2.7

Same link - http://rebel13.nl/pgn4web-3.05/live-test.html

strong result against Ethereal about +105 Elo. Thanks for your tests..

Admin wrote:

Code:

Name                      Gms  :  Win  : Draw  : Lose  :
Lc0-v28                   200  :  86+  :  94=  :  20-  :
Ethereal_13.25-NNUE       200  :  20+  :  94=  :  86-  :

It wouldn't surprise me if Lc0 will take over the second place of the Dragon.

Next : Lc0-v28 vs Slowchess 2.7

Same link - http://rebel13.nl/pgn4web-3.05/live-test.html

You could change you mind after testing Fat Titz. It could be Stockfish that is #2.

Code:

Rank Name                          Elo     +/-   Games    Wins  Losses   Draws   Points   Score    Draw
   1 Fat Titz 1.0                   17      20      84       5       1      78     44.0   52.4%   92.9%
   2 Stockfish 08/27/21             13      18      83       4       1      78     43.0   51.8%   94.0%
   3 Stockfish 14                   13      22      83       5       2      76     43.0   51.8%   91.6%
   4 Lc0 0.28.0                    -42      27      84       1      11      72     37.0   44.0%   85.7%

167 of 600 games finished.

I don't include derivatives. Besides of that I do understand the irony but nevertheless I find the name offensive to women.

Admin wrote:

I don't include derivatives. Besides of that I do understand the irony but nevertheless I find the name offensive to women.

Fat Titz name has nothing to do with woman. Know your German.

Example.

Titz is a municipality in the district of Düren in the state of North Rhine-Westphalia,


Fat Titz is a legal fork.

"I don't include derivatives." But you do!

Stockfish (chess)

The program originated from Glaurung, an open-source chess engine created by Romstad and first released in 2004. Four years later, Costalba, inspired by the strong open-source engine, decided to fork the project. He named it Stockfish because it was "produced in Norway and cooked in Italy" (Romstad is Norwegian, Costalba is Italian). The first version, Stockfish 1.0, was released in November 2008.[7][8] For a while, new ideas and code changes were transferred between the two programs in both directions, until Romstad decided to discontinue Glaurung in the favor of Stockfish, which was the more advanced engine at the time.[9] The last Glaurung (version 2.2) was released in December 2008.

Ich verstehe Deutsch und weiss was Titz bedeutet.

Uberigens eine "fork" = Derivat.

Wie wurde Stockfish zu einer Gabel? Und kein Derivat?

Admin wrote:

...I do understand the irony but nevertheless I find the name offensive to women.

I am somewhat conflicted: I think the way it has been used is sufficiently ironic to get away with it - especially in the context of the person who said it. On this basis, I'm happy to tolerate it.

The risk is that people whose speech is less well calibrated will see that it was used and garnered positive feedback, and will say such things in a less classy way. Basically, it's good, but it's a bad example to others.

Code:

Name                Gms  :  Win  : Draw  : Lose  :  Pts  
Lc0-v28             200  :  90+  : 100=  :  10-  : 140.0
SlowChess_2.7       200  :  10+  : 100=  :  90-  :  60.0

70% that's a lot.

Next : Lc0-v28 vs RubiChess 2.2

Same link - http://rebel13.nl/pgn4web-3.05/live-test.html