ProDeo
Would you like to react to this message? Create an account in a few clicks or log in to continue.
ProDeo

Computer Chess
 
HomeHome  CalendarCalendar  FAQFAQ  SearchSearch  MemberlistMemberlist  UsergroupsUsergroups  RegisterRegister  Log in  

 

 Testing the new Lc0

Go down 
5 posters
Go to page : 1, 2  Next
AuthorMessage
Admin
Admin
Admin


Posts : 1301
Join date : 2020-11-17
Location : Netherlands

Testing the new Lc0  Empty
PostSubject: Testing the new Lc0    Testing the new Lc0  EmptyThu Aug 26, 2021 7:00 pm

Lc0-v28 - SF14 for the GRL

200 games.

Live support via - http://rebel13.nl/pgn4web-3.05/live-test.html

TheSelfImprover likes this post

Back to top Go down
View user profile http://rebel13.nl/
Admin
Admin
Admin


Posts : 1301
Join date : 2020-11-17
Location : Netherlands

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptyFri Aug 27, 2021 4:33 pm

Code:
Name               Gms  :  Win  : Draw  : Lose  :  Pts  
Stockfish 14       200  :  63+  : 121=  :  16-  : 123.5  
Lc0-v28            200  :  16+  : 121=  :  63-  :  76.5  

History
Code:
Lc0-v27 - Stockfish 13  :  32%
Lc0-v28 - Stockfish 14  :  38.5%

Next Lc0-v28 - Komodo Dragon 2
Back to top Go down
View user profile http://rebel13.nl/
Admin
Admin
Admin


Posts : 1301
Join date : 2020-11-17
Location : Netherlands

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySat Aug 28, 2021 6:21 am

Code:
 7)  Lc0-v27             3532.5 :    800 (+307,=388,-105),  62.6 %
     vs.                        :  games (   +,   =,   -),   (%) :    Diff,    SD, CFS (%)
     Stockfish_13               :    100 (   3,  58,  39),  32.0 :  -139.6,  10.1,    0.0
     Komodo-Dragon              :    100 (  13,  67,  20),  46.5 :   -53.0,  11.4,    0.0
     Komodo_14                  :    100 (  43,  46,  11),  66.0 :  +108.1,  13.1,  100.0
     RubiChess_2.1              :    100 (  54,  40,   6),  74.0 :  +149.8,   9.1,  100.0
     SlowChess_2.5              :    100 (  51,  38,  11),  70.0 :  +164.4,   8.4,  100.0
     Igel_3.0.5                 :    100 (  42,  50,   8),  67.0 :  +171.0,  11.9,  100.0
     Ethereal_12.75             :    100 (  52,  42,   6),  73.0 :  +174.4,  11.8,  100.0
     Igel_3.0.0                 :    100 (  49,  47,   4),  72.5 :  +184.5,  12.0,  100.0

Looking at the (above) results of Lc0-v27 the new Lc0 looks a lot stronger, it scored 6% better against Stockfish and after 94 games it is in the clear lead against Komodo Dragon 2 while version 27 lost.

Code:
     Name             Gms  :  Win  : Draw  : Lose  :  Pts
Lc0-v28                94  :  26+  :  52=  :  16-  :  52.
Komodo-Dragon_2        94  :  16+  :  52=  :  26-  :  42.
Games played 94, games to go 106

http://rebel13.nl/pgn4web-3.05/live-test.html

TheSelfImprover likes this post

Back to top Go down
View user profile http://rebel13.nl/
Admin
Admin
Admin


Posts : 1301
Join date : 2020-11-17
Location : Netherlands

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySat Aug 28, 2021 6:48 pm

Code:
Lc0-v28               200  :  50+  : 104=  :  46-  : 102.
Komodo-Dragon 2       200  :  46+  : 104=  :  50-  :  98.

Next run - Lc0-v28 - Ethereal 13.25 NNUE.

Previous run - Lc0-v27 - Ethereal 12.75 : 73%
Back to top Go down
View user profile http://rebel13.nl/
Dio




Posts : 88
Join date : 2021-08-28

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySat Aug 28, 2021 7:56 pm

Hello Ed, thanks you for your test with LC0. For your tests on CPU, I would be interested to know which networks were used here (703810 for v.0.27 and 752187 for v.0.28 ?). I have been testing Lc0 on CPU myself for years. Did you use the DNNL version (which is much faster) or the blas version for these
tests ?
Back to top Go down
View user profile
Admin
Admin
Admin


Posts : 1301
Join date : 2020-11-17
Location : Netherlands

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySat Aug 28, 2021 8:03 pm

For v27 and 28 I used the net that came with the downloads.

v27 - 703810
v28 - 744204

Using indeed the DNNL version.

Welcome BTW.

Dio likes this post

Back to top Go down
View user profile http://rebel13.nl/
Dio




Posts : 88
Join date : 2021-08-28

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySat Aug 28, 2021 8:11 pm

Thanks for your answer, of course 744204 as this is the new standard network for blas/dnnl. I am curious about your tests on GPU, so far there have been no tests of large networks on GTX 1650.
Back to top Go down
View user profile
Admin
Admin
Admin


Posts : 1301
Join date : 2020-11-17
Location : Netherlands

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySat Aug 28, 2021 8:21 pm

The current gauntlets run on a GTX 1060 6Gb, network 609992.pb (130 Mb).

Mclane likes this post

Back to top Go down
View user profile http://rebel13.nl/
Dio




Posts : 88
Join date : 2021-08-28

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySat Aug 28, 2021 8:40 pm

Thanks Ed, very interesting, your results on CPU corresponds very well to my own
results, is it possible to post a png file here ?
Back to top Go down
View user profile
Admin
Admin
Admin


Posts : 1301
Join date : 2020-11-17
Location : Netherlands

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySat Aug 28, 2021 10:07 pm

[pgn][Event "Lc0-v28-et1325"]
[Site "EDDY-PC"]
[Date "2021.08.28"]
[Round "21"]
[White "Lc0-v28"]
[Black "Ethereal_13.25-NNUE"]
[Result "1-0"]
[BlackElo "2200"]
[ECO "C37"]
[Opening "KGA"]
[Time "22:01:25"]
[Variation "Sörensen Gambit"]
[WhiteElo "2200"]
[TimeControl "40/120:40/120:40/120"]
[Termination "adjudication"]
[PlyCount "42"]
[WhiteType "program"]
[BlackType "program"]

1. e4 e5 2. f4 exf4 3. Nf3 g5 {+0.40/20 6} 4. d4 {-0.24/19 3} g4 {+0.88/21
3} 5. Ne5 {-0.26/23 2} Qh4+ {+1.92/22 6} 6. g3 {-0.26/20 2} fxg3 {+1.79/25
7} 7. Qxg4 {-0.29/20 3} g2+ {+0.88/23 4} 8. Qxh4 {0.00/19 6} gxh1=Q
{0.00/24 7} 9. Nc3 {0.00/20 3} Bb4 {+0.20/21 2} 10. Kf2 {+0.40/10 6} h5
{-0.71/23 12} 11. Nd5 {+0.99/8 2} Be7 {-1.25/23 10} 12. Bg5 {+0.81/10 3} f6
{-1.86/23 9} 13. Qf4 {+1.05/14 0} Na6 {-0.73/24 5} 14. Qf5 {+3.90/8 2}
Qxh2+ {-8.99/20 5} 15. Bg2 {+5.71/10 1} fxe5 {-10.12/22 5} 16. Qg6+
{+6.43/12 4} Kd8 {-11.63/22 4} 17. Qg7 {+9.02/12 5} d6 {-11.98/22 3} 18.
Rh1 {+12.15/12 8} Qxg2+ {-8.97/22 4} 19. Kxg2 {+11.17/12 2} Bxg5 {-9.44/21
1} 20. Qxg5+ {+13.29/10 2} Kd7 {-9.95/22 2} 21. Qg7+ {+18.36/1 0} Kc6
{-9.73/22 3  Arena Adjudication} 1-0[/pgn]

Nice firework.

TheSelfImprover and Dio like this post

Back to top Go down
View user profile http://rebel13.nl/
Mclane

Mclane


Posts : 2157
Join date : 2020-11-17
Age : 55
Location : United States of Europe, Germany, Ruhr area

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySun Aug 29, 2021 2:14 am

I am testing lc0 on a 2070 vs. Stockfish and other running 8 cores and permanent brain
Back to top Go down
View user profile http://www.thorstenczub.de
Admin
Admin
Admin


Posts : 1301
Join date : 2020-11-17
Location : Netherlands

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySun Aug 29, 2021 9:29 am

Mclane wrote:
I am testing lc0 on a 2070 vs. Stockfish and other running 8 cores and permanent brain

Please post results.

BTW, I am toying with the idea to open a second list, top-10 engines that compete on 16 or 20 cores. It has never been done before but I can't do it alone.

Are you and Mark Young willing to participate?

Mclane likes this post

Back to top Go down
View user profile http://rebel13.nl/
Admin
Admin
Admin


Posts : 1301
Join date : 2020-11-17
Location : Netherlands

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySun Aug 29, 2021 10:56 am

Made a start, current top-10 GRL, 20 cores, hash 4Gb, games 9000 [!], each playing each the 200 gambit positions.

http://rebel13.nl/b/grl.htm

For the moment at bullet 40/10.

Somehow Lc0-v28 should move in but at a good GPU.


Mclane likes this post

Back to top Go down
View user profile http://rebel13.nl/
Mclane

Mclane


Posts : 2157
Join date : 2020-11-17
Age : 55
Location : United States of Europe, Germany, Ruhr area

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySun Aug 29, 2021 11:18 am

I have a 16 core machine.
because i am doing permanent brain, i chose 8 cores for each engine plus LC0 on 2070.
of course another test could be done with more cores but only if permanent brain is off then.

i am testing with arena so far. the advantage is that arena is doing all alone and i can watch the whole
tournament like a cinema-movie before my eyes.

it automatically ranks the engines. it automatically changes opponents. it automatically arranges the whole thing.

a new participant (e.g. a new version of LC0 in arena) means it only has to play the 300-400 games and ranks automatically in the big data pool. so the tournament is an elo list on its own.

Back to top Go down
View user profile http://www.thorstenczub.de
Dio




Posts : 88
Join date : 2021-08-28

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySun Aug 29, 2021 6:15 pm

Testing the new Lc0  11_rat10
my own 1min + 1sec "rating-list" on CPU (2 threads each) with Lc0 703810 and 74706
Back to top Go down
View user profile
Dio




Posts : 88
Join date : 2021-08-28

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySun Aug 29, 2021 6:26 pm

the Elo difference between 703810 and 744706 fits perfectly with your own tests on CPU.

Admin likes this post

Back to top Go down
View user profile
Admin
Admin
Admin


Posts : 1301
Join date : 2020-11-17
Location : Netherlands

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySun Aug 29, 2021 8:22 pm

Code:
Name                      Gms  :  Win  : Draw  : Lose  :
Lc0-v28                   200  :  86+  :  94=  :  20-  :
Ethereal_13.25-NNUE       200  :  20+  :  94=  :  86-  :

It wouldn't surprise me if Lc0 will take over the second place of the Dragon.

Next : Lc0-v28 vs Slowchess 2.7

Same link - http://rebel13.nl/pgn4web-3.05/live-test.html

Dio likes this post

Back to top Go down
View user profile http://rebel13.nl/
Dio




Posts : 88
Join date : 2021-08-28

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySun Aug 29, 2021 8:32 pm

strong result against Ethereal about +105 Elo. Thanks for your tests..
Back to top Go down
View user profile
mwyoung

mwyoung


Posts : 501
Join date : 2020-11-25
Location : USA

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySun Aug 29, 2021 9:17 pm

Admin wrote:
Code:
Name                      Gms  :  Win  : Draw  : Lose  :
Lc0-v28                   200  :  86+  :  94=  :  20-  :
Ethereal_13.25-NNUE       200  :  20+  :  94=  :  86-  :

It wouldn't surprise me if Lc0 will take over the second place of the Dragon.

Next : Lc0-v28 vs Slowchess 2.7

Same link - http://rebel13.nl/pgn4web-3.05/live-test.html

You could change you mind after testing Fat Titz. It could be Stockfish that is #2.

Code:
Rank Name                          Elo     +/-   Games    Wins  Losses   Draws   Points   Score    Draw
   1 Fat Titz 1.0                   17      20      84       5       1      78     44.0   52.4%   92.9%
   2 Stockfish 08/27/21             13      18      83       4       1      78     43.0   51.8%   94.0%
   3 Stockfish 14                   13      22      83       5       2      76     43.0   51.8%   91.6%
   4 Lc0 0.28.0                    -42      27      84       1      11      72     37.0   44.0%   85.7%

167 of 600 games finished.
Back to top Go down
View user profile
Admin
Admin
Admin


Posts : 1301
Join date : 2020-11-17
Location : Netherlands

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySun Aug 29, 2021 9:20 pm

I don't include derivatives. Besides of that I do understand the irony but nevertheless I find the name offensive to women.
Back to top Go down
View user profile http://rebel13.nl/
mwyoung

mwyoung


Posts : 501
Join date : 2020-11-25
Location : USA

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySun Aug 29, 2021 9:25 pm

Admin wrote:
I don't include derivatives. Besides of that I do understand the irony but nevertheless I find the name offensive to women.

Fat Titz name has nothing to do with woman. Know your German.

Example.

Titz is a municipality in the district of Düren in the state of North Rhine-Westphalia,


Fat Titz is a legal fork.

"I don't include derivatives." But you do!

Stockfish (chess)

The program originated from Glaurung, an open-source chess engine created by Romstad and first released in 2004. Four years later, Costalba, inspired by the strong open-source engine, decided to fork the project. He named it Stockfish because it was "produced in Norway and cooked in Italy" (Romstad is Norwegian, Costalba is Italian). The first version, Stockfish 1.0, was released in November 2008.[7][8] For a while, new ideas and code changes were transferred between the two programs in both directions, until Romstad decided to discontinue Glaurung in the favor of Stockfish, which was the more advanced engine at the time.[9] The last Glaurung (version 2.2) was released in December 2008.
Back to top Go down
View user profile
Admin
Admin
Admin


Posts : 1301
Join date : 2020-11-17
Location : Netherlands

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySun Aug 29, 2021 9:34 pm

Ich verstehe Deutsch und weiss was Titz bedeutet.

Uberigens eine "fork" = Derivat.
Back to top Go down
View user profile http://rebel13.nl/
mwyoung

mwyoung


Posts : 501
Join date : 2020-11-25
Location : USA

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySun Aug 29, 2021 9:40 pm

Wie wurde Stockfish zu einer Gabel? Und kein Derivat?
Back to top Go down
View user profile
TheSelfImprover

TheSelfImprover


Posts : 2004
Join date : 2020-11-18

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptySun Aug 29, 2021 11:33 pm

Admin wrote:
...I do understand the irony but nevertheless I find the name offensive to women.


I am somewhat conflicted: I think the way it has been used is sufficiently ironic to get away with it - especially in the context of the person who said it. On this basis, I'm happy to tolerate it.

The risk is that people whose speech is less well calibrated will see that it was used and garnered positive feedback, and will say such things in a less classy way. Basically, it's good, but it's a bad example to others.
Back to top Go down
View user profile
Admin
Admin
Admin


Posts : 1301
Join date : 2020-11-17
Location : Netherlands

Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  EmptyMon Aug 30, 2021 6:50 pm

Code:
Name                Gms  :  Win  : Draw  : Lose  :  Pts  
Lc0-v28             200  :  90+  : 100=  :  10-  : 140.0
SlowChess_2.7       200  :  10+  : 100=  :  90-  :  60.0

70% that's a lot.

Next : Lc0-v28 vs RubiChess 2.2

Same link - http://rebel13.nl/pgn4web-3.05/live-test.html

Dio likes this post

Back to top Go down
View user profile http://rebel13.nl/
Sponsored content





Testing the new Lc0  Empty
PostSubject: Re: Testing the new Lc0    Testing the new Lc0  Empty

Back to top Go down
 
Testing the new Lc0
Back to top 
Page 1 of 2Go to page : 1, 2  Next
 Similar topics
-
» Stockfish Testing 03/08/21
» Testing new util
» Stockfish 13 Gauntlet Testing
» Book testing (tuning) with ProDeo
» Testing at fixed depth -- a question to Ed

Permissions in this forum:You cannot reply to topics in this forum
ProDeo :: Computer Chess-
Jump to: