ProDeo
Would you like to react to this message? Create an account in a few clicks or log in to continue.
ProDeo

Computer Chess
 
HomeHome  CalendarCalendar  Latest imagesLatest images  FAQFAQ  SearchSearch  MemberlistMemberlist  UsergroupsUsergroups  RegisterRegister  Log in  

 

 First part of the time-odds experiment finished

Go down 
4 posters
AuthorMessage
Admin
Admin
Admin


Posts : 2608
Join date : 2020-11-17
Location : Netherlands

First part of the time-odds experiment finished Empty
PostSubject: First part of the time-odds experiment finished   First part of the time-odds experiment finished EmptyFri Oct 08, 2021 10:20 am

First part of the time-odds experiment finished.

First part of the time-odds experiment finished Draw-rate

See page - http://rebel13.nl/rebel13/time-odds-matches.html

Next, measuring the diminishing returns.

http://rebel13.nl/b/grl.htm

TheSelfImprover and adminx like this post

Back to top Go down
http://rebel13.nl/
TheSelfImprover

TheSelfImprover


Posts : 3112
Join date : 2020-11-18

First part of the time-odds experiment finished Empty
PostSubject: Re: First part of the time-odds experiment finished   First part of the time-odds experiment finished EmptyFri Oct 08, 2021 1:26 pm

Interesting: Dragon makes better use of the extra time than Stockfish does.

My guess is that this is down to more accurate evaluation of positions. One could test this by repeating the experiment with handwritten stockfish eval v NN stockfish eval and see whether the gap remains as large.
Back to top Go down
Admin
Admin
Admin


Posts : 2608
Join date : 2020-11-17
Location : Netherlands

First part of the time-odds experiment finished Empty
PostSubject: Re: First part of the time-odds experiment finished   First part of the time-odds experiment finished EmptyFri Oct 08, 2021 4:59 pm

Probably so. Komodo scales a lot better than Stockfish. It could already be concluded from the GRL at 20 cores. This is a second proof.
Back to top Go down
http://rebel13.nl/
Uri Blass




Posts : 207
Join date : 2020-11-28

First part of the time-odds experiment finished Empty
PostSubject: Re: First part of the time-odds experiment finished   First part of the time-odds experiment finished EmptyFri Oct 08, 2021 5:19 pm

Admin wrote:
Probably so. Komodo scales a lot better than Stockfish. It could already be concluded from the GRL at 20 cores. This is a second proof.

I do not see that komodo scales a lot better than stockfish.
Komodo cannot win against stockfish even with many cores based on mark young's matches.

I suspect that the bigger rating difference may be because of contempt that stockfish does not have.
Note that I prefer to test stockfish developement version and not stockfish14 but I do not believe Dragon can even beat stockfish14 at long time control(something that I could expect to see at some time control if dragon komodo scales better).


Last edited by Uri Blass on Fri Oct 08, 2021 9:20 pm; edited 1 time in total (Reason for editing : I did a mistake when I explained stockfish has contempt when I meant the opposite)
Back to top Go down
mwyoung

mwyoung


Posts : 880
Join date : 2020-11-25
Location : USA

First part of the time-odds experiment finished Empty
PostSubject: Re: First part of the time-odds experiment finished   First part of the time-odds experiment finished EmptyFri Oct 08, 2021 9:12 pm

Admin wrote:
First part of the time-odds experiment finished.

First part of the time-odds experiment finished Draw-rate

See page - http://rebel13.nl/rebel13/time-odds-matches.html

Next, measuring the diminishing returns.

http://rebel13.nl/b/grl.htm

I know some think that this is caused by the opening book. Well if you take the book out of the equation(chess 960). You still get the same results.
Back to top Go down
Admin
Admin
Admin


Posts : 2608
Join date : 2020-11-17
Location : Netherlands

First part of the time-odds experiment finished Empty
PostSubject: Re: First part of the time-odds experiment finished   First part of the time-odds experiment finished EmptySat Oct 09, 2021 7:11 am

Uri Blass wrote:
Admin wrote:
Probably so. Komodo scales a lot better than Stockfish. It could already be concluded from the GRL at 20 cores. This is a second proof.

I do not see that komodo scales a lot better than stockfish.
Komodo cannot win against stockfish even with many cores based on mark young's matches.

Code:
GRL one core       3693 - 3649 = 44 elo
Stockfish 14       3693
Dragon 2.5         3649

GRL 20 cores       3608 - 3595 = 13 elo
Stockfish 14       3608
Dragon 2.5         3595
Back to top Go down
http://rebel13.nl/
Admin
Admin
Admin


Posts : 2608
Join date : 2020-11-17
Location : Netherlands

First part of the time-odds experiment finished Empty
PostSubject: Re: First part of the time-odds experiment finished   First part of the time-odds experiment finished EmptySat Oct 09, 2021 8:58 am

mwyoung wrote:
Admin wrote:
First part of the time-odds experiment finished.

First part of the time-odds experiment finished Draw-rate

See page - http://rebel13.nl/rebel13/time-odds-matches.html

Next, measuring the diminishing returns.

http://rebel13.nl/b/grl.htm

I know some think that this is caused by the opening book. Well if you take the book out of the equation(chess 960). You still get the same results.

I never test with books, no single chess programmer will, no rating list also except the SSDF. Here is why:

I played a quick 1000 game match, the latest SF-DEV vs SF14 with the Perfect-2021 you are using. I have an util SOMU -> Double Openings, it states:

Code:
Processing game 1.000
Storing game    656
Double openings 344

Nevertheless, the DEV version won convincingly, TC=40/10.

Code:
Score of stockfish_21-10-06 vs sf14: 145 - 54 - 801  [0.545] 1000
...      stockfish_21-10-06 playing White: 104 - 11 - 385  [0.593] 500
...      stockfish_21-10-06 playing Black: 41 - 43 - 416  [0.498] 500
...      White vs Black: 147 - 52 - 801  [0.547] 1000
Elo difference: 31.7 +/- 9.5, LOS: 100.0 %, DrawRatio: 80.1 %
Finished match

BTW, I have seen much worse books.

First part of the time-odds experiment finished Somu1
Back to top Go down
http://rebel13.nl/
Admin
Admin
Admin


Posts : 2608
Join date : 2020-11-17
Location : Netherlands

First part of the time-odds experiment finished Empty
PostSubject: Re: First part of the time-odds experiment finished   First part of the time-odds experiment finished EmptySat Oct 09, 2021 9:56 am

Started a match for the GRL with the 21-10-06 version.

http://rebel13.nl/a/grl.htm
Back to top Go down
http://rebel13.nl/
Uri Blass




Posts : 207
Join date : 2020-11-28

First part of the time-odds experiment finished Empty
PostSubject: Re: First part of the time-odds experiment finished   First part of the time-odds experiment finished EmptySat Oct 09, 2021 11:07 am

Admin wrote:
mwyoung wrote:
Admin wrote:
First part of the time-odds experiment finished.

First part of the time-odds experiment finished Draw-rate

See page - http://rebel13.nl/rebel13/time-odds-matches.html

Next, measuring the diminishing returns.

http://rebel13.nl/b/grl.htm

I know some think that this is caused by the opening book. Well if you take the book out of the equation(chess 960). You still get the same results.

I never test with books, no single chess programmer will, no rating list also except the SSDF. Here is why:

I played a quick 1000 game match, the latest SF-DEV vs SF14 with the Perfect-2021 you are using. I have an util SOMU -> Double Openings, it states:

Code:
Processing game 1.000
Storing game    656
Double openings 344

Nevertheless, the DEV version won convincingly, TC=40/10.

Code:
Score of stockfish_21-10-06 vs sf14: 145 - 54 - 801  [0.545] 1000
...      stockfish_21-10-06 playing White: 104 - 11 - 385  [0.593] 500
...      stockfish_21-10-06 playing Black: 41 - 43 - 416  [0.498] 500
...      White vs Black: 147 - 52 - 801  [0.547] 1000
Elo difference: 31.7 +/- 9.5, LOS: 100.0 %, DrawRatio: 80.1 %
Finished match

BTW, I have seen much worse books.

First part of the time-odds experiment finished Somu1

1)Only ssdf test with own books but usually testers test with books but not when every engine use its own book.
The only test without book is in FRC games
Testing with Perfect-2021 is testing with books because maybe some decisive results are result of the book.

2)I see no problem with no books in normal chess and I think that it may be interesting if there are books that can improve the rating of engines relative to no book.

It is possible to avoid double opening by learning(assuming you do not like to repeat the same line that you lost with black or drew with white) but
If the engines do not learn then it is possible to develop learning by the interface in the following way assuming the engines support multi-pv

Use multi-pv with 2 options in the first moves and choose a different move that the engine never got bad result with it that has the same score as the best move as fast as possible.

A bad result is a draw for white or a loss for black.
Back to top Go down
Admin
Admin
Admin


Posts : 2608
Join date : 2020-11-17
Location : Netherlands

First part of the time-odds experiment finished Empty
PostSubject: Re: First part of the time-odds experiment finished   First part of the time-odds experiment finished EmptySat Oct 09, 2021 9:21 pm

Admin wrote:
Started a match for the GRL with the 21-10-06 version.

http://rebel13.nl/a/grl.htm

After 1000 games :

Code:
   # PLAYER                 :  RATING  ERROR  POINTS  PLAYED   (%)  CFS(%)     W     D     L  D(%)
   1 Stockfish 21-10-06     :  3703.7   15.8   769.5    1000    77      90   556   427    17    43
   2 Stockfish 14           :  3691.4   13.9  3527.0    4500    78      99  2657  1740   103    39
   3 Stockfish 13           :  3674.9    9.0  2660.0    3500    76     100  1915  1490    95    43
   4 Stockfish 21-05-18     :  3657.3   16.3   841.0    1100    76      89   617   448    35    41
   5 Komodo-Dragon 2.5      :  3647.8   15.2  1527.5    2200    69     100   998  1059   143    48
   6 Stockfish 12           :  3623.7    9.1  1903.0    2800    68     100  1222  1362   216    49
   7 Komodo-Dragon 2        :  3590.7    9.2  3388.5    4700    72      62  2385  2007   308    43
   8 Komodo-Dragon          :  3588.2   13.3  1998.5    3000    67      51  1317  1363   320    45
   9 Lc0 v28                :  3587.8   30.6   591.5    1000    59      99   335   513   152    51
  10 Lc0 v27                :  3535.1   20.3   501.0     800    63      96   307   388   105    49
Back to top Go down
http://rebel13.nl/
Sponsored content





First part of the time-odds experiment finished Empty
PostSubject: Re: First part of the time-odds experiment finished   First part of the time-odds experiment finished Empty

Back to top Go down
 
First part of the time-odds experiment finished
Back to top 
Page 1 of 1
 Similar topics
-
» Stockfish and Dragon time odds matches
» how to beat stockfish latest version with queen odds at 1+1 time control
» FUN with rating lists [ PART ONE]
» FUN with rating lists [ PART TWO]
» Experiment by Gothamchess

Permissions in this forum:You cannot reply to topics in this forum
ProDeo :: Computer Chess-
Jump to: