ProDeo
Would you like to react to this message? Create an account in a few clicks or log in to continue.
ProDeo

Computer Chess
 
HomeHome  CalendarCalendar  Latest imagesLatest images  FAQFAQ  SearchSearch  MemberlistMemberlist  UsergroupsUsergroups  RegisterRegister  Log in  

 

 SSDF Rating List 23-05-24

Go down 
+4
Dio
Ozymandias
Admin
Lars Sandin
8 posters
Go to page : 1, 2  Next
AuthorMessage
Lars Sandin




Posts : 11
Join date : 2023-05-24

SSDF Rating List 23-05-24 Empty
PostSubject: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptyWed May 24, 2023 7:04 pm

Code:
THE SSDF RATING LIST 2023-05-24*  162991 games played by  435 computers                     
                                          Rating  +     -  Games  Won  Oppo
                                          ------  ---  --- -----  ---  ----
  1 Lc0 0.29.0 Cuda-808544 3060Ti          3610  45  -41  287  68%  3466
  2 Lc0 0.28.2 Cuda-611213 3060Ti          3571  26  -25  780  66%  3456
  3 Lc0 0.26.3 Cuda(67362) 3060Ti          3570  29  -27  680  68%  3441
  4 Stockfish 15 x64 1800X 3.6 GHz          3570  39  -36  367  66%  3456
  5 Stockfish 13 x64 1800X 3.6 GHz          3568  35  -32  480  69%  3432
  6 Dragon Komodo 3 x64 1800X 3.6 GHz      3567  40  -38  329  62%  3482
  7 Dragon Komodo 2.51 x64 1800X 3.6 GHz    3565  42  -39  320  65%  3454
  8 Stockfish 14 x64 1800X 3.6 GHz          3555  37  -35  400  65%  3452
  9 Stockfish 12 NNUE x64 1800X 3.6 GHz    3549  26  -25  760  60%  3479
  10 Rebel 16.2 x64 1800X 3.6 GHz            3544  48  -47  209  53%  3522
  11 Dragon by Komodo x64 1800X 3.6 GHz      3536  34  -32  460  63%  3443
  12 Stockfish 11 x64 1800X 3.6 GHz          3529  36  -34  450  70%  3386
  13 Arasan 23.4 x64 1800X 3.6 GHz          3524  54  -55  160  47%  3560
  14 Dragon Komodo 3.1 MCTS 1800X 3.6 GHz    3514  34  -33  440  60%  3447
  15 Stockfish 10 x64 1800X 3.6 GHz          3501  25  -23  920  67%  3378
  16 Booot 7 x64 1800X 3.6 GHz              3477  39  -40  300  46%  3507
  17 Dragon Komodo 2 MCTS x64 1800X 3.6 GHz  3473  45  -44  240  55%  3439
  18 Stockfish 9 x64 1800X 3.6 GHz          3464  24  -23  962  67%  3345
  19 Komodo 14 x64 1800X 3.6 GHz            3455  30  -30  520  52%  3441
  20 Arasan 23.01 x64 1800X 3.6 GHz          3449  33  -34  420  48%  3462
  21 Arasan 23.4 x64 Q6600 2.4 GHz          3439  40  -37  361  68%  3314
  22 Pedone 3.1 x64 1800X 3.6 GHz            3433  38  -39  320  45%  3468
  23 Wasp 6.5 x64 1800X 3.6 GHz              3433  44  -45  242  44%  3478
  24 Stockfish 9 x64 Q6600 2.4 GHz          3427  32  -31  480  56%  3384
  25 Booot 6.5 x64 1800X 3.6 GHz            3426  34  -34  400  48%  3436
  26 Wasp 5.5 x64 1800X 3.6 GHz              3378  37  -39  360  36%  3486
  27 Arasan 22.3 x64 1800X 3.6 GHz          3360  39  -42  320  35%  3468
  28 Marvin 6.0.0 x64 1800X 3.6 GHz          3358  47  -53  220  31%  3494
  29 Pedone 3.1 x64 Q6600 2.4 GHZ            3356  38  -37  340  58%  3303
  30 Deep Shredder 13 x64 1800X 3.6 GHz      3347  24  -23  920  62%  3261
  31 Chiron 5.01 x64 1800X 3.6 GHz          3344  33  -35  440  34%  3455
  32 Marvin 5.2.0 x64 1800X 3.6 GHz          3325  37  -40  360  32%  3456
  33 Chiron 5.01 x64 Q6600 2.4 GHz          3318  34  -34  400  52%  3301
  34 Vajolet2 2.8 x64 1800X 3.6 GHz          3279  27  -28  652  38%  3364
  35 Deep Hiarcs 14 1800X 3.6 GHz            3206  23  -24  880  38%  3289
  36 Revelation 2 A.E. Komodo 12.1 A7 1 GHz  3177  73  -68  100  59%  3113
  37 Revelation 2 Hiarcs 14.1 PXA320 800 MHz 2924  47  -45  228  56%  2881
  38 Chessmaster King 3.5 x64 Q6600 2.4 GHz  2857  24  -25  932  30%  3005
  39 Revelation Hiarcs 13.3 PXA255 500 MHz  2772  57  -52  177  66%  2660
  40 Revelation Shredder 12 PXA255 500 MHz  2706  60  -58  140  56%  2666
  41 Revelation Rybka 2.2 PXA255 500 MHz    2635  45  -43  260  63%  2546
  42 Revelation Deep Sjeng 3 PXA255 500 MHz  2600  62  -67  120  41%  2665
  43 Millennium The King Exclusive 300 MHz  2534  49  -48  200  55%  2501
  44 Revelation Ruffian 2.1 PXA255 500 MHz  2346  68  -71  100  45%  2385
  45 Millennium ChessGenius Excl. M7 300 MHz 2249  50  -48  204  58%  2193
  46 Mephisto London 68030 33 MHz            2195  31  -31  482  50%  2192
  47 Millennium ChessGenius Pro M4 120 MHz  2163  59  -54  160  63%  2067
  48 Mephisto London 68020 12 MHz            2089  60  -53  171  68%  1957
  49 Millennium ChessGenius ARM M4 48 MHz    2070  45  -43  251  58%  2015
  50 Mephisto Roma II 68000 10 MHz          1898  63  -59  132  58%  1843

 1 Lc0 0.29.0 Cuda-808544 3060Ti, 3610                                 
Stoc15 1800X      4-3    Reb162 1800X    21,5-18,5  Aras234 1800X    46-34
Stock9 1800X      29-11    Wasp65 1800X    28,5-11,5  Wasp55 1800X      66-14

 10 Rebel 16.2 x64 1800X 3.6 GHz, 3544                                 
Lc029 3060Ti    18,5-21,5  Dra3 1800X      4,5-4,5  Stoc14 1800X      19-21
Aras234 1800X  20,5-19,5  Boo7 1800X      23,5-16,5  Wasp65 1800X      24-16

 13 Arasan 23.4 x64 1800X 3.6 GHz, 3524                                 
Lc029 3060Ti      34-46    Reb162 1800X    19,5-20,5  Boo7 1800X        21-19

 21 Arasan 23.4 x64 Q6600 2.4 GHz, 3439                                 
Komo12 Q6600    21,5-18,5  Stock8 Q6600    21,5-18,5  Ped31 Q6600    22,5-17,5
Chi5 Q6600      85,5-34,5  DShre13 Q6600    24-16    Vaj28 Q6600        1-0
Mar51 Q6600    33,5-6,5  DRybka3 Q6600  34,5-5,5 

 23 Wasp 6.5 x64 1800X 3.6 GHz, 3433                                 
Lc029 3060Ti    11,5-28,5  Stoc15 1800X    11,5-28,5  Reb162 1800X      16-24
Ped3 1800X      44,5-35,5  Komo11 1800X    21,5-18,5  Vaj28 1800X        2-0

 36 Revelation 2 A.E. Komodo 12.1 A7 1 GHz, 3177                                 
DShre13 Q6600      6-14    DHiar14 Q6600    9,5-10,5  DJuni12 Q6600  11,5-8,5
Baro343 Q6600  15,5-4,5  Naum 4  A1200  16,5-3,5 

Since the last rating list we can now present six new entrants in the list.

First one out is our new leader in the list: Lc0 0.29.0, and also the first
program that has broken the 3600-barrier in our rating list. More games will
be needed to lower the error bars, but after the first 297 games it has placed
itself 39 points over the last version tested with a huge rating of 3610!
It will be interesting to see how it will hold up as more games are played
against the top programs.
The Lc0-version which we have tested this time is: Lc0 0.29.0, with the
19 filter + 512 block network, named 808544 (date: 2023-01-12). As with the
earlier Lc0-versions, we have used the free opening book "perfect2021.abk"
by Sedat Canbaz for the testing of Lc0.

Our next newcomer in the list are one really strong comeback from the team of
Ed Schroeder and Chris Wittington. Chris Wittington is formely known for
programs such as: Chess Player 2150/2175, Complete Chess System
and Chess System Tal (I and II).
Ed Schroeder is of course known for many of the old dedicated chess computers
like for instance: Mephisto Polgar, MM IV/V, RISC (I and II), Rebell
and also for versions of his program in the ChessMachine ISA-plugin card for PC.
On the PC-scene he has made the Gideon and Rebel-program famous and lately also
made several interesting and strong versions of his ProDeo-program.

Since last year they have utilized Neural Net-technology in their joint
effort and has seemingly improved fast. We have waited some time to test it,
since we wanted it to share most of the functions - like for instance Pondering
and Multi-Processor functionality, which the others programs that we test on our
latest hardware uses. The version which we have tested is named: Rebel 16.2 and
we have used its own opening book, made by the famous Jeroen Noomen, for the
testing. After the first 209 games, this strong combo of the program: Rebel 16.2,
and the opening book of Jeroen Noomen, has managed to get an impressive rating
of 3544 on our 1800X hardware. It is placed 10th in the list at the moment,
just 5 points shy of the Stockfish 12-program.
A very strong comeback by Ed, Chris and Jeroen!

Our next addition to the list are Jon Dart's strong and reliable Arasan-program.
The version that we have tested on our 1800X and Q6600 hardware are Arasan 23.4.
On our 1800X hardware it has reached a rating of 3524 after the initial 160
games played. That is at the moment 75 points ahead of the formerly tested 23.01
version and a very nice improvement indeed! More games will of course be needed
to lower the error bars a bit.
On our Q6600 hardware, Arasan 23.4 has reached a rating of 3439 after 361 games
played. We never tested the 23.01 version on the Q6600 hardware, but Arasan 23.4
are a whopping 202 points ahead of the 21.2 version at least!
We have, as always, used Arasan's own opening book for the testing.                 

We are also glad to be able to present a new strong Wasp version by John Stanback,
namely Wasp 6.5, on our latest rating list. After the first 242 games played, it
has reached a rating of 3433! This is 55 points stronger than the 5.5 version
which we tested before, so a nice improvement there also! We have used Wasp's own
opening book for the testing.

And we can now finally show the first rating of the manually tested:
Phoenix Revelation II Anniversary Edition chess computer by Ruud Martin/DGT. This
Anniversary Edition of the Revelation II, features an updated hardware relative
to the ordinary Revelation II. The hardware for the Anniversary Edition is an
Colibri iMX7D with 2 ARM Cortex A7 processors running at 1GHz. The software which
we have used for the testing is the program Komodo 12.1 by Mark Lefler and
Larry Kaufman.
The rating that has been reached after the first 100 games is 3177! This is a
rating that is 253 points ahead of Hiarcs 14.1, which we tested formerly
on the ordinary Revelation II. It is clearly the strongest dedicated chess
computer which we have tested so far and it has been able to perform admirably
against the strong Q6600 opposition which it mostly has faced in our testing
so far.

Aside from producing more games with the already mentioned newcomers in the
rating list, we are at the moment testing Richard Pijl's - The Baron 3.45,
and also Alex Morozov's - Booot 7.1. We also hope to have some more programs
ready for the next rating list. 

* This rating list was produced and based on the games/results that were finished
on the first of May, but the publication of this rating list was unfortunately a
bit delayed this time.
The testing has of course not stood still since then, and the games/results
that has been played since this date will of course feature in the next rating
list, which we plan to have out sometime in august/september.

Lars Sandin

Admin, Mclane, mwyoung, Dio, Ipmanchess and Ghppn like this post

Back to top Go down
Admin
Admin
Admin


Posts : 2528
Join date : 2020-11-17
Location : Netherlands

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptyWed May 24, 2023 10:39 pm

That's a very nice performance, thank you for testing.

Do you still play the games at 40/2h tournament time control?

Ghppn likes this post

Back to top Go down
http://rebel13.nl/
Admin
Admin
Admin


Posts : 2528
Join date : 2020-11-17
Location : Netherlands

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptyFri May 26, 2023 1:41 pm

By the lack of answer, assuming the SSDF still plays 40/2h SF hardly made progress after version 12.

Ghppn likes this post

Back to top Go down
http://rebel13.nl/
Ozymandias




Posts : 622
Join date : 2020-11-23

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptyFri May 26, 2023 3:13 pm

SSDF wrote:
All games have been played on the tournament level, 40 moves/2 hours followed by 20 moves/each following hour. In matches between PC-programs, two separate PCs have been used, connected with an auto232-cable.

What I don't know is what books are used with engines which don't have one and what CPU the 3060Ti enjoys.

Ghppn likes this post

Back to top Go down
https://centaurchess.boards.net/
Dio




Posts : 214
Join date : 2021-08-28

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptyFri May 26, 2023 7:40 pm

with the GPU used, a relatively up-to-date CPU is certainly used. The CPU used is practically irrelevant in tests with Lc0 when using a graphics card. Only 2 threads are used by default. However, there are "experts" who use 16 threads or more and are then surprised about the poor performance of Lc0.

Ghppn and Lars Sandin like this post

Back to top Go down
Lars Sandin




Posts : 11
Join date : 2023-05-24

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptySat May 27, 2023 6:56 pm

Admin wrote:
That's a very nice performance, thank you for testing.

Do you still play the games at 40/2h tournament time control?

Yes, we are still testing with the same settings/time controls as before, that is 40moves/2h and then 20moves/1h continously.

Admin, Mclane, Dio, Ghppn and kaissa like this post

Back to top Go down
Lars Sandin




Posts : 11
Join date : 2023-05-24

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptySat May 27, 2023 7:40 pm

Ozymandias wrote:
SSDF wrote:
All games have been played on the tournament level, 40 moves/2 hours followed by 20 moves/each following hour. In matches between PC-programs, two separate PCs have been used, connected with an auto232-cable.

What I don't know is what books are used with engines which don't have one and what CPU the 3060Ti enjoys.

We try to get some general advice from the programmer in question when we chose which opening book we use. We have a policy which we try to follow for this part (if a program is released without an own book that is). Here are the guidelines which follows our present policy for testing free engines without an own book and which are released without it's own auto232-compatible GUI:

1. The programmer's wish for a suitable opening book is of course the most prefered choice. If the programmer doesn't want to chose or if we can't get in contact with the programmer, we can chose a general book in this case.
1.1 The opening book must be available for free. We are not allowing private books which are not available for the public.
1.2 As we test all free engines in the freely availble GUI: Arena (3.5.1) - the opening book chosen must be of either .abk or .bin type (or internal that can be accessed by the program). We can't use a .ctg book because that forces us to use a commercial GUI to test the free engine. Someone that want to rerun our tests of a freely available engine should be able to do so without any extra cost.
1.3 The opening book chosen should not have been specifically made for any other engine than the one we test. It's therefore perfectly acceptable to use a dedicated opening book made by someone for a specific engine (like when we test SF), but in that case the book is not suitable for testing any other engine. It must also adhere to 1.1 and 1.2 of course.  

Any typo or something missing from the above could be blamed on my memory, as the above mentioned guidelines are directly taken from memory and not from a formerly written prospect.

We are using the 3060Ti graphic card on a 1800X-machine and use 2 threads on the 1800X, but as Dio already mentioned in his informative post, it is relatively unimportant as it is the 3060Ti which do most of the work when testing Lc0 with a dedicated graphic card.

Dio and Ghppn like this post

Back to top Go down
Lars Sandin




Posts : 11
Join date : 2023-05-24

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptySat May 27, 2023 7:48 pm

I forgot to mention that the opening book chosen for the engines should be named in the information to the rating list in which the engine was first introduced.

Dio and Ghppn like this post

Back to top Go down
Ozymandias




Posts : 622
Join date : 2020-11-23

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptySat May 27, 2023 7:57 pm

Better to have that info in the list itself.

Lars Sandin wrote:
We are using the 3060Ti graphic card on a 1800X-machine and use 2 threads on the 1800X, but as Dio already mentioned in his informative post, it is relatively unimportant as it is the 3060Ti which do most of the work when testing Lc0 with a dedicated graphic card.

Most of the work, but not all. If you plug it into one of the Q6600 machines, you'll measure a drop in performance.

Ghppn likes this post

Back to top Go down
https://centaurchess.boards.net/
Chris Whittington




Posts : 1254
Join date : 2020-11-17
Location : France

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptySat May 27, 2023 8:03 pm

Lars Sandin wrote:
Ozymandias wrote:
SSDF wrote:
All games have been played on the tournament level, 40 moves/2 hours followed by 20 moves/each following hour. In matches between PC-programs, two separate PCs have been used, connected with an auto232-cable.

What I don't know is what books are used with engines which don't have one and what CPU the 3060Ti enjoys.

We try to get some general advice from the programmer in question when we chose which opening book we use. We have a policy which we try to follow for this part (if a program is released without an own book that is). Here are the guidelines which follows our present policy for testing free engines without an own book and which are released without it's own auto232-compatible GUI:

1. The programmer's wish for a suitable opening book is of course the most prefered choice. If the programmer doesn't want to chose or if we can't get in contact with the programmer, we can chose a general book in this case.
1.1 The opening book must be available for free. We are not allowing private books which are not available for the public.
1.2 As we test all free engines in the freely availble GUI: Arena (3.5.1) - the opening book chosen must be of either .abk or .bin type (or internal that can be accessed by the program). We can't use a .ctg book because that forces us to use a commercial GUI to test the free engine. Someone that want to rerun our tests of a freely available engine should be able to do so without any extra cost.
1.3 The opening book chosen should not have been specifically made for any other engine than the one we test. It's therefore perfectly acceptable to use a dedicated opening book made by someone for a specific engine (like when we test SF), but in that case the book is not suitable for testing any other engine. It must also adhere to 1.1 and 1.2 of course.  

Any typo or something missing from the above could be blamed on my memory, as the above mentioned guidelines are directly taken from memory and not from a formerly written prospect.

We are using the 3060Ti graphic card on a 1800X-machine and use 2 threads on the 1800X, but as Dio already mentioned in his informative post, it is relatively unimportant as it is the 3060Ti which do most of the work when testing Lc0 with a dedicated graphic card.

I don't really know what book mine uses (will use), all I know is that it's from Ed, called book.bin and is possibly quite widely used, but I've no idea by whom (that's the problem with publicly available books). It has depth and randomiser settings which are arguably unique. Not sure if that would pass your rules?

Mclane and Ghppn like this post

Back to top Go down
Dio




Posts : 214
Join date : 2021-08-28

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptySat May 27, 2023 8:11 pm

I think that only a few people are aware of how much time it takes to play games at this time control (and sometimes, when using board computers, to enter the moves by hand). Thank you very much for your work.

Mclane, Ghppn and Lars Sandin like this post

Back to top Go down
Dio




Posts : 214
Join date : 2021-08-28

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptySat May 27, 2023 8:17 pm

@ozymandias: here I would like to see facts and not "opinions". I have been testing Lc0 for 5 years, both on GPU and CPU only.

Mclane and Ghppn like this post

Back to top Go down
Ozymandias




Posts : 622
Join date : 2020-11-23

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptySat May 27, 2023 9:44 pm

I didn't say how much of a drop. Are you saying there wouldn't be any?

Ghppn likes this post

Back to top Go down
https://centaurchess.boards.net/
Lars Sandin




Posts : 11
Join date : 2023-05-24

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptySun May 28, 2023 11:18 am

Ozymandias wrote:
Better to have that info in the list itself.

Lars Sandin wrote:
We are using the 3060Ti graphic card on a 1800X-machine and use 2 threads on the 1800X, but as Dio already mentioned in his informative post, it is relatively unimportant as it is the 3060Ti which do most of the work when testing Lc0 with a dedicated graphic card.

Most of the work, but not all. If you plug it into one of the Q6600 machines, you'll measure a drop in performance.

It would of course be the best to have it included in the list, but it isn't possible without footnotes or some kind of hyperlinks. It could be achieved, but it would demand some effort so it would have to be on the "to-do-list" for now, as I have many other things on the same "to-do-list" which aren't fixed. But it is a interesting thought and well worth it I guess!

I never imagined to have the 3060Ti in a Q6600-machine or any of my older machines, but in my mind I silently thought about CPU:s similar to the 1800X and newer ones - and there I would guess that the GPU is the main source of the strength behind the Lc0. I haven't made any tests though, so I don't know for certain how much difference it will make.

Ghppn likes this post

Back to top Go down
Lars Sandin




Posts : 11
Join date : 2023-05-24

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptySun May 28, 2023 11:30 am

Chris Whittington wrote:
Lars Sandin wrote:
Ozymandias wrote:
SSDF wrote:
All games have been played on the tournament level, 40 moves/2 hours followed by 20 moves/each following hour. In matches between PC-programs, two separate PCs have been used, connected with an auto232-cable.

What I don't know is what books are used with engines which don't have one and what CPU the 3060Ti enjoys.

We try to get some general advice from the programmer in question when we chose which opening book we use. We have a policy which we try to follow for this part (if a program is released without an own book that is). Here are the guidelines which follows our present policy for testing free engines without an own book and which are released without it's own auto232-compatible GUI:

1. The programmer's wish for a suitable opening book is of course the most prefered choice. If the programmer doesn't want to chose or if we can't get in contact with the programmer, we can chose a general book in this case.
1.1 The opening book must be available for free. We are not allowing private books which are not available for the public.
1.2 As we test all free engines in the freely availble GUI: Arena (3.5.1) - the opening book chosen must be of either .abk or .bin type (or internal that can be accessed by the program). We can't use a .ctg book because that forces us to use a commercial GUI to test the free engine. Someone that want to rerun our tests of a freely available engine should be able to do so without any extra cost.
1.3 The opening book chosen should not have been specifically made for any other engine than the one we test. It's therefore perfectly acceptable to use a dedicated opening book made by someone for a specific engine (like when we test SF), but in that case the book is not suitable for testing any other engine. It must also adhere to 1.1 and 1.2 of course.  

Any typo or something missing from the above could be blamed on my memory, as the above mentioned guidelines are directly taken from memory and not from a formerly written prospect.

We are using the 3060Ti graphic card on a 1800X-machine and use 2 threads on the 1800X, but as Dio already mentioned in his informative post, it is relatively unimportant as it is the 3060Ti which do most of the work when testing Lc0 with a dedicated graphic card.

I don't really know what book mine uses (will use), all I know is that it's from Ed, called book.bin and is possibly quite widely used, but I've no idea by whom (that's the problem with publicly available books). It has depth and randomiser settings which are arguably unique. Not sure if that would pass your rules?

It isn't a problem just because it is widely used as long as it is available for anyone to get/use (often this is the case when the book is included in the package so to speak). As long as the opening book isn't a verbatim copy of some other programs main opening book - like for instance if one copies the dedicated opening book for Arasan or some other program which include its own dedicated opening book - it would be ok. We have for instance used Sedat Canbaz's: Perfect books, for many programs tested in the rating list without any problem. It is of course fun when there are some variations between books, as part of our testing is to test the combination of a dedicated opening book+engine. There are other rating lists that test without opening books and/or with a standard opening book that everyone use, so it's always nice for us when testing different opening books with possibly different ideas/lines.

Ghppn likes this post

Back to top Go down
Ozymandias




Posts : 622
Join date : 2020-11-23

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptySun May 28, 2023 8:53 pm

Lars Sandin wrote:
Ozymandias wrote:
Better to have that info in the list itself.

It would of course be the best to have it included in the list, but it isn't possible without footnotes or some kind of hyperlinks. It could be achieved, but it would demand some effort so it would have to be on the "to-do-list" for now, as I have many other things on the same "to-do-list" which aren't fixed. But it is a interesting thought and well worth it I guess!

I understand there's a lot of books out there, and that succinctly specifying which one you're using may not be feasible. Since you don't want the list to include long lines, an option could be to just add an alias for the book to the engine name in the list, as if it were a parameter, or a network. Then you can write notes outside the list, explaining what version of the book it is, where it can be found and any other useful info you may think of.

Ghppn likes this post

Back to top Go down
https://centaurchess.boards.net/
Admin
Admin
Admin


Posts : 2528
Join date : 2020-11-17
Location : Netherlands

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptySun May 28, 2023 9:30 pm

Since day one the SSDF tested engines with books included by the programmer. It's something unique they have.

Mclane, Ghppn and Lars Sandin like this post

Back to top Go down
http://rebel13.nl/
Ozymandias




Posts : 622
Join date : 2020-11-23

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptySun May 28, 2023 10:26 pm

I know, I've been following the list for over 15 years, but there were no free notable programs without a book, back then. Now the two main contenders are left to play with unknown books.

Ghppn likes this post

Back to top Go down
https://centaurchess.boards.net/
Chris Whittington




Posts : 1254
Join date : 2020-11-17
Location : France

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptyMon May 29, 2023 12:36 am

Ozymandias wrote:
I know, I've been following the list for over 15 years, but there were no free notable programs without a book, back then. Now the two main contenders are left to play with unknown books.


NN’s ought not to need a book, their moves should be better than human knowledge. Only real reason for book nowadays is variance

Mclane and Ghppn like this post

Back to top Go down
Ozymandias




Posts : 622
Join date : 2020-11-23

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptyMon May 29, 2023 11:33 am

Variance is right. Lower draw ratios if the book contains unbalanced positions is something testers and developers look for, too.
Back to top Go down
https://centaurchess.boards.net/
mwyoung

mwyoung


Posts : 880
Join date : 2020-11-25
Location : USA

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptyMon May 29, 2023 5:01 pm

Ozymandias wrote:
Better to have that info in the list itself.

Lars Sandin wrote:
We are using the 3060Ti graphic card on a 1800X-machine and use 2 threads on the 1800X, but as Dio already mentioned in his informative post, it is relatively unimportant as it is the 3060Ti which do most of the work when testing Lc0 with a dedicated graphic card.

Most of the work, but not all. If you plug it into one of the Q6600 machines, you'll measure a drop in performance.

Q6600 is a very old CPU. I have not used that CPU in many years.

I have been using Lc0 since Lc0 have been out. And I could used only 1 thread without any issues.

If you look at the task manage and CPU usage. You can see if any CPU need more threads with Lc0 + GPU.

If you are only using say 6% CPU usage with one thread running Lc0. And you add more threads to Lc0, and it still reads the same CPU usage. Then you do not need any more threads for Lc0. You will gain nothing.
Back to top Go down
Uri Blass




Posts : 207
Join date : 2020-11-28

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptyMon May 29, 2023 8:59 pm

Chris Whittington wrote:
Ozymandias wrote:
I know, I've been following the list for over 15 years, but there were no free notable programs without a book, back then. Now the two main contenders are left to play with unknown books.


NN’s ought not to need a book, their moves should be better than human knowledge. Only real reason for book nowadays is variance

Books can help to save time on the clock.
Also if you want to get first place in the SSDF rating list then
when you play many games against the same oponnent it is better to learn to repeat lines that you won games with them and avoid lines that you did not win with them.
Back to top Go down
Uri Blass




Posts : 207
Join date : 2020-11-28

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptyMon May 29, 2023 9:10 pm

Lars Sandin wrote:
Code:
THE SSDF RATING LIST 2023-05-24*   162991 games played by  435 computers
                                           Rating   +  -  Games   Won  Oppo
                                           ------  ---   --- -----   ---  ----
   1 Lc0 0.29.0 Cuda-808544 3060Ti           3610   45   -41   287   68%  3466
   2 Lc0 0.28.2 Cuda-611213 3060Ti           3571   26   -25   780   66%  3456
   3 Lc0 0.26.3 Cuda(67362) 3060Ti           3570   29   -27   680   68%  3441
   4 Stockfish 15 x64 1800X 3.6 GHz          3570   39   -36   367   66%  3456
   5 Stockfish 13 x64 1800X 3.6 GHz          3568   35   -32   480   69%  3432
   6 Dragon Komodo 3 x64 1800X 3.6 GHz       3567   40   -38   329   62%  3482
   7 Dragon Komodo 2.51 x64 1800X 3.6 GHz    3565   42   -39   320   65%  3454
   8 Stockfish 14 x64 1800X 3.6 GHz          3555   37   -35   400   65%  3452
   9 Stockfish 12 NNUE x64 1800X 3.6 GHz     3549   26   -25   760   60%  3479
  10 Rebel 16.2 x64 1800X 3.6 GHz            3544   48   -47   209   53%  3522
  11 Dragon by Komodo x64 1800X 3.6 GHz      3536   34   -32   460   63%  3443
  12 Stockfish 11 x64 1800X 3.6 GHz          3529   36   -34   450   70%  3386
  13 Arasan 23.4 x64 1800X 3.6 GHz           3524   54   -55   160   47%  3560
  14 Dragon Komodo 3.1 MCTS 1800X 3.6 GHz    3514   34   -33   440   60%  3447
  15 Stockfish 10 x64 1800X 3.6 GHz          3501   25   -23   920   67%  3378
  16 Booot 7 x64 1800X 3.6 GHz               3477   39   -40   300   46%  3507
  17 Dragon Komodo 2 MCTS x64 1800X 3.6 GHz  3473   45   -44   240   55%  3439
  18 Stockfish 9 x64 1800X 3.6 GHz           3464   24   -23   962   67%  3345
  19 Komodo 14 x64 1800X 3.6 GHz             3455   30   -30   520   52%  3441
  20 Arasan 23.01 x64 1800X 3.6 GHz          3449   33   -34   420   48%  3462
  21 Arasan 23.4 x64 Q6600 2.4 GHz           3439   40   -37   361   68%  3314
  22 Pedone 3.1 x64 1800X 3.6 GHz            3433   38   -39   320   45%  3468
  23 Wasp 6.5 x64 1800X 3.6 GHz              3433   44   -45   242   44%  3478
  24 Stockfish 9 x64 Q6600 2.4 GHz           3427   32   -31   480   56%  3384
  25 Booot 6.5 x64 1800X 3.6 GHz             3426   34   -34   400   48%  3436
  26 Wasp 5.5 x64 1800X 3.6 GHz              3378   37   -39   360   36%  3486
  27 Arasan 22.3 x64 1800X 3.6 GHz           3360   39   -42   320   35%  3468
  28 Marvin 6.0.0 x64 1800X 3.6 GHz          3358   47   -53   220   31%  3494
  29 Pedone 3.1 x64 Q6600 2.4 GHZ            3356   38   -37   340   58%  3303
  30 Deep Shredder 13 x64 1800X 3.6 GHz      3347   24   -23   920   62%  3261
  31 Chiron 5.01 x64 1800X 3.6 GHz           3344   33   -35   440   34%  3455
  32 Marvin 5.2.0 x64 1800X 3.6 GHz          3325   37   -40   360   32%  3456
  33 Chiron 5.01 x64 Q6600 2.4 GHz           3318   34   -34   400   52%  3301
  34 Vajolet2 2.8 x64 1800X 3.6 GHz          3279   27   -28   652   38%  3364
  35 Deep Hiarcs 14 1800X 3.6 GHz            3206   23   -24   880   38%  3289
  36 Revelation 2 A.E. Komodo 12.1 A7 1 GHz  3177   73   -68   100   59%  3113
  37 Revelation 2 Hiarcs 14.1 PXA320 800 MHz 2924   47   -45   228   56%  2881
  38 Chessmaster King 3.5 x64 Q6600 2.4 GHz  2857   24   -25   932   30%  3005
  39 Revelation Hiarcs 13.3 PXA255 500 MHz   2772   57   -52   177   66%  2660
  40 Revelation Shredder 12 PXA255 500 MHz   2706   60   -58   140   56%  2666
  41 Revelation Rybka 2.2 PXA255 500 MHz     2635   45   -43   260   63%  2546
  42 Revelation Deep Sjeng 3 PXA255 500 MHz  2600   62   -67   120   41%  2665
  43 Millennium The King Exclusive 300 MHz   2534   49   -48   200   55%  2501
  44 Revelation Ruffian 2.1 PXA255 500 MHz   2346   68   -71   100   45%  2385
  45 Millennium ChessGenius Excl. M7 300 MHz 2249   50   -48   204   58%  2193
  46 Mephisto London 68030 33 MHz            2195   31   -31   482   50%  2192
  47 Millennium ChessGenius Pro M4 120 MHz   2163   59   -54   160   63%  2067
  48 Mephisto London 68020 12 MHz            2089   60   -53   171   68%  1957
  49 Millennium ChessGenius ARM M4 48 MHz    2070   45   -43   251   58%  2015
  50 Mephisto Roma II 68000 10 MHz           1898   63   -59   132   58%  1843

 1 Lc0 0.29.0 Cuda-808544 3060Ti, 3610
Stoc15 1800X       4-3     Reb162 1800X    21,5-18,5  Aras234 1800X     46-34
Stock9 1800X      29-11    Wasp65 1800X    28,5-11,5  Wasp55 1800X      66-14

 10 Rebel 16.2 x64 1800X 3.6 GHz, 3544
Lc029 3060Ti    18,5-21,5  Dra3 1800X       4,5-4,5   Stoc14 1800X      19-21
Aras234 1800X   20,5-19,5  Boo7 1800X      23,5-16,5  Wasp65 1800X      24-16

 13 Arasan 23.4 x64 1800X 3.6 GHz, 3524
Lc029 3060Ti      34-46    Reb162 1800X    19,5-20,5  Boo7 1800X        21-19

 21 Arasan 23.4 x64 Q6600 2.4 GHz, 3439
Komo12 Q6600    21,5-18,5  Stock8 Q6600    21,5-18,5  Ped31 Q6600     22,5-17,5
Chi5 Q6600      85,5-34,5  DShre13 Q6600     24-16    Vaj28 Q6600        1-0
Mar51 Q6600     33,5-6,5   DRybka3 Q6600   34,5-5,5  

 23 Wasp 6.5 x64 1800X 3.6 GHz, 3433
Lc029 3060Ti    11,5-28,5  Stoc15 1800X    11,5-28,5  Reb162 1800X      16-24
Ped3 1800X      44,5-35,5  Komo11 1800X    21,5-18,5  Vaj28 1800X        2-0

 36 Revelation 2 A.E. Komodo 12.1 A7 1 GHz, 3177
DShre13 Q6600      6-14    DHiar14 Q6600    9,5-10,5  DJuni12 Q6600   11,5-8,5
Baro343 Q6600   15,5-4,5   Naum 4  A1200   16,5-3,5  

Since the last rating list we can now present six new entrants in the list.

First one out is our new leader in the list: Lc0 0.29.0, and also the first
program that has broken the 3600-barrier in our rating list. More games will
be needed to lower the error bars, but after the first 297 games it has placed
itself 39 points over the last version tested with a huge rating of 3610!
It will be interesting to see how it will hold up as more games are played
against the top programs.
The Lc0-version which we have tested this time is: Lc0 0.29.0, with the
19 filter + 512 block network, named 808544 (date: 2023-01-12). As with the
earlier Lc0-versions, we have used the free opening book "perfect2021.abk"
by Sedat Canbaz for the testing of Lc0.

Our next newcomer in the list are one really strong comeback from the team of
Ed Schroeder and Chris Wittington. Chris Wittington is formely known for
programs such as: Chess Player 2150/2175, Complete Chess System
and Chess System Tal (I and II).
Ed Schroeder is of course known for many of the old dedicated chess computers
like for instance: Mephisto Polgar, MM IV/V, RISC (I and II), Rebell
and also for versions of his program in the ChessMachine ISA-plugin card for PC.
On the PC-scene he has made the Gideon and Rebel-program famous and lately also
made several interesting and strong versions of his ProDeo-program.

Since last year they have utilized Neural Net-technology in their joint
effort and has seemingly improved fast. We have waited some time to test it,
since we wanted it to share most of the functions - like for instance Pondering
and Multi-Processor functionality, which the others programs that we test on our
latest hardware uses. The version which we have tested is named: Rebel 16.2 and
we have used its own opening book, made by the famous Jeroen Noomen, for the
testing. After the first 209 games, this strong combo of the program: Rebel 16.2,
and the opening book of Jeroen Noomen, has managed to get an impressive rating
of 3544 on our 1800X hardware. It is placed 10th in the list at the moment,
just 5 points shy of the Stockfish 12-program.
A very strong comeback by Ed, Chris and Jeroen!

Our next addition to the list are Jon Dart's strong and reliable Arasan-program.
The version that we have tested on our 1800X and Q6600 hardware are Arasan 23.4.
On our 1800X hardware it has reached a rating of 3524 after the initial 160
games played. That is at the moment 75 points ahead of the formerly tested 23.01
version and a very nice improvement indeed! More games will of course be needed
to lower the error bars a bit.
On our Q6600 hardware, Arasan 23.4 has reached a rating of 3439 after 361 games
played. We never tested the 23.01 version on the Q6600 hardware, but Arasan 23.4
are a whopping 202 points ahead of the 21.2 version at least!
We have, as always, used Arasan's own opening book for the testing.                  

We are also glad to be able to present a new strong Wasp version by John Stanback,
namely Wasp 6.5, on our latest rating list. After the first 242 games played, it
has reached a rating of 3433! This is 55 points stronger than the 5.5 version
which we tested before, so a nice improvement there also! We have used Wasp's own
opening book for the testing.

And we can now finally show the first rating of the manually tested:
Phoenix Revelation II Anniversary Edition chess computer by Ruud Martin/DGT. This
Anniversary Edition of the Revelation II, features an updated hardware relative
to the ordinary Revelation II. The hardware for the Anniversary Edition is an
Colibri iMX7D with 2 ARM Cortex A7 processors running at 1GHz. The software which
we have used for the testing is the program Komodo 12.1 by Mark Lefler and
Larry Kaufman.
The rating that has been reached after the first 100 games is 3177! This is a
rating that is 253 points ahead of Hiarcs 14.1, which we tested formerly
on the ordinary Revelation II. It is clearly the strongest dedicated chess
computer which we have tested so far and it has been able to perform admirably
against the strong Q6600 opposition which it mostly has faced in our testing
so far.

Aside from producing more games with the already mentioned newcomers in the
rating list, we are at the moment testing Richard Pijl's - The Baron 3.45,
and also Alex Morozov's - Booot 7.1. We also hope to have some more programs
ready for the next rating list.  

* This rating list was produced and based on the games/results that were finished
on the first of May, but the publication of this rating list was unfortunately a
bit delayed this time.
The testing has of course not stood still since then, and the games/results
that has been played since this date will of course feature in the next rating
list, which we plan to have out sometime in august/september.

Lars Sandin

I see no games and I wonder if the leader only won or drew and did not lose a single game.
I expect that at some point the SSDF leaders are not going to lose a single game when the only difference between them is going to be in the result that they can beat weaker opponents.
Back to top Go down
Peter Berger




Posts : 120
Join date : 2020-11-20

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptyTue May 30, 2023 12:56 am

Chris Whittington wrote:

NN’s ought not to need a book, their moves should be better than human knowledge. Only real reason for book nowadays is variance

I used to think that, too. I have since changed my mind.

In like the first 15 moves of the game humans (preferrably with a lot of  help of computers) are still superior to engines. Look at the nonsense Stockfish comes up with when thinking for a very long time completely on its own - this Berlin game Zukertort-Steinitz, that is clearly just an easy draw.

The cumulated knowledge of mankind is still worth something. The NN engines come pretty close, but they are still lacking.

Once both got some pieces out, humans are toast. But it is still definitely possible to improve on the opening play of NN engines.
Back to top Go down
Lars Sandin




Posts : 11
Join date : 2023-05-24

SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 EmptyTue May 30, 2023 8:41 am

Ozymandias wrote:
Lars Sandin wrote:
Ozymandias wrote:
Better to have that info in the list itself.

It would of course be the best to have it included in the list, but it isn't possible without footnotes or some kind of hyperlinks. It could be achieved, but it would demand some effort so it would have to be on the "to-do-list" for now, as I have many other things on the same "to-do-list" which aren't fixed. But it is a interesting thought and well worth it I guess!

I understand there's a lot of books out there, and that succinctly specifying which one you're using may not be feasible. Since you don't want the list to include long lines, an option could be to just add an alias for the book to the engine name in the list, as if it were a parameter, or a network. Then you can write notes outside the list, explaining what version of the book it is, where it can be found and any other useful info you may think of.

I would rather not having the book info in the name because of some reasons:

* The information in the name should be short (due to space constraint), as easy and informative as possible.
* An alias/parameter in f.e. character- (AA) or numberformat (01) would mean little for the person that isn't actively seeking up what it stands for. It adds more complexity for very little gain.
* We already have some rather redundant information in the names which would be more acute to get rid of (before adding more info that is). The designation x64 is nowadays rather pointless, since nearly every program that we have tested on our Q6600 and 1800X-hardware has been supporting 64 bit on these two platforms (both having 64 bit OS's). It would have been better to exclude x64 now, and add 32-bit to the few that hasn't had support for the standard 64-bit OS.
* Other than that I should go through the naming for each and every of the 435 computers and try to standardize them, so that most of them are following the same standard. Not every computer in the list does this today. This would clean up the list a bit.

But I like your general idea, as it would be helpful for me also. I have most used books in memory, but this info isn't that easy to access for other than myself I guess. Very Happy But it demands some work and as I mentioned earlier, I would rather simplify things (first at least) than to make everything more complex than it already is.
Back to top Go down
Sponsored content





SSDF Rating List 23-05-24 Empty
PostSubject: Re: SSDF Rating List 23-05-24   SSDF Rating List 23-05-24 Empty

Back to top Go down
 
SSDF Rating List 23-05-24
Back to top 
Page 1 of 2Go to page : 1, 2  Next
 Similar topics
-
» New Rating List ?
» Speedy rating list
» First Gambit Rating List
» Rating List Experiment
» Gambit Rating List - May 20, 2021

Permissions in this forum:You cannot reply to topics in this forum
ProDeo :: Computer Chess-
Jump to: