ProDeo
Would you like to react to this message? Create an account in a few clicks or log in to continue.
ProDeo

Computer Chess
 
HomeHome  CalendarCalendar  FAQFAQ  SearchSearch  MemberlistMemberlist  UsergroupsUsergroups  RegisterRegister  Log in  

 

 Gambit Tournament, first results available

Go down 
4 posters
AuthorMessage
Admin
Admin
Admin


Posts : 1158
Join date : 2020-11-17
Location : Netherlands

Gambit Tournament, first results available Empty
PostSubject: Gambit Tournament, first results available   Gambit Tournament, first results available EmptyFri Apr 23, 2021 10:48 am

Ratings of chess engines are well established on the known rating lists. Will anything change when we feed engines with only (playable) gambit positions?

We carefully created 50 gambit opening positions to find out.

With 6 top entries that means 500 games each. First run at 40/2 ccrl style as a warming up followed by the real thing 40/15 ccrl style.

In the 40/15 tournament Lc0 v27 (default net) will participate also but it will play at 40/2 to overcome the GPU hardware advantage.

....

http://rebel13.nl/misc/gambit-tournament.html

Mclane, mwyoung and polotoprak like this post

Back to top Go down
View user profile http://rebel13.nl/
Admin
Admin
Admin


Posts : 1158
Join date : 2020-11-17
Location : Netherlands

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptyFri Apr 23, 2021 11:13 pm

The mighty Dragon moved in. The gap between SF, Komodo and the rest of the pack is amazing.

Meanwhile I have given up on Lc0, it doesn't work well enough using multiple entries and running 700 games single threaded at 40/2 takes too long to my taste. Pity because I believe in these kind of positions Lc0 can beat the almighty Stockfish.
Back to top Go down
View user profile http://rebel13.nl/
Admin
Admin
Admin


Posts : 1158
Join date : 2020-11-17
Location : Netherlands

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptySat Apr 24, 2021 8:48 am

40 moves in 15 minutes
Tournament is expected to run for 5-7 days, there will be an update each day about its progress.
Update after 198 (of 1500) games are played
Code:
No. Engine               1     2     3     4     5     6  Score  Games Perc    Moves
------------------------------------------------------------------------------------
 1 Stockfish 13       xxxx   6.5  10.0  10.5  10.5  12.0   49.5 / 66  (75.00%)  71.7  
 2 Komodo-Dragon       6.5  xxxx   9.0  10.0  10.5  10.5   46.5 / 66  (70.45%)  75.2  
 3 Ethereal_12.75      3.0   4.0  xxxx   7.0   5.0   7.5   26.5 / 66  (40.15%)  74.9  
 4 Igel_3.0.0          3.5   3.0   6.0  xxxx   7.5   5.5   25.5 / 66  (38.64%)  70.7  
 5 RubiChess_2.1       2.5   3.5   8.0   5.5  xxxx   6.0   25.5 / 66  (38.64%)  70.9  
 6 SlowChess_2.5       1.0   2.5   6.5   7.5   7.0  xxxx   24.5 / 66  (37.12%)  65.7  
Back to top Go down
View user profile http://rebel13.nl/
Admin
Admin
Admin


Posts : 1158
Join date : 2020-11-17
Location : Netherlands

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptySun Apr 25, 2021 10:04 am

Code:
  PLAYER            :  RATING  POINTS  PLAYED   (%)    W    D    L  D(%)  OppAvg
1 Stockfish 13      :  3657.6   483.0     600    81  375  216    9    36  3396.7
2 Komodo Dragon     :  3572.2   421.5     600    70  294  255   51    43  3411.0
3 Komodo 14         :  3425.6   296.0     600    49  165  262  173    44  3435.4
4 RubiChess 2.1     :  3363.0   240.0     600    40  110  260  230    43  3445.8
5 SlowChess 2.5     :  3353.3   231.5     600    39  101  261  238    44  3447.4
6 Ethereal 12.75    :  3341.3   221.0     600    37  101  240  259    40  3449.5
7 Igel 3.0.0        :  3325.0   207.0     600    35   70  274  256    46  3452.2

In the 40/2 edition SF lost only 9 games!

An overall lesser draw rate also.
Back to top Go down
View user profile http://rebel13.nl/
Admin
Admin
Admin


Posts : 1158
Join date : 2020-11-17
Location : Netherlands

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptySun Apr 25, 2021 11:03 am

I have decided Lc0 plays the winner of the 40/2 tournament, thus sf13 -Lc0-v27, the 100 gambit games which is doable within an acceptable time, estimated 2 days.

I tested the speed of 3 nets:
128x10 - nps 20,000
256x20 - nps 4,000
384x30 - nps 1,200

I have chosen for the 256 net.
Back to top Go down
View user profile http://rebel13.nl/
Admin
Admin
Admin


Posts : 1158
Join date : 2020-11-17
Location : Netherlands

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptyTue Apr 27, 2021 11:21 am

New set up -

Ratings of chess engines are well established on the known rating lists. Will anything change when we feed engines with only (playable) gambit positions? We carefully created 50 gambit opening positions to find out.

As first step following the established ratings on the CCRL rating list we divide (only active) engines into groups based on their strength and run a robin match. Thus a group of 6 engines will play 500 games each and 1500 games in total. Two time controls CCRL style, 40/2 and 40/15.

As second step with a reasonable number of engines and games we will create a rating list. Games will be available for download when we have created a second set of 50 gambit positions.

Excluded Engines
1. Inactive engines, those engines who haven't been updated for a long time.
2. Known derivatives.

http://rebel13.nl/misc/gambit-tournament.html
Back to top Go down
View user profile http://rebel13.nl/
polotoprak




Posts : 13
Join date : 2021-04-28

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptyWed Apr 28, 2021 11:07 pm

Very interesting setup, the results without surprise to anyone had an expected outcome
Can you upload the 50 gambit book that was used during the tournament please?
Back to top Go down
View user profile
Admin
Admin
Admin


Posts : 1158
Join date : 2020-11-17
Location : Netherlands

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptyThu Apr 29, 2021 10:52 am

polotoprak wrote:
Very interesting setup, the results without surprise to anyone had an expected outcome
Can you upload the 50 gambit book that was used during the tournament please?

I am in the process of creating a rating list from the all the games (pools) which is not a simple task. When finished the games will be available for download.

Meanwhile the new Igel 3.0.5 is running the gauntlet and Lc0 still running for the premier league. So far:

Code:
No. Engine               1     2     3     4     5     6  Score  Games  Perc   Moves
------------------------------------------------------------------------------------
 1 Lc0 v27 RTX-1060   xxxx  32.0  46.5  66.0  70.0  73.0  287.5 / 500 (57.50%)  83.9  
 2 Stockfish 13       68.0  xxxx   0.0   0.0   0.0   0.0   68.0 / 100 (68.00%)  65.1  
 3 Komodo Dragon      53.5   0.0  xxxx   0.0   0.0   0.0   53.5 / 100 (53.50%)  100.7  
 4 Komodo 14          34.0   0.0   0.0  xxxx   0.0   0.0   34.0 / 100 (34.00%)  88.9  
 5 SlowChess 2.5      30.0   0.0   0.0   0.0  xxxx   0.0   30.0 / 100 (30.00%)  79.5  
 6 Ethereal 12.75     27.0   0.0   0.0   0.0   0.0  xxxx   27.0 / 100 (27.00%)  85.5  

It still must play Rubi and Igel and it's not unlikely Lc0 is going to win the tournament while it has lost badly against SF13. The others with the exception of the Dragon are crushed by Lc0. Point of attention (criticism) is the hardware advantage of Lc0. On the RTX-1600 playing with a 256 net Lc0 does  ±4000 NPS.

polotoprak likes this post

Back to top Go down
View user profile http://rebel13.nl/
polotoprak




Posts : 13
Join date : 2021-04-28

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptyThu Apr 29, 2021 2:21 pm

Admin wrote:
polotoprak wrote:
Very interesting setup, the results without surprise to anyone had an expected outcome
Can you upload the 50 gambit book that was used during the tournament please?

It still must play Rubi and Igel and it's not unlikely Lc0 is going to win the tournament while it has lost badly against SF13. The others with the exception of the Dragon are crushed by Lc0. Point of attention (criticism) is the hardware advantage of Lc0. On the RTX-1600 playing with a 256 net Lc0 does  ±4000 NPS.

I honestly thought Lc0 will match SF13, gambit like positions are Lc0's best time to shine... kinda disappointed by the results, will wait for the final results

Admin likes this post

Back to top Go down
View user profile
Mclane

Mclane


Posts : 1971
Join date : 2020-11-17
Age : 55
Location : United States of Europe, Germany, Ruhr area

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptySat May 01, 2021 5:16 pm

Lc0 is a little stronger then latest stockfish.
If there is a huge gap in a tournament rating list between Stockfish and lc0,
There must be a major glitch.
Either the gpu is too old, or the net is wrong. Or any other influence.

I have a xeon 6 core with amd 540 (4 GB)
Amd phenom II 6 core with nvidia 1060
Acer laptop with i7 quad core with nvidia 1060 (6 GB)
Ryzen 5 3600 6 core with AMD 570 (8 GB)
AMD ryzen 16 core with nvidia 2070 (8GB)
AMD ryzen 3 2200G (built in gpu, appropriate for LC0)

On the ryzen 5 the gpu is a little too weak.
On the ryzen 9 3950x i am using 8 cores so far for stockfish and 2070 is ahead.
I could of course scale this up to see when the 2070 gets beaten with Stockfish getting more cores.

Dragon by komodo makers is mostly arround 2 ranks beneath Stockfish 13 but close.

I do test with permanent brain on and in arena. Arround 5 minutes per game. Normal time control, no blitz time controls. E.g. 40/5 etc.

polotoprak likes this post

Back to top Go down
View user profile http://www.thorstenczub.de
Admin
Admin
Admin


Posts : 1158
Join date : 2020-11-17
Location : Netherlands

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptySat May 01, 2021 5:23 pm

It's better to have a benchmark first before jumping into conclusions. I will start a new subject.

Mclane likes this post

Back to top Go down
View user profile http://rebel13.nl/
polotoprak




Posts : 13
Join date : 2021-04-28

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptySat May 01, 2021 8:10 pm

Hello, today I found very similar Gambit tournament that was done in the early 2016 by Walter Eigenmann.
Using a large Fritz format gambit book the top 10 engines during that period of 2016 battled for the winning spot of the Blitz Tournament.

In the article he shares the Opening gambit book and the completed .pgn game database
if you are interested make sure to check out his article here: https://glarean-verlag.ch/schach/live-schachpartien/
Back to top Go down
View user profile
Admin
Admin
Admin


Posts : 1158
Join date : 2020-11-17
Location : Netherlands

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptySat May 01, 2021 8:45 pm

Ah, the famous Traxler.
Back to top Go down
View user profile http://rebel13.nl/
Admin
Admin
Admin


Posts : 1158
Join date : 2020-11-17
Location : Netherlands

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptyTue May 04, 2021 9:10 pm

Back to top Go down
View user profile http://rebel13.nl/
Chris Whittington
Admin



Posts : 306
Join date : 2020-11-17

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptyTue May 04, 2021 9:27 pm

Admin wrote:
Added live support for 2 PC's.

http://rebel13.nl/misc/gambit-rating-list.html

If you like, there’s probably some time on my most powerful PC to make a strong hardware match.
Same PC, SF can have 64 cores, unlimited RAM, LC0 gets a 3090 GPU. Disable pondering.
Back to top Go down
View user profile
Admin
Admin
Admin


Posts : 1158
Join date : 2020-11-17
Location : Netherlands

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptyTue May 04, 2021 9:50 pm

Interesting Chris, it would mean I first should make the software for live support available. I can do that. I suppose you don't have an own website? Else it must run on mine.
Back to top Go down
View user profile http://rebel13.nl/
Admin
Admin
Admin


Posts : 1158
Join date : 2020-11-17
Location : Netherlands

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptyThu May 06, 2021 12:05 pm

Fun, ProDeo 3.0 on CCRL is 78 elo higher than Benjamin (the gambit version of ProDeo), nevertheless Benjamin rates higher on the gambit list.

Secondly the draw rate is clearly lower than on CCRL, that was to be expected and hoped for.

Code:
   Gambit Rating List  : 2021-05-06
   Time Control        : 40 moves in 2 minutes repeating
   Software            : cutechess-cli 1.1.0.f.1.0
   Elo calculation     : Ordo 1.2.6
   Games               : 18.200
  
   # PLAYER            :  RATING  ERROR  POINTS  PLAYED   (%)     LOS    W    D    L  DRAWS
   1 Stockfish 13      :  3572.7   18.2   724.5     900    81     100  563  323   14   36%
   2 Komodo Dragon     :  3480.5   38.4   629.5     900    70     100  427  405   68   45%
   3 Lc0-v27           :  3429.1   23.0   501.0     800    63     100  307  388  105   49%
   4 Komodo 14         :  3331.3   27.8   443.5     900    49      98  235  417  248   46%
   5 Anchor Engine     :  3300.0   14.3   827.0    1400    59      96  550  554  296   40%
   6 RubiChess 2.1     :  3271.0   32.7   365.0     900    41      52  162  406  332   45%
   7 Pedone 3.1        :  3270.3   26.7   529.0     800    66      67  386  286  128   36%
   8 SlowChess 2.5     :  3263.6   27.8   355.5     900    40      58  148  415  337   46%
   9 Igel 3.0.5        :  3260.7   26.6   307.5     800    38      74  108  399  293   50%
  10 Ethereal 12.75    :  3254.5   21.5   344.0     900    38      77  156  376  368   42%
  11 Igel 3.0.0        :  3242.2   28.1   328.5     900    37      98  111  435  354   48%
  12 Nemorino 6.00     :  3211.3   22.0   461.0     800    58     100  332  258  210   32%
  13 Pedone 3.0        :  3177.0   22.2   420.0     800    53      69  265  310  225   39%
  14 rofChade 2.3      :  3171.3   25.6   413.5     800    52      56  262  303  235   38%
  15 Booot 6.5         :  3169.7   31.9   411.5     800    51     100  242  339  219   42%
  16 Seer 2.0.1        :  3097.1   27.2   476.0     700    68      95  375  202  123   29%
  17 Anchor Engine     :  3074.9   13.4  1245.5    2200    57      54  907  677  616   31%
  18 Halogen 10        :  3073.5   21.1   484.5     800    61      61  359  251  190   31%
  19 Danasah 8.8       :  3071.3   23.7   411.5     700    59      69  317  189  194   27%
  20 Wasp 4.50         :  3062.1   29.1   284.5     800    36      93  145  279  376   35%
  21 Seer 2.0.0        :  3029.8   28.9   552.0     900    61      52  425  254  221   28%
  22 Weiss 1.3         :  3028.7   26.5   592.0    1000    59      63  434  316  250   32%
  23 Minic 3.04        :  3023.3   19.3   381.0     700    54      71  272  218  210   31%
  24 Berserk 3.3.0     :  3012.8   26.3   491.5     700    70      53  407  169  124   24%
  25 Berserk 4.0.0     :  3011.5   22.6   446.5     800    56      54  298  297  205   37%
  26 Arasan 2.22       :  3009.4   37.5   227.5     800    28      63  117  221  462   28%
  27 Tucano 9.0        :  3001.5   18.5   644.0    1000    64      59  532  224  244   22%
  28 Marvin 5.0        :  2999.3   12.3   354.5     700    51      73  232  245  223   35%
  29 Mr Bob 1.0.0      :  2992.1   17.9   631.5    1000    63      90  522  219  259   22%
  30 Orion 0.8         :  2979.6   12.2   332.0     700    47      56  217  230  253   33%
  31 Stash 29.0        :  2977.8   23.1   330.0     700    47      72  217  226  257   32%
  32 Vajolet2 2.8      :  2968.7   20.9   320.0     700    46      87  214  212  274   30%
  33 Topple 0.8.0      :  2951.4   21.2   552.5    1200    46      84  364  377  459   31%
  34 Cheng 4.41        :  2939.2   29.7   469.5    1000    47      59  314  311  375   31%
  35 Counter 3.7       :  2935.8   26.6   525.0    1200    44      53  326  398  476   33%
  36 Seer 1.2.1        :  2934.4   20.5   430.0    1000    43      76  280  300  420   30%
  37 FabChess 1.16     :  2925.9   28.1   478.0    1100    43      91  310  336  454   31%
  38 Drofa 3.0.0       :  2885.6   36.6   357.0     500    71      61  303  108   89   22%
  39 Cheng 4.40        :  2878.4   23.7   369.5    1000    37      62  236  267  497   27%
  40 Amoeba 3.3        :  2872.3   30.8   218.5     700    31      88  122  193  385   28%
  41 Anchor Engine     :  2850.0   14.3   482.5    1300    37      90  309  347  644   27%
  42 Cheese 2.2        :  2839.3   17.0   370.5     800    46      81  272  197  331   25%
  43 Olithink 5.9.3    :  2823.6   22.6   278.0     600    46      91  209  138  253   23%
  44 ProDeo 3.1        :  2804.1   16.7   329.5     800    41      79  238  183  379   23%
  45 Velvet 1.2.0      :  2781.7   44.2   342.5     500    69      99  296   93  111   19%
  46 Benjamin          :  2754.3   45.2   720.0    1200    60      58  610  220  370   18%
  47 ProDeo 3.0        :  2749.0   23.5   267.5     800    33      71  186  163  451   20%
  48 k2 0.99           :  2735.1   36.1   377.5     700    54      90  294  167  239   24%
  49 Dumb 1.8          :  2718.9   43.5   396.5     800    50      72  308  177  315   22%
  50 Anchor Engine     :  2710.3   26.1   485.5    1200    40      77  368  235  597   20%
  51 Supernova 2.3     :  2699.9   42.2   623.0    1200    52      68  501  244  455   20%
  52 Fridolin 3.10     :  2695.4   37.1   615.0    1200    51     100  495  240  465   20%
  53 Galjoen 0.41.1    :  2543.0   39.9   164.5     500    33     100  120   89  291   18%
  54 CT800 1.42        :  2438.1   41.9    97.0     500    19      96   65   64  371   13%
  55 Foxsee 7.8        :  2407.6   49.5    85.0     600    14     ---   44   82  474   14%

White advantage = -13.69 +/- 1.96
Draw rate (equal opponents) = 34.24 % +/- 0.27
Back to top Go down
View user profile http://rebel13.nl/
polotoprak




Posts : 13
Join date : 2021-04-28

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptyFri May 07, 2021 4:15 pm

Admin wrote:
Fun, ProDeo 3.0 on CCRL is 78 elo higher than Benjamin (the gambit version of ProDeo), nevertheless Benjamin rates higher on the gambit list.

Secondly the draw rate is clearly lower than on CCRL, that was to be expected and hoped for.

True but Benjamin played 400 games more than ProDeo 3.0, maybe that has something to do with the 8 ELO difference in favor of Benjamin. I am happy to see Benjamin in front of ProDeo 3 when it comes to aggressive gambit postions, I've seen some really fun games with Benjamin
Back to top Go down
View user profile
Admin
Admin
Admin


Posts : 1158
Join date : 2020-11-17
Location : Netherlands

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptySat May 08, 2021 11:28 am

Got web-match (live broadcast eng-eng matches) working again, started Benjamin - ProDeo 3.0 to find out.

http://rebel13.nl/pgn4web-3.05/live-test.html

100 gambit games and Benjamin won the first one Very Happy
Back to top Go down
View user profile http://rebel13.nl/
Admin
Admin
Admin


Posts : 1158
Join date : 2020-11-17
Location : Netherlands

Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available EmptySat May 08, 2021 9:45 pm

Code:

                            Result Table

     Name        Gms  :  Win  : Draw  : Lose  :  Pts   :    S-B

ProDeo 3.0       100  :  44+  :  28=  :  28-  :  58.0  :  2436.00
Benjamin         100  :  28+  :  28=  :  44-  :  42.0  :  2436.00

Games played 100, games to go 0


Tournament finished

Sadly ProDeo 3.0 won convincingly.


Back to top Go down
View user profile http://rebel13.nl/
Sponsored content





Gambit Tournament, first results available Empty
PostSubject: Re: Gambit Tournament, first results available   Gambit Tournament, first results available Empty

Back to top Go down
 
Gambit Tournament, first results available
Back to top 
Page 1 of 1
 Similar topics
-
» Gambit Rating List - Gauntlet Results
» Top Chess Engine Tournament Results
» something seem strange with results of the queen odd matches
» Corona Virus Chess 1.011 results so far
» Knight / Bishop / Rook and Queen odds results

Permissions in this forum:You cannot reply to topics in this forum
ProDeo :: Computer Chess-
Jump to: