What were the nodes per move in the Crafty-Rebel NP Challenge?

It may be interesting to make a similiar challange with stockfish against Stockfish.

I will not be surprised if we are going to get today draws in more than 90% of the games.

Following the link https://www.rebel.nl/match.htm the NPS of that time was between 50,000 - 100,000

Admin wrote:

Following the link https://www.rebel.nl/match.htm the NPS of that time was between 50,000 - 100,000

Thanks

I guess that it is something close to 15M nodes per move for Rebel against 1500M nodes per move for Crafty.
I guess that stockfish with 15M nodes per move can draw most games against stockfish with 1500M nodes per move.

Even 500K nodes per move draw often and drew most games so far against 50M nodes per move based on my testing.

50M got only 6 wins with white out of 20 games after 1.a3 1.a4 1.b3 1.b4....

I believe that black has the advantage in this experiement because black is the better side after part of the moves like 1.g4 or 1.f3 but even with black it is not clear that 50M is going to win most of the games(so far it has 2 wins against 1.a4 and 1.b3 and 3 draws against 1.a3 1.b4 1.c3)

It's an interesting idea, at which time odds Sf14 starts to win from Sf14.

If you know how to use cutechess then you can do this easily using multiple games at the same time using the -concurrency option and different time controls for each engine. A test model could be:

run-1 : sf14 (40/10) vs sf14 (40/20)
run-2 : sf14 (40/10) vs sf14 (40/40)
run-3 : sf14 (40/10) vs sf14 (40/80)

run-4 : sf14 (40/20) vs sf14 (40/40)
run-5 : sf14 (40/20) vs sf14 (40/80)
run-6 : sf14 (40/20) vs sf14 (40/160)

Admin wrote:

It's an interesting idea, at which time odds Sf14 starts to win from Sf14.

If you know how to use cutechess then you can do this easily using multiple games at the same time using the -concurrency option and different time controls for each engine. A test model could be:

run-1 : sf14 (40/10) vs sf14 (40/20)
run-2 : sf14 (40/10) vs sf14 (40/40)
run-3 : sf14 (40/10) vs sf14 (40/80)

run-4 : sf14 (40/20) vs sf14 (40/40)
run-5 : sf14 (40/20) vs sf14 (40/80)
run-6 : sf14 (40/20) vs sf14 (40/160)

If I use multiple games at the same time then I guess that I need to use less hash tables for an engine.
I also believe that with time odds except very fast time control I will have almost 100% draws with a normal book(even with 40/20 seconds against 40/160 seconds)

I may try time matches after I finish my 1:100 nodes match and I also consider to try nodes/time matches to see the time control I need to beat 500K nodes per move in most games not only by stockfish but also by different engines.

Note that with 500000 k nodes against 50M nodes per move
White won 6:0 and 14 draws when black is leading 4:0 and 7 draws(probably win for game also in game 12) so it is already clear that I will have at least 21 draws out of 40.

Admin wrote:

It's an interesting idea, at which time odds Sf14 starts to win from Sf14.

If you know how to use cutechess then you can do this easily using multiple games at the same time using the -concurrency option and different time controls for each engine. A test model could be:

run-1 : sf14 (40/10) vs sf14 (40/20)
run-2 : sf14 (40/10) vs sf14 (40/40)
run-3 : sf14 (40/10) vs sf14 (40/80)

run-4 : sf14 (40/20) vs sf14 (40/40)
run-5 : sf14 (40/20) vs sf14 (40/80)
run-6 : sf14 (40/20) vs sf14 (40/160)

I used latest developement version
run 1 is 5 minutes per game+3 seconds per move for white against 30 seconds per game+0.3 seconds per move
Every player got 7 cores.

Result is 20 draws.

I consider getting a draw with 1.g4 to be a good result but I consider failing to win with moves like 1.e4 1.d4 or 1.Nf3 to be a bad result.

result in the second match is 3-0 and 17 draws when black could win 3 games.
black could win against c4 f3 and g4.

Total result is 3-0 and 37 draws out of 40 in 10:1 time handicap match.

Seems your estimations are right. Recently member Mark Young ran a SF14 vs SF14 test to measure the elo progress between one core vs 4, 8 and 16 cores. Also a very high draw rate.

Admin wrote:

Seems your estimations are right. Recently member Mark Young ran a SF14 vs SF14 test to measure the elo progress between one core vs 4, 8 and 16 cores. Also a very high draw rate.

And this goes to show you, and as I keep saying. A high draw ratio does not mean anything. Expect the 2 engines are near the same playing strength. It does not mean we are near perfect chess, or anything else. Others then the 2 engines find it hard to capitalize on the other engines mistakes. And they are many.

mwyoung wrote:

Admin wrote:

Seems your estimations are right. Recently member Mark Young ran a SF14 vs SF14 test to measure the elo progress between one core vs 4, 8 and 16 cores. Also a very high draw rate.

And this goes to show you, and as I keep saying. A high draw ratio does not mean anything. Expect the 2 engines are near the same playing strength. It does not mean we are near perfect chess, or anything else. Others then the 2 engines find it hard to capitalize on the other engines mistakes. And they are many.

My theory is that there are not many mistakes in the meaning of losing moves based on the 32 piece tablebases but that engines simply do not play in a way that cause the opponent to make mistakes.

I can give a simple example for stockfish's stupid moves in endgames that I watched that do not change the theoretical result of the game.
I played the games without tablebases and the following position happened in stockfish-stockfish games


[puzzle]
[SetUp "1"]
[FEN "8/8/4B3/8/2K5/8/2N1k3/6r1 w - - 51 103"]
103. Ne3? Rg6? 104.Kd4? Rxe6 {Long Draw} 1-0
[/puzzle]


After 103.Ne3? Kxe3 cause white to suffer KR vs KB
After 103...Rg6? white has a simple draw by 104.Nf5 or 104.Nd5 but for some reason stockfish chose 104.Kd4? that is also a draw but white needed to suffer 50 moves in KR vs KN endgame.

Uri Blass wrote:

mwyoung wrote:

Admin wrote:

Seems your estimations are right. Recently member Mark Young ran a SF14 vs SF14 test to measure the elo progress between one core vs 4, 8 and 16 cores. Also a very high draw rate.

And this goes to show you, and as I keep saying. A high draw ratio does not mean anything. Expect the 2 engines are near the same playing strength. It does not mean we are near perfect chess, or anything else. Others then the 2 engines find it hard to capitalize on the other engines mistakes. And they are many.

My theory is that there are not many mistakes in the meaning of losing moves based on the 32 piece tablebases but that engines simply do not play in a way that cause the opponent to make mistakes.

I can give a simple example for stockfish's stupid moves in endgames that I watched that do not change the theoretical result of the game.
I played the games without tablebases and the following position happened in stockfish-stockfish games


[puzzle]
[SetUp "1"]
[FEN "8/8/4B3/8/2K5/8/2N1k3/6r1 w - - 51 103"]
103. Ne3? Rg6? 104.Kd4? Rxe6 {Long Draw} 1-0
[/puzzle]


After 103.Ne3? Kxe3 cause white to suffer KR vs KB
After 103...Rg6? white has a simple draw by 104.Nf5 or 104.Nd5 but for some reason stockfish chose 104.Kd4? that is also a draw but white needed to suffer 50 moves in KR vs KN endgame.

I have tested your theory with perfect play. And we do have perfect play for 7 man and under. And the engines are just blind! And if you are mistaken at 7 man and below. How many errors do you think happen with 32 man. A 32 man tablebase would crush the current Stockfish.

Remember you only need to make 1 mistake in a game to lose against perfect play!

Admin wrote:

It's an interesting idea, at which time odds Sf14 starts to win from Sf14.

If you know how to use cutechess then you can do this easily using multiple games at the same time using the -concurrency option and different time controls for each engine. A test model could be:

run-1 : sf14 (40/10) vs sf14 (40/20)
run-2 : sf14 (40/10) vs sf14 (40/40)
run-3 : sf14 (40/10) vs sf14 (40/80)

run-4 : sf14 (40/20) vs sf14 (40/40)
run-5 : sf14 (40/20) vs sf14 (40/80)
run-6 : sf14 (40/20) vs sf14 (40/160)

I do not know why you locked the topic. But thanks for running this test. It answers the question of Cores vs Time odds. Same huge drop off in performance. So it is not thread efficiency.

And we no longer need to upgrade are computers with more cores.


Stockfish Results
Code:
No. Name           Win Draw Loss Unf.  Score Games       %
----------------------------------------------------------
 1 SF14-two-secs +133 =841  -26   *0  553.5  1000   55.4%  +37 elo
 2 SF14-one-sec   +26 =841 -133   *0  446.5  1000   44.6%
Code:
No. Name            Win Draw Loss Unf.  Score Games       %
-----------------------------------------------------------
 1 SF14-four-secs +212 =774  -14   *0  599.0  1000   59.9%  +70 elo
 2 SF14-one-sec    +14 =774 -212   *0  401.0  1000   40.1%
Code:
No. Name             Win Draw Loss Unf.  Score Games       %
------------------------------------------------------------
 1 SF14-eight-secs +132 =361   -7   *0  312.5   500   62.5%  +87 elo
 2 SF14-one-sec      +7 =361 -132   *0  187.5   500   37.5%

I locked it temporarily to keep it clean, when finished it will be open.

Unlocked it, I just will keep the OP up to date.

game 36 is clearly a book win for the 1 second against 16 seconds.