Subject: Is NNUE a dead end? Wed Nov 10, 2021 9:45 pm
Is NNUE a dead end?
I will not use my data, but it shows the exact same thing. Here is CCRL testing data for Stockfish NNUE.
Time control 40 moves in 15 mins, but not really. (Equivalent to 40 moves in 15 minutes on an Intel i7-4770k.)
Stockfish NNUE 4 CPU ratings.
Stockfish 14 64-bit 4CPU 3547 Stockfish 13 64-bit 4CPU 3541 Stockfish 14.1 64-bit 4CPU 3518... You have to love how CCRL always stops testing. If it is not going SF's way. 196 games played. 200 games is official. Stockfish 12 64-bit 4CPU 3512
Despite Stockfish's fish testing results proclaiming major gains. No real progress has been made.
Nezhman likes this post
TheSelfImprover
Posts : 3112 Join date : 2020-11-18
Subject: Re: Is NNUE a dead end? Wed Nov 10, 2021 11:28 pm
mwyoung wrote:
Is NNUE a dead end?
I'm voting "yes".
It's a HUGE leap forward from before, but I think progress will be slow going forward.
Firstly, it would be good to know at what elo level a player becomes "almost unbeatable" (AB). I'm not going to argue about what AB means, though, mainly because my second point is a lot more important.
Secondly, and more importantly: these NN train on billions of positions - many orders of magnitude more than any human will ever see - but at ply 1 they are a lot weaker than top humans. That tells me they're "good", but they're not "it".
mwyoung
Posts : 880 Join date : 2020-11-25 Location : USA
Subject: Re: Is NNUE a dead end? Wed Nov 10, 2021 11:48 pm
TheSelfImprover wrote:
mwyoung wrote:
Is NNUE a dead end?
I'm voting "yes".
It's a HUGE leap forward from before, but I think progress will be slow going forward.
Firstly, it would be good to know at what elo level a player becomes "almost unbeatable" (AB). I'm not going to argue about what AB means, though, mainly because my second point is a lot more important.
Secondly, and more importantly: these NN train on billions of positions - many orders of magnitude more than any human will ever see - but at ply 1 they are a lot weaker than top humans. That tells me they're "good", but they're not "it".
""it would be good to know at what elo level a player becomes "almost unbeatable""
I am sorry you have not understood my previous posting. But this question is not knowable.
As chess is not solvable. And we do not know if chess is either a draw with best play, or a win with best play.
The best we can do is gauge what we have!
mwyoung
Posts : 880 Join date : 2020-11-25 Location : USA
Subject: Re: Is NNUE a dead end? Thu Nov 11, 2021 1:11 am
Secondly, and more importantly: these NN train on billions of positions - many orders of magnitude more than any human will ever see - but at ply 1 they are a lot weaker than top humans.
I must reveal what I know. And this statement is false. Neural networks are not created equally. Lc0's networks are stronger then NNUE networks. And with just 1 node not one ply are better then most human players. As I test everything I can.
If you doubt this take the 1 node Lc0 network challenge. As I have said in the past. And you will see just how strong Lc0 is in a pure network.
Uri Blass
Posts : 207 Join date : 2020-11-28
Subject: Re: Is NNUE a dead end? Thu Nov 11, 2021 8:14 am
TheSelfImprover wrote:
mwyoung wrote:
Is NNUE a dead end?
I'm voting "yes".
It's a HUGE leap forward from before, but I think progress will be slow going forward.
Firstly, it would be good to know at what elo level a player becomes "almost unbeatable" (AB). I'm not going to argue about what AB means, though, mainly because my second point is a lot more important.
Secondly, and more importantly: these NN train on billions of positions - many orders of magnitude more than any human will ever see - but at ply 1 they are a lot weaker than top humans. That tells me they're "good", but they're not "it".
Top humans cannot play at 1 ply and even playing bullet is not the same because I am sure they caluclate in some lines more than 1 ply forward.
Uri Blass
Posts : 207 Join date : 2020-11-28
Subject: Re: Is NNUE a dead end? Thu Nov 11, 2021 8:31 am
I decided to look how stockfish14.1 lost against RubiChess and I suspect bad time management for x minutes/40 moves
Stockfish blundered at move 40 by searching to depth 1 when at move 39 it searched to depth 69 and spent 3 seconds. Not sure if this is a new problem with 14.1
Subject: Re: Is NNUE a dead end? Thu Nov 11, 2021 10:32 am
mwyoung wrote:
Secondly, and more importantly: these NN train on billions of positions - many orders of magnitude more than any human will ever see - but at ply 1 they are a lot weaker than top humans.
I must reveal what I know. And this statement is false. Neural networks are not created equally. Lc0's networks are stronger then NNUE networks. And with just 1 node not one ply are better then most human players. As I test everything I can.
If you doubt this take the 1 node Lc0 network challenge. As I have said in the past. And you will see just how strong Lc0 is in a pure network.
Most != All. Top humans can easily beat LC0 at ply 1.
I have no doubt whatsoever that it would beat me.
Nezhman
Posts : 74 Join date : 2020-11-27
Subject: Re: Is NNUE a dead end? Sun Nov 21, 2021 12:33 am
It is only a dead end if they fail to address the inherent weakness of NNUE: the safety-first approach, playing as if not to lose, especially with Black (and that accounts for %50 of the games).
This tendency results in too many draws, again mostly with Black, even against engines as much as 500-600 Elo below SF NNUE! There is also an excessive reliance on winning based on conversion to superior endgames, or simply squeezing out long grind-out wins, instead of playing more aggressively in order to put weaker opponents to the test.
They haven't been able to incorporate a functional contempt into NNUE, unlike the Komodo team, who may eventually surpass them on the rating lists!
This is why I think it's essential for testers to pit the top engines against weaker engines (one class or more lower), to uncover such hidden weaknesses and account for them in the final Elo.
Home 
