ProDeo
Would you like to react to this message? Create an account in a few clicks or log in to continue.
ProDeo

Computer Chess
 
HomeHome  CalendarCalendar  Latest imagesLatest images  FAQFAQ  SearchSearch  MemberlistMemberlist  UsergroupsUsergroups  RegisterRegister  Log in  

 

 Is NNUE a dead end?

Go down 
4 posters
AuthorMessage
mwyoung

mwyoung


Posts : 880
Join date : 2020-11-25
Location : USA

Is NNUE a dead end? Empty
PostSubject: Is NNUE a dead end?   Is NNUE a dead end? EmptyWed Nov 10, 2021 9:45 pm

Is NNUE a dead end?

I will not use my data, but it shows the exact same thing. Here is CCRL testing data for Stockfish NNUE.

Time control 40 moves in 15 mins, but not really. (Equivalent to 40 moves in 15 minutes on an Intel i7-4770k.)

Stockfish NNUE 4 CPU ratings.

Stockfish 14 64-bit 4CPU 3547
Stockfish 13 64-bit 4CPU 3541
Stockfish 14.1 64-bit 4CPU 3518... You have to love how CCRL always stops testing. If it is not going SF's way. 196 games played. 200 games is official.
Stockfish 12 64-bit 4CPU 3512

Stockfish NNUE 1 CPU ratings.

Stockfish 14 64-bit 3505
Stockfish 13 64-bit 3504
Stockfish 14.1 64-bit 3503
Stockfish 12 64-bit 3475

Despite Stockfish's fish testing results proclaiming major gains. No real progress has been made.

Nezhman likes this post

Back to top Go down
TheSelfImprover

TheSelfImprover


Posts : 2826
Join date : 2020-11-18

Is NNUE a dead end? Empty
PostSubject: Re: Is NNUE a dead end?   Is NNUE a dead end? EmptyWed Nov 10, 2021 11:28 pm

mwyoung wrote:
Is NNUE a dead end?


I'm voting "yes".

It's a HUGE leap forward from before, but I think progress will be slow going forward.

Firstly, it would be good to know at what elo level a player becomes "almost unbeatable" (AB). I'm not going to argue about what AB means, though, mainly because my second point is a lot more important.

Secondly, and more importantly: these NN train on billions of positions - many orders of magnitude more than any human will ever see - but at ply 1 they are a lot weaker than top humans. That tells me they're "good", but they're not "it".
Back to top Go down
mwyoung

mwyoung


Posts : 880
Join date : 2020-11-25
Location : USA

Is NNUE a dead end? Empty
PostSubject: Re: Is NNUE a dead end?   Is NNUE a dead end? EmptyWed Nov 10, 2021 11:48 pm

TheSelfImprover wrote:
mwyoung wrote:
Is NNUE a dead end?


I'm voting "yes".

It's a HUGE leap forward from before, but I think progress will be slow going forward.

Firstly, it would be good to know at what elo level a player becomes "almost unbeatable" (AB). I'm not going to argue about what AB means, though, mainly because my second point is a lot more important.

Secondly, and more importantly: these NN train on billions of positions - many orders of magnitude more than any human will ever see - but at ply 1 they are a lot weaker than top humans. That tells me they're "good", but they're not "it".



""it would be good to know at what elo level a player becomes "almost unbeatable""

I am sorry you have not understood my previous posting. But this question is not knowable.

As chess is not solvable. And we do not know if chess is either a draw with best play, or a win with best play.

The best we can do is gauge what we have!
Back to top Go down
mwyoung

mwyoung


Posts : 880
Join date : 2020-11-25
Location : USA

Is NNUE a dead end? Empty
PostSubject: Re: Is NNUE a dead end?   Is NNUE a dead end? EmptyThu Nov 11, 2021 1:11 am

Secondly, and more importantly: these NN train on billions of positions - many orders of magnitude more than any human will ever see - but at ply 1 they are a lot weaker than top humans.

I must reveal what I know. And this statement is false. Neural networks are not created equally. Lc0's  networks are stronger then NNUE networks. And with just 1 node not one ply are better then most human players. As I test everything I can.

If you doubt this take the 1 node Lc0 network challenge. As I have said in the past. And you will see just how strong Lc0 is in a pure network.
Back to top Go down
Uri Blass




Posts : 182
Join date : 2020-11-28

Is NNUE a dead end? Empty
PostSubject: Re: Is NNUE a dead end?   Is NNUE a dead end? EmptyThu Nov 11, 2021 8:14 am

TheSelfImprover wrote:
mwyoung wrote:
Is NNUE a dead end?


I'm voting "yes".

It's a HUGE leap forward from before, but I think progress will be slow going forward.

Firstly, it would be good to know at what elo level a player becomes "almost unbeatable" (AB). I'm not going to argue about what AB means, though, mainly because my second point is a lot more important.

Secondly, and more importantly: these NN train on billions of positions - many orders of magnitude more than any human will ever see - but at ply 1 they are a lot weaker than top humans. That tells me they're "good", but they're not "it".

Top humans cannot play at 1 ply and even playing bullet is not the same because I am sure they caluclate in some lines more than 1 ply forward.
Back to top Go down
Uri Blass




Posts : 182
Join date : 2020-11-28

Is NNUE a dead end? Empty
PostSubject: Re: Is NNUE a dead end?   Is NNUE a dead end? EmptyThu Nov 11, 2021 8:31 am

I decided to look how stockfish14.1 lost against RubiChess and I suspect bad time management for x minutes/40 moves

Stockfish blundered at move 40 by searching to depth 1 when at move 39 it searched to depth 69 and spent 3 seconds.
Not sure if this is a new problem with 14.1

[Event "CCRL 40/15"]
[Site "CCRL"]
[Date "2021.11.05"]
[Round "796.2.857"]
[White "RubiChess 2.2 64-bit 4CPU"]
[Black "Stockfish 14.1 64-bit 4CPU"]
[Result "1-0"]
[ECO "E60"]
[Opening "King's Indian defence"]
[PlyCount "89"]
[WhiteElo "3428"]

1. d4 {+0.00/1 0s} Nf6 {+0.00/1 0s} 2. c4 {+0.00/1 0s} g6 {+0.00/1 0s} 3. f3
{+0.00/1 0s} Nc6 {+0.00/1 0s} 4. d5 {+0.00/1 0s} Ne5 {+0.00/1 0s} 5. e4
{+0.00/1 0s} d6 {+0.00/1 0s} 6. Nc3 {+0.00/1 0s} Bg7 {+0.00/1 0s} 7. f4
{+0.00/1 0s} Ned7 {+0.00/1 0s} 8. Be3 {+0.00/1 0s} O-O {+0.00/1 0s} 9. Nh3
{+0.28/33 26s} Nb6 {(b6) -0.43/36 27s} 10. Nf2 {(Nf2) +0.29/33 16s} e6 {(e6)
-0.24/36 9s} 11. Be2 {(Be2) +0.24/36 18s} exd5 {(Re8) -0.29/36 25s} 12. cxd5
{(cxd5) +0.19/34 21s} Re8 {(Re8) -0.40/35 9s} 13. O-O {(Rc1) +0.50/34 18s} c6
{(c6) -0.23/33 14s} 14. dxc6 {(dxc6) +0.32/31 20s} bxc6 {(bxc6) -0.32/35 13s}
15. Rc1 {(Rc1) +0.30/34 25s} Bb7 {(Bb7) -0.17/33 43s} 16. Bf3 {(Qd2) +0.35/33
28s} Ba6 {(d5) -0.15/35 16s} 17. Re1 {(Re1) +0.49/35 16s} Nc4 {(Nc4) +0.00/37
10s} 18. Qa4 {(Qa4) +0.20/37 16s} Bb5 {(Bb5) +0.00/39 11s} 19. Nxb5 {(Nxb5)
+0.30/37 24s} cxb5 {(cxb5) -0.12/42 17s} 20. Qb3 {(Qd1) +0.42/40 37s} Nd7
{(Nd7) +0.00/41 11s} 21. Red1 {(Red1) +0.24/36 25s} a5 {(a5) +0.00/39 13s} 22.
e5 {(e5) +0.00/36 32s} Rc8 {(Rc8) +0.00/42 13s} 23. Bg4 {(Bg4) +0.00/37 16s} h5
{(a4) +0.00/47 14s} 24. Bh3 {(Bh3) +0.00/37 20s} f5 {(a4) +0.00/46 14s} 25.
exf6 {(exf6) +0.00/36 24s} Rxe3 {(Rxe3) +0.00/48 14s} 26. Qxb5 {(Qxb5) +0.00/41
17s} Rc5 {(Rc5) +0.00/52 25s} 27. Qxd7 {(Qxd7) +0.00/43 29s} Qxf6 {(Qxf6)
+0.00/52 15s} 28. g3 {(g3) +0.00/44 16s} Nxb2 {(Nxb2) +0.00/51 16s} 29. Rxc5
{(Rxc5) +0.00/45 23s} Nxd1 {(Nxd1) +0.00/58 15s} 30. Nxd1 {(Nxd1) +0.00/46 16s}
Qd4 {(Qd4) +0.00/65 18s} 31. Rc8+ {(Rc8) +0.00/48 19s} Kh7 {(Kh7) +0.00/65 17s}
32. Nxe3 {(Nxe3) +0.00/51 28s} Qxe3+ {(Qxe3) +0.00/73 80s} 33. Kf1 {(Kf1)
+0.00/54 17s} Qd3+ {(Qf3) +0.00/82 19s} 34. Ke1 {(Ke1) +0.00/56 24s} Qe3+
{(Qe3) +0.00/82 80s} 35. Kd1 {(Kd1) +0.00/56 13s} Qd3+ {(Qd3) +0.00/78 29s} 36.
Kc1 {(Kc1) +0.00/58 20s} Qe3+ {(Qe3) +0.00/82 36s} 37. Kb1 {(Kc2) +0.00/51 16s}
Qe4+ {(Qe4) +0.00/64 4s} 38. Kc1 {(Kc1) +0.00/54 12s} Qe1+ {(Qe1) +0.00/64 14s}
39. Kc2 {(Kc2) +0.00/1 0s} Qe4+ {(Qe4) +0.00/69 3s} 40. Kb3 {(Kd2) +0.00/54 8s}
Qb1+ {(Qb4) +0.00/1 0s} 41. Kc4 {(Kc4) +7.08/30 16s} Qb4+ {(Qxa2) -8.25/33
113s} 42. Kd5 {(Kd5) +8.39/30 12s} Qd4+ {(Qd2) -8.35/42 30s} 43. Kc6 {(Kc6)
+9.41/32 17s} Qa4+ {(Qa4) -8.54/44 11s} 44. Kxd6 {(Kxd6) +9.83/30 11s} Qd4+
{(Qxa2) -9.83/50 123s} 45. Kc7 {(Kc7) +10.75/31 13s} 1-0
Back to top Go down
TheSelfImprover

TheSelfImprover


Posts : 2826
Join date : 2020-11-18

Is NNUE a dead end? Empty
PostSubject: Re: Is NNUE a dead end?   Is NNUE a dead end? EmptyThu Nov 11, 2021 10:32 am

mwyoung wrote:
Secondly, and more importantly: these NN train on billions of positions - many orders of magnitude more than any human will ever see - but at ply 1 they are a lot weaker than top humans.

I must reveal what I know. And this statement is false. Neural networks are not created equally. Lc0's  networks are stronger then NNUE networks. And with just 1 node not one ply are better then most human players. As I test everything I can.

If you doubt this take the 1 node Lc0 network challenge. As I have said in the past. And you will see just how strong Lc0 is in a pure network.


Most != All. Top humans can easily beat LC0 at ply 1.

I have no doubt whatsoever that it would beat me.
Back to top Go down
Nezhman




Posts : 74
Join date : 2020-11-27

Is NNUE a dead end? Empty
PostSubject: Re: Is NNUE a dead end?   Is NNUE a dead end? EmptySun Nov 21, 2021 12:33 am

It is only a dead end if they fail to address the inherent weakness of NNUE: the safety-first approach, playing as if not to lose, especially with Black (and that accounts for %50 of the games).

This tendency results in too many draws, again mostly with Black, even against engines as much as 500-600 Elo below SF NNUE! There is also an excessive reliance on winning based on conversion to superior endgames, or simply squeezing out long grind-out wins, instead of playing more aggressively in order to put weaker opponents to the test.

They haven't been able to incorporate a functional contempt into NNUE, unlike the Komodo team, who may eventually surpass them on the rating lists!

This is why I think it's essential for testers to pit the top engines against weaker engines (one class or more lower), to uncover such hidden weaknesses and account for them in the final Elo.
Back to top Go down
Sponsored content





Is NNUE a dead end? Empty
PostSubject: Re: Is NNUE a dead end?   Is NNUE a dead end? Empty

Back to top Go down
 
Is NNUE a dead end?
Back to top 
Page 1 of 1
 Similar topics
-
» Rodent NNUE
» My first NNUE
» glasses for NNUE?
» Texel 1.09 NNUE
» Ethereal 13 NNUE

Permissions in this forum:You cannot reply to topics in this forum
ProDeo :: Computer Chess-
Jump to: