Subject: Re: My first NNUE Wed Dec 15, 2021 9:34 pm
Admin wrote:
TheSelfImprover wrote:
matejst wrote:
The key will be to preserve Benjamin's style of play in the further development of the NN...
I could be wrong, but this might well be challenging. I remember when CS-Tal was under development, Chris used to complain that a side effect of small changes to improve the standard of play was often that the program would then stop making the sacrifices that were an important part of it being so special and so much fun.
Well, I am not entirely sure but with NNUE you have little to say about playing style quite contrary to HCE when you are in control. NNUE is magic and inscrutable, you play ~10 million games at depth=6 full of blunders in a couple of weeks and it replaces your evaluation (with a huge elo jump) you have been working on for 20-30 years. And you start to wonder, what have I been missing all those years....
One of these things that's easy to say when you're not doing it yourself, but I think you could influence the style by giving high scores to positions you like in the training data.
As to what you've missed: there are things GMs know that they don't even know they know. Hidden patterns that us weaker players are completely blind to.
For me, the evidence is good that even better patterns could be found with EFs that are smaller and faster than NNs - but that discussion is probably for a different thread. For now, NNs are the best known universal approximators for general use.
Mclane
Posts : 3022 Join date : 2020-11-17 Age : 57 Location : United States of Europe, Germany, Ruhr area
Subject: Re: My first NNUE Wed Dec 15, 2021 9:37 pm
The magic is the self learning. I mean as humans we know that learning is something special, Learning means fall down and stand up. And learning means make it better then the effort before.
matejst likes this post
Admin Admin
Posts : 2608 Join date : 2020-11-17 Location : Netherlands
Subject: Re: My first NNUE Wed Dec 15, 2021 9:49 pm
matejst wrote:
Ed, my name is Boban
Easy to remember, could be a dutch name
Quote :
The Benjamin net time and time again gives me lines that are perhaps losing against an engine, but are killing it in human games. The key will be to preserve Benjamin's style of play in the further development of the NN, and multiPV is very important. These second, third best moves in many positions are sometimes crucial.
In addition to what I said to Graham, perhaps you remember the odds-games threads here and at Talkchess and how bad Stockfish did, so NNUE is not blissful. Contrary to HCE you have no idea what it is doing except that it works, but may have some unexpected negative side effects. But in general the sentiment is that the created NNUE (provided with enough positions) becomes a sort of blue print of your evaluation regarding playing style. And so I am not too worried but time will tell.
Quote :
Then, I really don't know what to tell you: what's important to me is counterproductive in the world of computerchess, where a good result in the TCEC is rewarding. I see that ordinary users do not care about the strength of the engine, and that these in the middle ground, like me, are not aware of the possibility of the software and of its limitations.
I am glad there are still people (like Brendan) that look at games.
Mclane, TheSelfImprover and matejst like this post
Mclane
Posts : 3022 Join date : 2020-11-17 Age : 57 Location : United States of Europe, Germany, Ruhr area
Subject: Re: My first NNUE Wed Dec 15, 2021 10:04 pm
I wonder what the people who only watch results call themselves.
matejst likes this post
TheSelfImprover
Posts : 3112 Join date : 2020-11-18
Subject: Re: My first NNUE Wed Dec 15, 2021 10:59 pm
Mclane wrote:
I wonder what the people who only watch results call themselves.
"Bottom line men".
matejst and Tibono like this post
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: My first NNUE Thu Dec 16, 2021 12:25 pm
Still analyzing with the Benjamin NN, and here I have a position where it avoids winning material for keeping the initiative. The game is won anyway, but nonetheless, in the final position of the pgn, Nf7 seems much simpler than Bf7+, which was the choice of Benjamin.
It is not the only position where Benjamin did not simply take material. Notice that other engines (Wasp, Zahak 9, Berserk 8 ) all chose 11.Nf7 with a double attack on the queen and rook here.
Admin Admin
Posts : 2608 Join date : 2020-11-17 Location : Netherlands
Subject: Re: My first NNUE Thu Dec 16, 2021 5:11 pm
matejst wrote:
Still analyzing with the Benjamin NN, and here I have a position where it avoids winning material for keeping the initiative. The game is won anyway, but nonetheless, in the final position of the pgn, Nf7 seems much simpler than Bf7+, which was the choice of Benjamin.
It is not the only position where Benjamin did not simply take material. Notice that other engines (Wasp, Zahak 9, Berserk 8 ) all chose 11.Nf7 with a double attack on the queen and rook here.
Tested the position with ProDeo 3.1 and Benjamin 1.1, both prefer Nxf7 also. With a huge score difference between Nxf7 and Bxf7+. Very nice of the NN. BTW, SF 14.1 plays Bxf7+ almost instantly.
Admin Admin
Posts : 2608 Join date : 2020-11-17 Location : Netherlands
Subject: Re: My first NNUE Thu Dec 16, 2021 5:13 pm
TheSelfImprover wrote:
Mclane wrote:
I wonder what the people who only watch results call themselves.
"Bottom line men".
Or just bean counters, copyright Chris (1997)
Mclane likes this post
TheSelfImprover
Posts : 3112 Join date : 2020-11-18
Subject: Re: My first NNUE Thu Dec 16, 2021 6:27 pm
Admin wrote:
TheSelfImprover wrote:
Mclane wrote:
I wonder what the people who only watch results call themselves.
"Bottom line men".
Or just bean counters, copyright Chris (1997)
Well remembered!
I think that what Chris meant was slightly different though: as I remember, he was talking about quick and simple evaluations (hence deeper search), as opposed to his preference, which was for an evaluation with a large amount of chess knowledge.
matejst likes this post
Mclane
Posts : 3022 Join date : 2020-11-17 Age : 57 Location : United States of Europe, Germany, Ruhr area
Subject: Re: My first NNUE Thu Dec 16, 2021 8:52 pm
The funny thing is that chris was telling these things in the early eighties when he made programs for sinclair zx spectrum.
He has lived a very straight and continued professional dream about knowledge in chess.
TheSelfImprover likes this post
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: My first NNUE Fri Dec 17, 2021 11:10 am
The first problem with implementing knowledge is the limited knowledge of the developers. While some things are easy to understand and implement (e.g., le problem of the white squares bishop in the Slav or the QG, the need to play e4 whenever we play Bc3 -- or e5 in conjunction with Sc6; more generally, the need to improve the worst piece, etc.), some are very complicated.
An example will be the way human think in endgames (what Thorsten calls "a plan" -- because it is a plan): we think first about the [winning] position we want to achieve, then about the moves that leads to this position. Technically, I feel it has to be a nightmare.
OTOH, since engine developing was always felt like an Elo chase (although, objectively, the moment engines started playing decent positional chess it became futile - and we see it with the success of phone apps, online game, bots, or simply the success of ChessMaster), a good search simply gave more Elos, and it was a technical problem engineers were formed to solve. The top ten engines on any rating list are all fast searchers.
Suddenly, with the NNs, knowledge has become an engineering problem: one collects games and positions, and with a magic wand just trains a strong engine with lots of knowledge. But deep down, the problem remains the same: knowledge can be only produced by learning, and learning by teaching. A good initial HCE is a way of teaching a NN to learn, but it is just a beginning. Engines will plateau until we learn how to teach them, and I am inclined to think that teaching will have to be similar to the way we teach people. Patches won't be enough -- we will have to have plan how to teach NNs.
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: My first NNUE Fri Dec 17, 2021 11:35 am
As an aside... My deception with the SF project is because it gave nothing really new. Matthew Lai developed a NN for chess, and this NN was tried by Peter Osterlund in TexelGI and it was already revolutionary. Positionally, it was above anything that was created in 2017. Then there was Leela, and then Jonathan Rosenthal made an original NN for Winter in 2019.
SF, at this point, remained on the same old path. It needed the help of Shogi programmers to switch to NNUE. SF is at the top of the rating lists, but is always behind the curve. It is simply a lot of computer power and electricity wasted for nothing. Even the cloners who are not fixed on Elos have better ideas which, if developed, could give more than what the SF team do. Albert Silver did alone, for a portion of the cost, a better net than the SF team had.
Where the SF project is a great asset is in making knowledge mainstream and sharing it. That's the reason why I felt that the war against "clones" was a bit unjust, and counterproductive. E.g., original ideas in Eman were ultimately made public in Sugar. Albert's work motivated the SF team to try bigger nets, and the explanations given by Jonathan Kreuzer to train several nets for different phases of the game.
Nonetheless, for the TCEC, all engines have to be "original", SF excepted, perhaps the least original of all.
TheSelfImprover likes this post
Mclane
Posts : 3022 Join date : 2020-11-17 Age : 57 Location : United States of Europe, Germany, Ruhr area
Subject: Re: My first NNUE Fri Dec 17, 2021 11:40 am
Since chris was a chess player himself , he very early understood the importance of knowledge in chess. David levy saw it similar. Also julio kaplan. All were good chess players and immediately saw the lack of the competition engines and gave their best to fight in the competition.
Its often not easy in those days. The resources were limited. For the brute force programmers it was much easier to create ELO from hardware, Because a brute force engine needs less variable and data storage and the tree search Also needs less ram. The source code can be more compact.
So julio kaplan, chris and levy had to find ways to beat Kittinger, Rathsman and Spracklen.
Levy found Broughton. But Broughton was lost when he changed job and went into television business creating effects for image systems. Then levy found richard lang. When Hegener and Glaser hired Richard Lang Levy experimented with Taylor.
Kaplan steadily made his knowledge engine better and better. With the final product called D+ and D++.
In Computerschach and Spiele they told the lie that his final engine Socrates was heuristic alphas and therefore Kaplans legacy.
I never believed in this lie. To me this was all a PR campaign ans i think it was don daily who wrote The engine.
Don daily was also involved in the all of a sudden progress of Novag Super Forte machines. To be precise, the success of the Super Forte / Expert C version.
With chris w. the story is IMO much easier to understand. He did it on his own, from the oxford company. The name oxford was a kind of trade mark for “intelligent” methods, and chris did a good job in bringing this together in his products.
Ed schroeder came into the scene in a moment when the brute force machines dominated the market, and his clever mixed engines played much better chess then The brute force competitors. That was eds success story.
TheSelfImprover and matejst like this post
Admin Admin
Posts : 2608 Join date : 2020-11-17 Location : Netherlands
Subject: Re: My first NNUE Fri Dec 17, 2021 11:53 am
Socrates was Don D. that he was involved in Kittinger's NOVAG is news for me. Regarding me, the emphasis has always been on evaluation, search is a necessarily evil and I am not good with it.
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: My first NNUE Fri Dec 17, 2021 12:34 pm
There is a stiff divide between pre-2000 and post-2000. To be more precise: pre-1995 probably, then there was a transitional period until 2005, and then post Fruit and Rybka. We were all fooled by the fact that Vasik was an IM - I remember explanations about Rybka's internals related to chess thinking, while it was in fact related to search and pruning.
When I look at the education and mindset of the new developers, they are all engineers first. Rosenthal is a chess master and he was focused on evaluation, others treat engines like an engineering feat, improving mostly the search, or the means to create strong NNs. And it all seems to revolve around numbers.
The mindset has changed, but I feel it was a must. Chess OTB also changed. When I look which openings are played today, which variations, it would be a heresy 25 years ago. Some chess knowledge is flowing the other way today -- not from men to machines, but from machines to men. And I mean concept, abstractions, not always concrete lines.
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: My first NNUE Fri Dec 17, 2021 12:46 pm
Ed success story, in retrospect, was his "pure chess" approach. In the years of sanctions and wars, when we could not import anything (my wages, in 1993, were 10 DM a month; after a good period when I could earn 400 DM in 1996/8, about 100 DM in 1999), I managed somehow to to put my hands on a Rebel Decade, with Elo limitations if I remember well. The interface was great. There was a database. The engine played sensible chess. It was a complete tool.
And it never looked stupid. All its moves made sense: I could have played most of them myself. But the GUI, the database, the search options, all of this were a part of Ed's success IMHO.
TheSelfImprover likes this post
Mclane
Posts : 3022 Join date : 2020-11-17 Age : 57 Location : United States of Europe, Germany, Ruhr area
Subject: Re: My first NNUE Fri Dec 17, 2021 2:27 pm
Admin wrote:
Socrates was Don D. that he was involved in Kittinger's NOVAG is news for me. Regarding me, the emphasis has always been on evaluation, search is a necessarily evil and I am not good with it.
In computerschach and spiele magazin, socrates was described on many pages as julio kaplans legacy when i remember it right, So i was very surprised to see don daily later coming out with star socrates. Don was behind many developments. Dont forget he began VERY EARLY on pc when they still had b/w hercules graphical cards (Rexchess) https://www.chessprogramming.org/RexChess
You maybe remember that the early novag machines Forte A/B were weak with positional things and typically for Kittinger, good at tactics. So they hired don and he gave them piece square tables. Suddenly the novag machines put the pieces on the right squares in opening and middlegames. This was of course not made public.
In the link above i gave there is also the story when they helped novag. So don and larry helped at least 2 well known dedicated chess programmers.
Similar with socrates. For the public socrates was a julio kaplan software from kaplans „heuristic alpha“ company. And the CSS article reads like a big PR campaign for julios final work.
Last edited by Mclane on Fri Dec 17, 2021 2:41 pm; edited 2 times in total
TheSelfImprover and matejst like this post
Mclane
Posts : 3022 Join date : 2020-11-17 Age : 57 Location : United States of Europe, Germany, Ruhr area
Subject: Re: My first NNUE Fri Dec 17, 2021 2:29 pm
matejst wrote:
Ed success story, in retrospect, was his "pure chess" approach. In the years of sanctions and wars, when we could not import anything (my wages, in 1993, were 10 DM a month; after a good period when I could earn 400 DM in 1996/8, about 100 DM in 1999), I managed somehow to to put my hands on a Rebel Decade, with Elo limitations if I remember well. The interface was great. There was a database. The engine played sensible chess. It was a complete tool.
And it never looked stupid. All its moves made sense: I could have played most of them myself. But the GUI, the database, the search options, all of this were a part of Ed's success IMHO.
Eds success began with Rebel 5.0 and 1986 cologne championship and the 6502 modules for HG. It was later continued with the Chessmachine Engine and for mephisto the Risc Modules.
Uri Blass
Posts : 207 Join date : 2020-11-28
Subject: Re: My first NNUE Fri Dec 17, 2021 3:37 pm
Admin wrote:
matejst wrote:
Ed, my name is Boban
Easy to remember, could be a dutch name
Quote :
The Benjamin net time and time again gives me lines that are perhaps losing against an engine, but are killing it in human games. The key will be to preserve Benjamin's style of play in the further development of the NN, and multiPV is very important. These second, third best moves in many positions are sometimes crucial.
In addition to what I said to Graham, perhaps you remember the odds-games threads here and at Talkchess and how bad Stockfish did, so NNUE is not blissful. Contrary to HCE you have no idea what it is doing except that it works, but may have some unexpected negative side effects. But in general the sentiment is that the created NNUE (provided with enough positions) becomes a sort of blue print of your evaluation regarding playing style. And so I am not too worried but time will tell.
Quote :
Then, I really don't know what to tell you: what's important to me is counterproductive in the world of computerchess, where a good result in the TCEC is rewarding. I see that ordinary users do not care about the strength of the engine, and that these in the middle ground, like me, are not aware of the possibility of the software and of its limitations.
I am glad there are still people (like Brendan) that look at games.
My guess is that the problem with odd games is that stockfish do not care about number of moves to mate and I believe that testing is done in a wrong way.
The score should be calculated in a way that number of moves to mate is important(for example the loser is going to get 0.0001*number of moves that is clearly lower than 0.5 because number of moves is less than 5000 when the winner get 1-0.0001*number of moves).
I also would like to see 10% of the games in testing from obviously winning position for one side(for example queen handicap games). Of course to reduce luck when testing the engine against itself you should play every position with both sides.
matejst likes this post
TheSelfImprover
Posts : 3112 Join date : 2020-11-18
Subject: Re: My first NNUE Fri Dec 17, 2021 4:03 pm
matejst wrote:
...we will have to have plan how to teach NNs.
My view: there are patterns we know, and there are patterns we don't know (or at least cannot explain). We cannot teach what we don't know!
Worse still: the evidence strongly suggests to me that even the biggest nets, like LC0, haven't found the best patterns yet.
My thinking is to find these best patterns without using an NN, because one cannot control what the NN is encoding very well at this time, and they will always encode lots of patterns which aren't as good as the best patterns.
In the very long run, uncovering the best chess patterns might even lead to the resolution of chess (if a really good pattern is simple enough to unravel).
matejst likes this post
Admin Admin
Posts : 2608 Join date : 2020-11-17 Location : Netherlands
Subject: Re: My first NNUE Sat Dec 18, 2021 9:14 am
matejst wrote:
-- we will have to have plan how to teach NNs.
That's currently in full swing, NNUE is only 1.5 years old.
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: My first NNUE Sat Dec 18, 2021 9:26 am
Ed,
A final remark about the Benjamin net. I use it daily for all my analysis. It is very good at finding dynamic variations and creating attacking chances. Here is a good example, at 30'+30" time controls:
[pgn]
[Event "Play against an engine"] [Site "Lucas Chess R 1.29c"] [Date "2021.12.17"] [Round "?"] [White "Rodent Botvinnik"] [Black "Benjamin"] [ECO "C07"] [Result "0-1"]
The plan of Kh8, g5, limiting the white queen is very interesting. I have to admit that I did not understand half the game when I was watching it -- it is only afterwards, in the analysis, that I got a sense of what was really happening. I had a clue only when I calculated more concrete treats at the end of the game. Usually, black is the one defending its position in the Tarrasch French, but here Benjamin managed to turn the game around.
In general, its evaluation is a bit inflated. When Benjamin gives +2,50, other engines see +1,25, e.g. The lines are good -- compared to engines of probably similar strength, Benjamin chose book moves more often, and one can rely on its choices. I still haven't tested it in endings -- hope I will have time from Monday when I plan to analyze some rook endings.
BTW, is there a possibility for you to let me try your new, improved net, the one you mentioned in a previous post? I would be very grateful.
Admin likes this post
Admin Admin
Posts : 2608 Join date : 2020-11-17 Location : Netherlands
Subject: Re: My first NNUE Sat Dec 18, 2021 10:24 am
matejst wrote:
Ed success story, in retrospect, was his "pure chess" approach. In the years of sanctions and wars, when we could not import anything (my wages, in 1993, were 10 DM a month; after a good period when I could earn 400 DM in 1996/8, about 100 DM in 1999), I managed somehow to to put my hands on a Rebel Decade, with Elo limitations if I remember well. The interface was great. There was a database. The engine played sensible chess. It was a complete tool.
And it never looked stupid. All its moves made sense: I could have played most of them myself. But the GUI, the database, the search options, all of this were a part of Ed's success IMHO.
Like to add, the playing style of the Rebel of the 90's is much better than the current ProDeo 3.1 while the latter is likely 300-400 elo better. The thing is, and this counts for every modern search engine, pruning the tree gives loads of elo but at the same time you also prune in the evaluation (you worked so hard on) because not every pruning is (can't be) correct and thus it will have its negative effects on the playing style.
As for the good news Benjamin NN (as it seems for the moment) is a repair job
Admin Admin
Posts : 2608 Join date : 2020-11-17 Location : Netherlands
Subject: Re: My first NNUE Sat Dec 18, 2021 10:46 am
matejst wrote:
Ed,
A final remark about the Benjamin net. I use it daily for all my analysis. It is very good at finding dynamic variations and creating attacking chances. Here is a good example, at 30'+30" time controls:
[pgn]
[Event "Play against an engine"] [Site "Lucas Chess R 1.29c"] [Date "2021.12.17"] [Round "?"] [White "Rodent Botvinnik"] [Black "Benjamin"] [ECO "C07"] [Result "0-1"]
The plan of Kh8, g5, limiting the white queen is very interesting. I have to admit that I did not understand half the game when I was watching it -- it is only afterwards, in the analysis, that I got a sense of what was really happening. I had a clue only when I calculated more concrete treats at the end of the game. Usually, black is the one defending its position in the Tarrasch French, but here Benjamin managed to turn the game around.
In general, its evaluation is a bit inflated. When Benjamin gives +2,50, other engines see +1,25, e.g. The lines are good -- compared to engines of probably similar strength, Benjamin chose book moves more often, and one can rely on its choices. I still haven't tested it in endings -- hope I will have time from Monday when I plan to analyze some rook endings.
Nice game....
matejst wrote:
BTW, is there a possibility for you to let me try your new, improved net, the one you mentioned in a previous post? I would be very grateful.
That's a 20Mb net (better knowledge) doesn't work in Marvin.
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: My first NNUE Sat Dec 18, 2021 10:52 am
Admin wrote:
That's a 20Mb net (better knowledge) doesn't work in Marvin.
I believe it would not be a problem. Several engines use 20Mb nets. Of course, if you think that it is not ready for testing yet, feel free to decline.