Posts : 2541 Join date : 2020-11-17 Location : Netherlands
Subject: My first NNUE Wed Dec 08, 2021 2:38 pm
Created my first net from roughly 800,00 Benjamin 1.1 games resulting in 500 million positions. With the help of Chris I was able to create a first Benjamin net and played a 10 game match in Arena using a 3200 elo rated host engine to test the net. Match ended in 5-5, not bad at all for a first try. Two games.
Nice shorty, it starts with a pawn sacrifice, 11.b4
24.f3 score +7.19, opponent 24..Qe6 0.00
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: My first NNUE Sun Dec 12, 2021 8:47 am
Ed, when do you expect you will be able to release the new Benjamin NN? It looks so far like a very, very interesting engine. The key point will be to preserve its style of play while at the same time improving its strength.
Admin Admin
Posts : 2541 Join date : 2020-11-17 Location : Netherlands
Subject: Re: My first NNUE Sun Dec 12, 2021 9:33 am
matejst wrote:
Ed, when do you expect you will be able to release the new Benjamin NN? It looks so far like a very, very interesting engine. The key point will be to preserve its style of play while at the same time improving its strength.
That will be a major job. First step, I must drop ProDeo/Benjamin since it's 32 assembler with an old and outdated compiler. Second step, go back to my first PC program, the Mephisto Gideon DOS of 1993, the only engine that was 100% written in C. Because C is required for modern compilers for NNUE. Second step is to update the 1993 engine to 2021 standards. An awful job regarding the time and energy it will take.
And so it seems best to use an old GPL engine first and add Benjamin NNUE, see how that works and then decide if it makes sense to go back about 30 years in time.
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: My first NNUE Sun Dec 12, 2021 10:30 am
I believe that Pawel would have no objections if you used Rodent as a starting point -- he is often posting on this forum, so it could be a solid solution. OTOH, there could be other solutions.
Anyway, with the shift toward diversity and the lack of engines being simply different, it would be a pity not to finish this project. I lately use more and more the new Wasp 5.00 and Marvin 5.2 (Zahak 9.0 too, because it has levels) -- they are not fast, but what they offer me while playing is a human feel I don't have when using SF derivatives or "top of the shelf" engines [usually without multiPV, levels, settings...]. I was trying lately Dark Toga because it has a lot of options, but I still do not understand how to use them efficiently. I tried Benjamin too, but it lacked consistency -- excellent in some positions, very bad in others, so I prefer Rodent with Tal personality or Wasp (Orion 0.8 is also very aggressive in attacking positions).
Mclane
Posts : 2976 Join date : 2020-11-17 Age : 57 Location : United States of Europe, Germany, Ruhr area
Subject: Re: My first NNUE Sun Dec 12, 2021 11:09 am
Using assembler code is always a 2 sided sword. On the one hand you get speed, but on the negative side you are bound to the hardware beeing used or the compiler beeing used.
Richard Lang used assembler code for his 68000/68020 machines and also had to start in c from the scratch and these new genius engines played different then.
matejst likes this post
Admin Admin
Posts : 2541 Join date : 2020-11-17 Location : Netherlands
Subject: Re: My first NNUE Mon Dec 13, 2021 1:12 pm
matejst wrote:
Anyway, with the shift toward diversity and the lack of engines being simply different, it would be a pity not to finish this project. I lately use more and more the new Wasp 5.00 and Marvin 5.2 (Zahak 9.0 too, because it has levels) --
Majest, if you want you can try epoch-850-108.nnue in Marvin 5.2, curious about your findings, strength and playing style. It's the latest benjamin 1.1 NNUE and it got a 51.9% score against Marvin 5.2 itself.
matejst likes this post
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: My first NNUE Mon Dec 13, 2021 2:13 pm
Ed, I will use it for comparative analysis as soon as I come home. I won't test it in eng-eng matches though (a brief one, probably, just to get an idea), but using it like it was done 20 years ago: trying its ideas, comparing with others engines I use (the ones I wrote), assessing if I can play its choices OTB, etc. I will write about my impressions tomorrow already, and try to save some positions (pgn/epd) to comment/compare. It would be about how much it is useful for human play.
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: My first NNUE Mon Dec 13, 2021 2:29 pm
Before leaving... First test passed. With new engines, I always check this line
After move 5, if the engine will find 6.Nc3, and 8. Rc1. It is a subjective test, but I hate when I see 6.e3 when g3 has already been played and I like 8.Rc1 to avoid the queens' exchange.
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: My first NNUE Mon Dec 13, 2021 10:30 pm
I guess it is too early, but my first impressions are quite good. First, to explain what I did: I analyzed openings I learn at depth ~20, and check the lines and the moves on the board. I also compared with Wasp 5.00 and Marvin 5.20. The positions/variations were from the h3 Najdorf (early d5, simple positions), the Philidor, the Cozio/Ruy Lopez.
My lines are taken from books written before the computer revolution, and many times, the NN proposed the human lines, even when Wasp and Marvin opted for something else. The net values initiative a lot -- I already saw a lot of positional sacrifices of a pawn for initiative, and, what seemed strange at first sight, was that these sacs were not "expensive" -- I could play these lines myself without feeling that a single misstep would mean a lost game. It was also able to preserve initiative in simple, queenless positions. At the same time, this NN checked immediately for material gains, just like humans. In OTB games, it is the first thing to check.
So, after two hours of analyses: it feels human, I understand the motivation of the moves (it could perhaps be because of the shallow depth), and I would gladly play many of the lines it proposes. It also felt different in many positions, opting for different kind of positions than other engines.
Once again: these are only my first impressions, and I should continue using this NN in the following days, when I will have more to say about it. But, I can already be adamant that you should persevere with this project, Ed.
Admin, Mclane and adminx like this post
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: My first NNUE Wed Dec 15, 2021 12:53 am
I continued my testing today, and I was remembered about the influence of the search on the behaviour of the engine. I ran a few short matches against Wasp, Marvin and Orion (5-5, 4,5-5,5 and 8,5-1,5 were the results), and I analyzed positions of the French and Slav defenses, from my repertoire as black, using also Wasp and Lc0 cpu.
My old laptop is a slow AMD quad, and in matches the depth Marvin achieves with the Benjamin net is about 15-16 plies. At these depths, the Benjamin net plays an entertaining, speculative chess, very reminiscent of human play. It wins, it loses, but it is attractive. During the analysis, on the other hand, with much more time, the evaluation becomes evidently more solid, and in general is excellent. The weak spot seems to be late middlegame positions, simple positions without dynamism (although it is just an impression from the games I watched, not the positions I analyzed). Perhaps adding an endgame net (or several) like Jonathan Kreuzer could be the way to go, but what do I know.
Finally, I can just hope you will find the way to develop (or bundle, or whatever) an engine based on this net. At least, Brendan Norman and I will enjoy it and use it daily. I also want to thank you for letting me test, and use this NN.
Admin and Mclane like this post
Admin Admin
Posts : 2541 Join date : 2020-11-17 Location : Netherlands
Subject: Re: My first NNUE Wed Dec 15, 2021 9:35 am
Majest, thank you. That sounds encouraging, much to my surprise because the net was build from only 6-ply games. Meanwhile I have created a 20Mb net and it scores even better, 57%.
I have started to play 7-ply games, who knows? Will take 3 (boring) weeks to complete.
Found an interesting engine (only 30-40 elo better than Benjamin, which is excellent) to put the NNUE stuff in.
TheSelfImprover and matejst like this post
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: My first NNUE Wed Dec 15, 2021 11:00 am
Ed, my name is Boban (and an unpronounceable Serbian surname). I am a linguist and a chess player -- so I don't know the internals -- how the search, the NN etc. work. I live in a small town without a chess club, I travel a lot for my work (my university is 250 km away), so I have to rely on good and useful programs. At my age, now that my academic career is stable, I have a lot of free time to play. I am not as good as Brendan Norman (he is an IM, at my best, in the nineties, I played at a 2200 level in the Serbian/Yugo third league, and I am hopelessly trying to reach that level again), but since I rely a lot on good training software (Lucas chess mostly), with time a few things slowly become clear to me.
-- Tactics first; solving chess problems, combinations is the first thing to do.
-- Then, you have to know the positions you play. Here a good engine becomes crucial: you can train these opening with the Maia engines, but you have to investigate them first, to understand how to play them, what the "price" of the moves is, etc. Each time I played a position I was familiar with, and each time I could develop an initiative, I won my game, or draw against a much better player.
So, I noted that in my training engines with a fast search are useless at my level. The best is a relatively slow engine, but with an outstanding evaluation. Many of the lines which are losing in eng-eng games are rich of possibilities in human games. OTOH, a good evaluation in NN engines seems to be in direct correlation with the quality of the HCE of the initial engine (while the results in eng-eng matches are in direct correlation with the speed of the search; take Koivisto or Berserk as examples).
The Benjamin net time and time again gives me lines that are perhaps losing against an engine, but are killing it in human games. The key will be to preserve Benjamin's style of play in the further development of the NN, and multiPV is very important. These second, third best moves in many positions are sometimes crucial.
Then, I really don't know what to tell you: what's important to me is counterproductive in the world of computerchess, where a good result in the TCEC is rewarding. I see that ordinary users do not care about the strength of the engine, and that these in the middle ground, like me, are not aware of the possibility of the software and of its limitations.
Finally, like I wrote yesterday, the Benjamin net needs its own search, to investigate a bit deeper combinations and tactics in general -- although I found it positionally extremely sound. So here you are.
Admin likes this post
TheSelfImprover
Posts : 3028 Join date : 2020-11-18
Subject: Re: My first NNUE Wed Dec 15, 2021 11:43 am
matejst wrote:
The key will be to preserve Benjamin's style of play in the further development of the NN...
I could be wrong, but this might well be challenging. I remember when CS-Tal was under development, Chris used to complain that a side effect of small changes to improve the standard of play was often that the program would then stop making the sacrifices that were an important part of it being so special and so much fun.
matejst likes this post
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: My first NNUE Wed Dec 15, 2021 12:54 pm
TheSelfImprover wrote:
I could be wrong, but this might well be challenging. I remember when CS-Tal was under development, Chris used to complain that a side effect of small changes to improve the standard of play was often that the program would then stop making the sacrifices that were an important part of it being so special and so much fun.
OTOH, it could be enough to continue having these sacrificial lines at shallower depth, and with a system of levels, the engine could play aggressive, attacking lines at lower levels, and be solid and more positional at full strength.
In this position, Vachier-Lagrave played 10. Be7 against a lesser known player in 2006; that's also the choice of the Benjamin NN at a depth 20+ in multiPV mode, before switching to Be3. Without multiPV, it focuses exclusively on 0-0-0. Objectively, Be7 seems to be a weaker possibility in this position -- ideas with O-O-O and Bb5+ or Be3 are perhaps better, but it gives you ideas, reveals how much a position is rich. In the main line with Be3, the Benjamin NN also finds a Knight sacrifice immediately: 10.Be3, Nbd7 11. 0-0-0, c6 11. Nf6+. It is a simple sacrifice, an exchange, any player would see it, but still...
Last edited by matejst on Wed Dec 15, 2021 1:01 pm; edited 1 time in total (Reason for editing : embed pgn)
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: My first NNUE Wed Dec 15, 2021 12:55 pm
Question: how to embed pgn in a post?
Find it!
Mclane
Posts : 2976 Join date : 2020-11-17 Age : 57 Location : United States of Europe, Germany, Ruhr area
Subject: Re: My first NNUE Wed Dec 15, 2021 4:29 pm
Put the pgn between the two [ ]insert pgn notation here[ ] And write pgn in the first bracket and /pgn In the second
matejst likes this post
Admin Admin
Posts : 2541 Join date : 2020-11-17 Location : Netherlands
Subject: Re: My first NNUE Wed Dec 15, 2021 9:11 pm
TheSelfImprover wrote:
matejst wrote:
The key will be to preserve Benjamin's style of play in the further development of the NN...
I could be wrong, but this might well be challenging. I remember when CS-Tal was under development, Chris used to complain that a side effect of small changes to improve the standard of play was often that the program would then stop making the sacrifices that were an important part of it being so special and so much fun.
Well, I am not entirely sure but with NNUE you have little to say about playing style quite contrary to HCE when you are in control. NNUE is magic and inscrutable, you play ~10 million games at depth=6 full of blunders in a couple of weeks and it replaces your evaluation (with a huge elo jump) you have been working on for 20-30 years. And you start to wonder, what have I been missing all those years....
Home 
