Posts : 1254 Join date : 2020-11-17 Location : France
Subject: Update on Rebel-15-with-ChrisW-NNUE dev-v2.2 Wed Mar 16, 2022 12:09 am
Rebel-15-with-ChrisW-NNUE dev-v2.2, bit of a mouthful, but that's its name at the moment.
First off, it works. I've been playing test games, so far against the Toga4-with-prior-NNUE, and keeping tabs on progress.
The new network inference engine is faster, and able to handle all the prior nets. Tests with the new inference engine vs the old inference engine, both inside Toga4, showed a 6%, 7% increase in nps (old version was getting 0.91 Mnps on my test machine, the new getting 0.99 Mnps). They ran for a few thousand test games at 40/10, W=935, L=807, D=2153 =52%. Then got the new larger net architecture going, I don't have a net from Ed yet, so tested with a few of my own nets. Nps is down, of course, net is twice the size, at 0.7 Mnps. Tried with three nets, 40/10, results varied between 54% and 56%. My other usual test is to grab a few hundred of the first PGNS, load them into Scid and count the relative wins and losses before move 50. Well, the batch I looked at, had something like 25 wins for the new net before it lost one game, which was about move 60, I think. That's a lot of Tal for a 56% overall run. Not sure where that comes from, no doubt Ed will discover it, but that's pretty good. Then tried a net at 40/40, which is running now, showing 57% with the early win rate down a little. I'ld imagine that an Ed-trained net with Ed-tuning tweaks will be able to improve on that, so I'll send him the code in the morning and over to Ed.
Admin, Mclane, adminx, matejst, Damir Desevac and Dio like this post
Chris Whittington
Posts : 1254 Join date : 2020-11-17 Location : France
Subject: Re: Update on Rebel-15-with-ChrisW-NNUE dev-v2.2 Wed Mar 16, 2022 2:08 am
Posts : 3012 Join date : 2020-11-17 Age : 57 Location : United States of Europe, Germany, Ruhr area
Subject: Re: Update on Rebel-15-with-ChrisW-NNUE dev-v2.2 Wed Mar 16, 2022 3:52 pm
This is a new Tal ?
Chris Whittington
Posts : 1254 Join date : 2020-11-17 Location : France
Subject: Re: Update on Rebel-15-with-ChrisW-NNUE dev-v2.2 Wed Mar 16, 2022 5:43 pm
Mclane wrote:
This is a new Tal ?
No. It’s an intermediate stage towards producing a releasable Rebel-15. Both Ed and I train our own nets and write our own search and there are no plans for releasing an Ed search with Chris net, nor a Chris search with an Ed net.
That said, I think Ed is getting some good results already and some good games. I guess he might publish games or even run it on GRL as a private engine, dunno, all up to him. My part would only be fixing net bugs or maybe suggesting some more net enhancements.
matejst and Dio like this post
Admin Admin
Posts : 2603 Join date : 2020-11-17 Location : Netherlands
Subject: Re: Update on Rebel-15-with-ChrisW-NNUE dev-v2.2 Wed Mar 16, 2022 7:34 pm
Mclane wrote:
This is a new Tal ?
Nope, what Chris said.
Chris created new net structures which (for instance) allows me to upgrade the existing Rebel 14.2 net and gain elo. We exchange nets for the benefit of both as in this case.
Mclane and matejst like this post
TheSelfImprover
Posts : 3095 Join date : 2020-11-18
Subject: Re: Update on Rebel-15-with-ChrisW-NNUE dev-v2.2 Wed Mar 16, 2022 9:14 pm
This is a high standard of play IMO. It looks simple, but I really struggled to understand several of the moves. This is in contrast to Mclane's 8-bit tournament, where I felt I understood most of the moves easily.
Admin and Mclane like this post
Mclane
Posts : 3012 Join date : 2020-11-17 Age : 57 Location : United States of Europe, Germany, Ruhr area
Subject: Re: Update on Rebel-15-with-ChrisW-NNUE dev-v2.2 Wed Mar 16, 2022 10:49 pm
Yes. 8 bit is easy for us. 32 or 64 bit is difficult for our brains. I once tried to visit chris in france. I was not succesful although i saw his letterbox in front of me. With me were my son and my father. 3 czubs. Maybe too much.
TheSelfImprover likes this post
Mclane
Posts : 3012 Join date : 2020-11-17 Age : 57 Location : United States of Europe, Germany, Ruhr area
Subject: Re: Update on Rebel-15-with-ChrisW-NNUE dev-v2.2 Thu Mar 17, 2022 10:34 pm
When will there be a new CSTAL with new gambit book.
Mclane
Posts : 3012 Join date : 2020-11-17 Age : 57 Location : United States of Europe, Germany, Ruhr area
Subject: Re: Update on Rebel-15-with-ChrisW-NNUE dev-v2.2 Mon Mar 21, 2022 9:46 pm
Any news?
Admin Admin
Posts : 2603 Join date : 2020-11-17 Location : Netherlands
Subject: Re: Update on Rebel-15-with-ChrisW-NNUE dev-v2.2 Mon Mar 21, 2022 11:34 pm
Working on a new neural net, so far small improvement, too little for release. The obvious way to improve is to add more positions so I am now busy doubling the size from 1.5 billion to 3 billion and that takes a lot of computer time even when you have 72 active threads available for the job.
TheSelfImprover and matejst like this post
adminx
Posts : 115 Join date : 2020-11-26 Location : Acworth, GA (USA)
Subject: Re: Update on Rebel-15-with-ChrisW-NNUE dev-v2.2 Wed Apr 06, 2022 9:50 am
Chris,
I really look forward to seeing the end product of CSTal. In the mean time I am enjoying viewing the games of the 10 minute version.
[pgn] [Event "HIARCS 15.0-CSTal match (2 games, 15 minutes per game plus 3 seconds per move)"] [Site "?"] [Date "2022.04.06"] [Round "2"] [White "CSTal"] [Black "HIARCS 15.0"] [Result "1/2-1/2"] [EventDate "2022.04.06"] [ECO "E12"]
Subject: Re: Update on Rebel-15-with-ChrisW-NNUE dev-v2.2 Thu Apr 07, 2022 2:24 am
In general, the main handicap of the CSTal/Rebel versions so far is the search. There is something that Marvin's search does well when paired with an aggressive evaluation function: somehow to find tactics. Yesterday I was analyzing a Sicilian position and while Toga Rebel gave a huge advantage to white, it did not see a mate in six (I can't remember how much time I spent on the position), something Marvin Rebel found immediately. I then tried CSTal 10mn, same problem.
I don't have a clue how the search/eval work together, but time after time I noticed the same patterns: Rebel/CSTal would sacrifice for initiative, and then miss a winning attack; or, in defense, simply succumb to a frontal attack that could have been prevented. In other words, there is no awareness of danger.
I already wrote that CSTal was a strange mixture of Karpov and Kasparov -- prevention and aggressiveness. In complicated opening positions it plays probably better than Toga Rebel. It seems to better understand tempi (e.g., it won't play Be3 before exf4) and space.
Chris Whittington
Posts : 1254 Join date : 2020-11-17 Location : France
In general, the main handicap of the CSTal/Rebel versions so far is the search. There is something that Marvin's search does well when paired with an aggressive evaluation function: somehow to find tactics. Yesterday I was analyzing a Sicilian position and while Toga Rebel gave a huge advantage to white, it did not see a mate in six (I can't remember how much time I spent on the position), something Marvin Rebel found immediately. I then tried CSTal 10mn, same problem.
I don't have a clue how the search/eval work together, but time after time I noticed the same patterns: Rebel/CSTal would sacrifice for initiative, and then miss a winning attack; or, in defense, simply succumb to a frontal attack that could have been prevented. In other words, there is no awareness of danger.
I already wrote that CSTal was a strange mixture of Karpov and Kasparov -- prevention and aggressiveness. In complicated opening positions it plays probably better than Toga Rebel. It seems to better understand tempi (e.g., it won't play Be3 before exf4) and space.
It's difficult to comment because we don't know really how the evaluation works either. There will be 'holes' in NN knowledge where it simply misevaluates, its the same with all nets, deeper search compensates, and the good "understanding" of an NN where there are no holes makes for an overall better system.
Any news? I was hoping for a Rebel 15 with a slightly improved Rebel net (the few glaring problems already discussed here solved), and a similar engine with a mature CSTal net, and some search improvements.
Chris Whittington
Posts : 1254 Join date : 2020-11-17 Location : France
Any news? I was hoping for a Rebel 15 with a slightly improved Rebel net (the few glaring problems already discussed here solved), and a similar engine with a mature CSTal net, and some search improvements.
Ed is building a net at the moment, with larger number of training positions and (I think) one the enhancements to the architecture. He is also experimenting with which positions go into the training set and which not. There really is a great deal of experimenting and testing involved, and only so much hardware to do it on. Actually, Ed could do with somebody donating him a PC (doesn’t need to be latest thing) with maybe about 4 cores and a GPU, even a 1060 is enough, believe it or not, for NN training, that’s his bottleneck.
I have a spare laptop we could use, but it is a AMD A8 with Radeon graphics, I think. So it is not much, but if I can help, I will gladly do it. Just tell me what.
Admin Admin
Posts : 2603 Join date : 2020-11-17 Location : Netherlands
I think I have about 50 elo by now but not happy yet, there must be more. Besides I am not 100% sure if it is still possible to release a SSE version. NNUE eval (by far) works best with AVX2 and new NNUE architectures developed by Chris at some point in the future may no longer work for SSE computers as they become an obstacle for progress.
I hope that Rebel 15 will be SSE compatible, no hard guarantee, but if so there will be also a Rodent NN SSE.
I have a spare laptop we could use, but it is a AMD A8 with Radeon graphics, I think. So it is not much, but if I can help, I will gladly do it. Just tell me what.
Laptops are probably not advised for this type of NN training because the training software is so thoroughly optimised (we are using PyTorch) that it continuously hammers both the GPU and CPU and that generates a lot of heat. training can last several days. Heat dissipation in laptops is not terribly good, in general. There’s quite possibly issues with farming out the training elsewhere, for example, I am currently working with 12 billion training positions and that is targeted to grow plenty more. That’s roughly 500 or 600 Gb of data, it can be compressed and I guess we could change data format and use the SF data format (I think that’s 2-3 bytes per position) but even so that’s ~30Gb for 12B positions. Completely impractical for me on rural French internet and probably not much better in practice with any “home” internet.
50 Elo is a huge increase at this level, this would bring Rebel 15.0 roughly to the level of Fritz 18NN or the new Wasp 5.50. A Rebel 15.0 would also be a nice Easter present. , thanks for your work Ed.
50 Elo is a huge increase at this level, this would bring Rebel 15.0 roughly to the level of Fritz 18NN or the new Wasp 5.50. A Rebel 15.0 would also be a nice Easter present. , thanks for your work Ed.
I second this...
Dio likes this post
Sponsored content
Subject: Re: Update on Rebel-15-with-ChrisW-NNUE dev-v2.2
Home 
, thanks for your work Ed.