My first NNUE

matejst wrote:

I would like to add a word about the reaction on Talkchess. Since the rise of this internet (a)culture, I see more and more little angry people everywhere. For some reason, they feel entitled to give their mostly negative opinions about everything. The levels of education and culture are so low nowadays that it is mind boggling.

The quickest way to discover the most important things about a person's personality is to ask them for their views on other people (discussed in the "Everything Else" forum - link).

If it is so easy to train a neural net, why not train a neural net based on Chess System Tal and put it into a modern smp open source. That should give a Tal kind of smp engine.

Admin wrote:

Tibono wrote:

p.s. I guess you compiled sse2 with hardware popcnt enabled, it crashes on my old 64bits CPU, despite being sse2 & sse3 compliant

I was already afraid this could happen, will look if it can be fixed.

Hi Ed,
I am really keen on running Rebel14, did you succeed in compiling a nopopcnt (software popcnt) version?
please please please...
Cheers,
Eric

Mclane wrote:

If it is so easy to train a neural net, why not train a neural net based on Chess System Tal and put it into a modern smp open source. That should give a Tal kind of smp engine.

Very interesting question!

It would be great if it played in the style of CS Tal - but I have a nagging feeling that it wouldn't make sacrifices, wouldn't be such a cheeky monkey, and wouldn't be quite as much great fun.

Tibono wrote:

Admin wrote:

Tibono wrote:

p.s. I guess you compiled sse2 with hardware popcnt enabled, it crashes on my old 64bits CPU, despite being sse2 & sse3 compliant

I was already afraid this could happen, will look if it can be fixed.

Hi Ed,
I am really keen on running Rebel14, did you succeed in compiling a nopopcnt (software popcnt) version?
please please please...
Cheers,
Eric

I looked at this a couple of days ago. There’s a popcount64 function in my code, but I’m fairly sure it is never actually invoked, so Rebel14 ought to still work for you.

Thank you very much for the check, Chris. Much appreciated!

Popcount is the "usual suspect" with failing 64bit code on my CPU (apart from more recent instruction sets such as sse4, bmi2 or avx2, of course).
The sse exe starts fine in a CMD window, after the welcome/authorship message I can enter initial basic commands uci and then isready (getting uciok & readyok acknowledgements); then a go command (e.g. go depth 5) crashes after the output: info depth 1.
So indeed, only part of the code is not compliant with my CPU (Intel Pentium Penryn, Instruction Set MMX, SSE, SSE2, SSE3, SSSE3, ET64, XD, EST).

Funny observation: I need to input Enter/Return twice after both commands uci and isready, to get Rebel14 to answer. Maybe the first one acts as a line feed the program expects? No other UCI engine I know of behaves the same. Arena GUI handles this specificity without any notice (checked in the engine log-window). But of course cannot get further than info depth 1 in the log, after requesting Rebel14 to move.

Anyway I do not want to monopolize valuable efforts on my case if unique, no worries.

Kind regards,
Eric

Tibono wrote:

Thank you very much for the check, Chris. Much appreciated!

Popcount is the "usual suspect" with failing 64bit code on my CPU (apart from more recent instruction sets such as sse4, bmi2 or avx2, of course).
The sse exe starts fine in a CMD window, after the welcome/authorship message I can enter initial basic commands uci and then isready (getting uciok & readyok acknowledgements); then a go command (e.g. go depth 5) crashes after the output: info depth 1.
So indeed, only part of the code is not compliant with my CPU (Intel Pentium Penryn, Instruction Set MMX, SSE, SSE2, SSE3, SSSE3, ET64, XD, EST).

Funny observation: I need to input Enter/Return twice after both commands uci and isready, to get Rebel14 to answer. Maybe the first one acts as a line feed the program expects? No other UCI engine I know of behaves the same. Arena GUI handles this specificity without any notice (checked in the engine log-window). But of course cannot get further than info depth 1 in the log, after requesting Rebel14 to move.

Anyway I do not want to monopolize valuable efforts on my case if unique, no worries.

Kind regards,
Eric

I don’t know about UCI code on this project, it has had many different programmers and I only needed to dive direct into the engine for my contribution. But I’ll take another look for use of popcount, it’s always possible the Fruitig code has its own one (I only searched for mine).

TheSelfImprover wrote:

Mclane wrote:

If it is so easy to train a neural net, why not train a neural net based on Chess System Tal and put it into a modern smp open source. That should give a Tal kind of smp engine.

Very interesting question!

It would be great if it played in the style of CS Tal - but I have a nagging feeling that it wouldn't make sacrifices, wouldn't be such a cheeky monkey, and wouldn't be quite as much great fun.

I was doubtful about the style of the Benjamin net, but it worked well. While I cannot predict what the final result will be, I believe that creating a net from CSTal could be a worthy attempt.

In general, since the evaluation of older programs is better at lower depths than the evaluation of modern engines (Ed had experimented with this, there is page on Rebel13.nl), one can create a very decent net with shallow search. So, why not? But then, I think that Chris tried his ideas from CSTal in c-v-chess.

Moved the NN to a new (and bigger) format, the gain is about 60-70 elo.

Code:

No. Name   Win Draw Loss Unf.  Score Games       %
--------------------------------------------------
  1 R14.001 +2075 =1933 -992   *0 3041.5  5000   60.8%
  2 R14     +992 =1933 -2075   *0 1958.5  5000   39.2%

The gain for the SEE version is considerable less, 56.2%

It also means that the compatibility with Marvin no longer exists.

Next step, search improvements.

No problem, Ed, about compability. I use Marvin for multiPV, and a bit faster search helps the engine play more aggressively. Then, I hope you will continue to let me test Rebel.

Admin wrote:

Moved the NN to a new (and bigger) format, the gain is about 60-70 elo.

Code:

No. Name   Win Draw Loss Unf.  Score Games       %
--------------------------------------------------
  1 R14.001 +2075 =1933 -992   *0 3041.5  5000   60.8%
  2 R14     +992 =1933 -2075   *0 1958.5  5000   39.2%

The gain for the SEE version is considerable less, 56.2%

It also means that the compatibility with Marvin no longer exists.

Next step, search improvements.

Well, I predicted 50 Elo, so not far off. Will be a little conservative with prediction for the next net advance. 40 Elo.

Chris, I am sure that Ed, Pawel and you can improve the search for another 50-100 Elos, and if Ed manages to improve the NN, it will be all bonus. Finally, Ed has a lot of ideas how to make his engines more comfortable, more useful, and I trust him to surprise us.

After 150 games, Rebel's CCRL rating is 3070. In my own limited testing, with Marvin's search, it was a bit better than Marvin, Zahak and Wasp, my "informed" guess it performed over 3200. With a better net, Rebel could easily achieve 3250 Elos while remaining slow enough for analysis and maintaining its "human" feel.

If things go well (and it looks that way) there will be a new version tomorrow. Self-play matches indicate a 98 elo progress on 40/10 and 40/40. Currently running the GRL positions at 40/120 against a 3220 elo pool in which Rebel 14 scored 43%.

http://rebel13.nl/a/grl.htm [AVX2]
http://rebel13.nl/b/grl.htm [SSE]

Changes

1. New and bigger neural network made from 1.2 billion position analyzed with Benjamin 1.1, estimated elo gain ~60.

2. Search changes, estimated elo gain ~40-50.

3. NNUE tuner parameter in Engine Settings. Changing the default of 100% to 125 will make Rebel even more aggressive, 150 up to the point of reckless, values lower than 100 will change the playing style to be more moderate. We tested the value 75 and it gave an extra 10 elo but it was preferred to leave the value at 100 exactly what the learner had created to keep the original Benjamin 1.1 playing style.

Next step, creating 1 billion analyzed epd positions, now not with Benjamin 1.1 but with the knowledge of Rebel 14.1 and creating a new neural nnue net. I currently have about 200 million meaning still 2-3 weeks to go for the rest. No idea if that will be successful or not, the nnue way is inscrutable. Trial and error is what is left.

Since I analyze all the time with Rebel, I can say that the net is already very strong right now, very good in openings (I can always compare with the book moves), and it is much stronger in endgames and simple positions than the first net you sent me. But the combo Marvin/rebel-net is a bit more aggressive, somehow more "Benjamin" than "Rebel". I did not try the NNUE-tuner, but from Pawel remark, I am not sure that it works. At STC, Marvin-Rebel is stronger than Rebel 14.1 on my system (without avx2), to the point I had doubts about my settings. The search is obviously Rebel's weak point -- Rebel's plateaus somewhere at 20 plies on my system, and I think that with NNUE, some more pruning could be done that would add 4-5 plies and 50+ Elos.

Going forward with Rebel 14.1 is the right choice. There is no need to repeat the NNUE building process with Pro Deo -- the results are excellent right now and I doubt there is much to achieve -- but scaling down the centipawn estimation would be useful: it can sometimes be misleading, even for me who is quite used to Rebel's way of evaluating.

In general, it would be very kind if you could make Rebel a bit more user-friendly, the way you did with Pro Deo: more settings, multiPV (especially), experiments with big data (although NNUE is a use of big data...). Otherwise, I am quite happy with Rebel already and it has become my engine of choice.

matejst wrote:

Since I analyze all the time with Rebel, I can say that the net is already very strong right now, very good in openings (I can always compare with the book moves), and it is much stronger in endgames and simple positions than the first net you sent me. But the combo Marvin/rebel-net is a bit more aggressive, somehow more "Benjamin" than "Rebel". I did not try the NNUE-tuner, but from Pawel remark, I am not sure that it works. At STC, Marvin-Rebel is stronger than Rebel 14.1 on my system (without avx2), to the point I had doubts about my settings. The search is obviously Rebel's weak point -- Rebel's plateaus somewhere at 20 plies on my system, and I think that with NNUE, some more pruning could be done that would add 4-5 plies and 50+ Elos.

You are quite right, search is the weakest point. But that I knew from the start choosing for an (nicely updated) engine of 2004  

Quote :

Going forward with Rebel 14.1 is the right choice. There is no need to repeat the NNUE building process with Pro Deo -- the results are excellent right now and I doubt there is much to achieve -- but scaling down the centipawn estimation would be useful: it can sometimes be misleading, even for me who is quite used to Rebel's way of evaluating.

There is still a fundamental feature in the learner I did not try yet because lack of (computer) time which may have a big influence on playing style and playing strength. For the moment I have chosen for the balanced default value [0.5] that equally calculates the relationship between the score of moves and game result. But it seems that nowadays nnue developers are moving to higher values like [0.8] or even [0.9] which means the emphasis becomes on the score of moves. Oh well, Chris can tell you more. And so the last word on Benjamin data has not been said.

Quote :

In general, it would be very kind if you could make Rebel a bit more user-friendly, the way you did with Pro Deo: more settings, multiPV (especially), experiments with big data (although NNUE is a use of big data...). Otherwise, I am quite happy with Rebel already and it has become my engine of choice.

Dietrich Kappe experimented with the ratio. He found that a mix of results and evaluation gave better results for his nets. Dark Horse and The White Rose are based on the same data, but Dark Horse is only evaluation, while The White Rose is some eval + some results, but I don't know the ratio. TWR is stronger, although I prefer DH's evaluation.

Berserk's evaluation seems to be the weakest part of the engine, and the author is working on a different net architecture right now. Its search is outstanding -- it is almost as fast as Koivisto and, although I didn't test these engines recently, Berserk seemed to be better tactically. In general, Jay Honnold seems to be a very competent engineer, making very fast progress.

In general, a lot can be found at the OpenBench discord channel, where several authors share their findings. It's here:

https://discord.com/invite/HNXsgfS

matejst wrote:

Berserk's evaluation seems to be the weakest part of the engine, and the author is working on a different net architecture right now. Its search is outstanding -- it is almost as fast as Koivisto and, although I didn't test these engines recently, Berserk seemed to be better tactically. In general, Jay Honnold seems to be a very competent engineer, making very fast progress.

Both Koivisto and Berserk need a good NN, the Berserk NN is based on game results only which hardly creates a better NN than HCE. Once they do that they will become competitive with Komodo.

We from CEGT have started first tests with Rebel 14.1 for the 40/4 list and the results so far are looking very good. We like the possibility of using additional nets from Chris Whittington. Because Chris has already participated in online tournaments with his own engine, we would like to test this engine with his nets, provided that it will be publicly available in the near future.

Dio wrote:

We from CEGT have started first tests with Rebel 14.1 for the 40/4 list and the results so far are looking very good. We like the possibility of using additional nets from Chris Whittington. Because Chris has already participated in online tournaments with his own engine, we would like to test this engine with his nets, provided that it will be publicly available in the near future.

It's not for me answer the question, I am only curious if you are testing the AVX2 or SSE version.

I found the link and so far the results look good indeed.

https://cegt.forumieren.com/t1669-testing-rebel-14-1nn

Keep up the good work.

Dio wrote:

We from CEGT have started first tests with Rebel 14.1 for the 40/4 list and the results so far are looking very good. We like the possibility of using additional nets from Chris Whittington. Because Chris has already participated in online tournaments with his own engine, we would like to test this engine with his nets, provided that it will be publicly available in the near future.

Yes and yes. Do you have a GitHub account?

Hello Chris, thank you for your answer.
Unfortunately I don't have a github account.
Maybe it would be possible to make a first contact via
Facebook, I found your profile.
I don't want to post my private email address in the forum.