ProDeo
Would you like to react to this message? Create an account in a few clicks or log in to continue.
ProDeo

Computer Chess
 
HomeHome  CalendarCalendar  Latest imagesLatest images  FAQFAQ  SearchSearch  MemberlistMemberlist  UsergroupsUsergroups  RegisterRegister  Log in  

 

 My first NNUE

Go down 
+4
TheSelfImprover
matejst
Mclane
Admin
8 posters
Go to page : Previous  1, 2, 3, 4, 5, 6  Next
AuthorMessage
TheSelfImprover

TheSelfImprover


Posts : 2874
Join date : 2020-11-18

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptyThu Jan 13, 2022 1:15 pm

matejst wrote:
I would like to add a word about the reaction on Talkchess. Since the rise of this internet (a)culture, I see more and more little angry people everywhere. For some reason, they feel entitled to give their mostly negative opinions about everything. The levels of education and culture are so low nowadays that it is mind boggling.


The quickest way to discover the most important things about a person's personality is to ask them for their views on other people (discussed in the "Everything Else" forum - link).
Back to top Go down
Mclane

Mclane


Posts : 2920
Join date : 2020-11-17
Age : 57
Location : United States of Europe, Germany, Ruhr area

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptySun Jan 16, 2022 11:49 am

If it is so easy to train a neural net, why not train a neural net based on Chess System Tal and put it into a modern smp open source. That should give a Tal kind of smp engine.

adminx and matejst like this post

Back to top Go down
http://www.thorstenczub.de
Tibono

Tibono


Posts : 26
Join date : 2020-11-27
Location : France

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptyMon Jan 17, 2022 1:46 pm

Admin wrote:
Tibono wrote:
p.s. I guess you compiled sse2 with hardware popcnt enabled, it crashes on my old 64bits CPU, despite being sse2 & sse3 compliant

I was already afraid this could happen, will look if it can be fixed.
Hi Ed,
I am really keen on running Rebel14, did you succeed in compiling a nopopcnt (software popcnt) version?
please please please... bounce
Cheers,
Eric
Back to top Go down
http://tibono.free.fr/Echiquiers_electroniques_eng.html
TheSelfImprover

TheSelfImprover


Posts : 2874
Join date : 2020-11-18

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptyMon Jan 17, 2022 3:36 pm

Mclane wrote:
If it is so easy to train a neural net, why not train a neural net based on Chess System Tal and put it into a modern smp open source. That should give a Tal kind of smp engine.


Very interesting question!

It would be great if it played in the style of CS Tal - but I have a nagging feeling that it wouldn't make sacrifices, wouldn't be such a cheeky monkey, and wouldn't be quite as much great fun.
Back to top Go down
Chris Whittington




Posts : 1254
Join date : 2020-11-17
Location : France

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptyMon Jan 17, 2022 5:02 pm

Tibono wrote:
Admin wrote:
Tibono wrote:
p.s. I guess you compiled sse2 with hardware popcnt enabled, it crashes on my old 64bits CPU, despite being sse2 & sse3 compliant

I was already afraid this could happen, will look if it can be fixed.
Hi Ed,
I am really keen on running Rebel14, did you succeed in compiling a nopopcnt (software popcnt) version?
please please please... bounce
Cheers,
Eric

I looked at this a couple of days ago. There’s a popcount64 function in my code, but I’m fairly sure it is never actually invoked, so Rebel14 ought to still work for you.
Back to top Go down
Tibono

Tibono


Posts : 26
Join date : 2020-11-27
Location : France

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptyMon Jan 17, 2022 8:11 pm

Thank you very much for the check, Chris. Much appreciated!

Popcount is the "usual suspect" with failing 64bit code on my CPU (apart from more recent instruction sets such as sse4, bmi2 or avx2, of course).
The sse exe starts fine in a CMD window, after the welcome/authorship message I can enter initial basic commands uci and then isready (getting uciok & readyok acknowledgements); then a go command (e.g. go depth 5) crashes after the output: info depth 1.
So indeed, only part of the code is not compliant with my CPU (Intel Pentium Penryn, Instruction Set MMX, SSE, SSE2, SSE3, SSSE3, ET64, XD, EST).

Funny observation: I need to input Enter/Return twice after both commands uci and isready, to get Rebel14 to answer. Maybe the first one acts as a line feed the program expects? No other UCI engine I know of behaves the same. Arena GUI handles this specificity without any notice (checked in the engine log-window). But of course cannot get further than info depth 1 in the log, after requesting Rebel14 to move.

Anyway I do not want to monopolize valuable efforts on my case if unique, no worries.

Kind regards,
Eric
Back to top Go down
http://tibono.free.fr/Echiquiers_electroniques_eng.html
Chris Whittington




Posts : 1254
Join date : 2020-11-17
Location : France

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptyMon Jan 17, 2022 9:38 pm

Tibono wrote:
Thank you very much for the check, Chris. Much appreciated!

Popcount is the "usual suspect" with failing 64bit code on my CPU (apart from more recent instruction sets such as sse4, bmi2 or avx2, of course).
The sse exe starts fine in a CMD window, after the welcome/authorship message I can enter initial basic commands uci and then isready (getting uciok & readyok acknowledgements); then a go command (e.g. go depth 5) crashes after the output: info depth 1.
So indeed, only part of the code is not compliant with my CPU (Intel Pentium Penryn, Instruction Set MMX, SSE, SSE2, SSE3, SSSE3, ET64, XD, EST).

Funny observation: I need to input Enter/Return twice after both commands uci and isready, to get Rebel14 to answer. Maybe the first one acts as a line feed the program expects? No other UCI engine I know of behaves the same. Arena GUI handles this specificity without any notice (checked in the engine log-window). But of course cannot get further than info depth 1 in the log, after requesting Rebel14 to move.

Anyway I do not want to monopolize valuable efforts on my case if unique, no worries.

Kind regards,
Eric

I don’t know about UCI code on this project, it has had many different programmers and I only needed to dive direct into the engine for my contribution. But I’ll take another look for use of popcount, it’s always possible the Fruitig code has its own one (I only searched for mine).
Back to top Go down
matejst

matejst


Posts : 612
Join date : 2020-11-26

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptyMon Jan 17, 2022 10:09 pm

TheSelfImprover wrote:
Mclane wrote:
If it is so easy to train a neural net, why not train a neural net based on Chess System Tal and put it into a modern smp open source. That should give a Tal kind of smp engine.


Very interesting question!

It would be great if it played in the style of CS Tal - but I have a nagging feeling that it wouldn't make sacrifices, wouldn't be such a cheeky monkey, and wouldn't be quite as much great fun.

I was doubtful about the style of the Benjamin net, but it worked well. While I cannot predict what the final result will be, I believe that creating a net from CSTal could be a worthy attempt.

In general, since the evaluation of older programs is better at lower depths than the evaluation of modern engines (Ed had experimented with this, there is page on Rebel13.nl), one can create a very decent net with shallow search. So, why not? But then, I think that Chris tried his ideas from CSTal in c-v-chess.

Mclane likes this post

Back to top Go down
Admin
Admin
Admin


Posts : 2522
Join date : 2020-11-17
Location : Netherlands

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptySat Jan 22, 2022 9:10 pm

Moved the NN to a new (and bigger) format, the gain is about 60-70 elo.

Code:
No. Name   Win Draw Loss Unf.  Score Games       %
--------------------------------------------------
  1 R14.001 +2075 =1933 -992   *0 3041.5  5000   60.8%
  2 R14     +992 =1933 -2075   *0 1958.5  5000   39.2%

The gain for the SEE version is considerable less, 56.2%

It also means that the compatibility with Marvin no longer exists.

Next step, search improvements.

matejst likes this post

Back to top Go down
http://rebel13.nl/
matejst

matejst


Posts : 612
Join date : 2020-11-26

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptySat Jan 22, 2022 9:36 pm

No problem, Ed, about compability. I use Marvin for multiPV, and a bit faster search helps the engine play more aggressively. Then, I hope you will continue to let me test Rebel.
Back to top Go down
Chris Whittington




Posts : 1254
Join date : 2020-11-17
Location : France

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptySat Jan 22, 2022 11:36 pm

Admin wrote:
Moved the NN to a new (and bigger) format, the gain is about 60-70 elo.

Code:
No. Name   Win Draw Loss Unf.  Score Games       %
--------------------------------------------------
  1 R14.001 +2075 =1933 -992   *0 3041.5  5000   60.8%
  2 R14     +992 =1933 -2075   *0 1958.5  5000   39.2%

The gain for the SEE version is considerable less, 56.2%

It also means that the compatibility with Marvin no longer exists.

Next step, search improvements.

Well, I predicted 50 Elo, so not far off. Will be a little conservative with prediction for the next net advance. 40 Elo.

Mclane and matejst like this post

Back to top Go down
matejst

matejst


Posts : 612
Join date : 2020-11-26

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptySat Jan 22, 2022 11:54 pm

Chris, I am sure that Ed, Pawel and you can improve the search for another 50-100 Elos, and if Ed manages to improve the NN, it will be all bonus. Finally, Ed has a lot of ideas how to make his engines more comfortable, more useful, and I trust him to surprise us.

Mclane and Damir Desevac like this post

Back to top Go down
matejst

matejst


Posts : 612
Join date : 2020-11-26

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptySun Jan 23, 2022 4:36 pm

After 150 games, Rebel's CCRL rating is 3070. In my own limited testing, with Marvin's search, it was a bit better than Marvin, Zahak and Wasp, my "informed" guess it performed over 3200. With a better net, Rebel could easily achieve 3250 Elos while remaining slow enough for analysis and maintaining its "human" feel.
Back to top Go down
Admin
Admin
Admin


Posts : 2522
Join date : 2020-11-17
Location : Netherlands

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptyTue Jan 25, 2022 7:34 pm

If things go well (and it looks that way) there will be a new version tomorrow. Self-play matches indicate a 98 elo progress on 40/10 and 40/40. Currently running the GRL positions at 40/120 against a 3220 elo pool in which Rebel 14 scored 43%.

http://rebel13.nl/a/grl.htm [AVX2]
http://rebel13.nl/b/grl.htm [SSE]

Changes

1. New and bigger neural network made from 1.2 billion position analyzed with Benjamin 1.1, estimated elo gain ~60.

2. Search changes, estimated elo gain ~40-50.

3. NNUE tuner parameter in Engine Settings. Changing the default of 100% to 125 will make Rebel even more aggressive, 150 up to the point of reckless, values lower than 100 will change the playing style to be more moderate. We tested the value 75 and it gave an extra 10 elo but it was preferred to leave the value at 100 exactly what the learner had created to keep the original Benjamin 1.1 playing style.

Mclane, TheSelfImprover and matejst like this post

Back to top Go down
http://rebel13.nl/
Admin
Admin
Admin


Posts : 2522
Join date : 2020-11-17
Location : Netherlands

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptySat Jan 29, 2022 2:05 am

Next step, creating 1 billion analyzed epd positions, now not with Benjamin 1.1 but with the knowledge of Rebel 14.1 and creating a new neural nnue net. I currently have about 200 million meaning still 2-3 weeks to go for the rest. No idea if that will be successful or not, the nnue way is inscrutable. Trial and error is what is left.

matejst likes this post

Back to top Go down
http://rebel13.nl/
matejst

matejst


Posts : 612
Join date : 2020-11-26

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptySat Jan 29, 2022 11:14 am

Since I analyze all the time with Rebel, I can say that the net is already very strong right now, very good in openings (I can always compare with the book moves), and it is much stronger in endgames and simple positions than the first net you sent me. But the combo Marvin/rebel-net is a bit more aggressive, somehow more "Benjamin" than "Rebel". I did not try the NNUE-tuner, but from Pawel remark, I am not sure that it works. At STC, Marvin-Rebel is stronger than Rebel 14.1 on my system (without avx2), to the point I had doubts about my settings. The search is obviously Rebel's weak point -- Rebel's plateaus somewhere at 20 plies on my system, and I think that with NNUE, some more pruning could be done that would add 4-5 plies and 50+ Elos.

Going forward with Rebel 14.1 is the right choice. There is no need to repeat the NNUE building process with Pro Deo -- the results are excellent right now and I doubt there is much to achieve -- but scaling down the centipawn estimation would be useful: it can sometimes be misleading, even for me who is quite used to Rebel's way of evaluating.

In general, it would be very kind if you could make Rebel a bit more user-friendly, the way you did with Pro Deo: more settings, multiPV (especially), experiments with big data (although NNUE is a use of big data...). Otherwise, I am quite happy with Rebel already and it has become my engine of choice.
Back to top Go down
Admin
Admin
Admin


Posts : 2522
Join date : 2020-11-17
Location : Netherlands

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptySun Jan 30, 2022 11:04 am

matejst wrote:
Since I analyze all the time with Rebel, I can say that the net is already very strong right now, very good in openings (I can always compare with the book moves), and it is much stronger in endgames and simple positions than the first net you sent me. But the combo Marvin/rebel-net is a bit more aggressive, somehow more "Benjamin" than "Rebel". I did not try the NNUE-tuner, but from Pawel remark, I am not sure that it works. At STC, Marvin-Rebel is stronger than Rebel 14.1 on my system (without avx2), to the point I had doubts about my settings. The search is obviously Rebel's weak point -- Rebel's plateaus somewhere at 20 plies on my system, and I think that with NNUE, some more pruning could be done that would add 4-5 plies and 50+ Elos.

You are quite right, search is the weakest point. But that I knew from the start choosing for an (nicely updated) engine of 2004  Very Happy

Quote :
Going forward with Rebel 14.1 is the right choice. There is no need to repeat the NNUE building process with Pro Deo -- the results are excellent right now and I doubt there is much to achieve -- but scaling down the centipawn estimation would be useful: it can sometimes be misleading, even for me who is quite used to Rebel's way of evaluating.

There is still a fundamental feature in the learner I did not try yet because lack of (computer) time which may have a big influence on playing style and playing strength. For the moment I have chosen for the balanced default value [0.5] that equally calculates the relationship between the score of moves and game result. But it seems that nowadays nnue developers are moving to higher values like [0.8] or even [0.9] which means the emphasis becomes on the score of moves. Oh well, Chris can tell you more. And so the last word on Benjamin data has not been said.

Quote :
In general, it would be very kind if you could make Rebel a bit more user-friendly, the way you did with Pro Deo: more settings, multiPV (especially), experiments with big data (although NNUE is a use of big data...). Otherwise, I am quite happy with Rebel already and it has become my engine of choice.
Back to top Go down
http://rebel13.nl/
Chris Whittington




Posts : 1254
Join date : 2020-11-17
Location : France

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptySun Jan 30, 2022 5:07 pm

Admin wrote:
matejst wrote:
Since I analyze all the time with Rebel, I can say that the net is already very strong right now, very good in openings (I can always compare with the book moves), and it is much stronger in endgames and simple positions than the first net you sent me. But the combo Marvin/rebel-net is a bit more aggressive, somehow more "Benjamin" than "Rebel". I did not try the NNUE-tuner, but from Pawel remark, I am not sure that it works. At STC, Marvin-Rebel is stronger than Rebel 14.1 on my system (without avx2), to the point I had doubts about my settings. The search is obviously Rebel's weak point -- Rebel's plateaus somewhere at 20 plies on my system, and I think that with NNUE, some more pruning could be done that would add 4-5 plies and 50+ Elos.

You are quite right, search is the weakest point. But that I knew from the start choosing for an (nicely updated) engine of 2004  Very Happy

Quote :
Going forward with Rebel 14.1 is the right choice. There is no need to repeat the NNUE building process with Pro Deo -- the results are excellent right now and I doubt there is much to achieve -- but scaling down the centipawn estimation would be useful: it can sometimes be misleading, even for me who is quite used to Rebel's way of evaluating.

There is still a fundamental feature in the learner I did not try yet because lack of (computer) time which may have a big influence on playing style and playing strength. For the moment I have chosen for the balanced default value [0.5] that equally calculates the relationship between the score of moves and game result. But it seems that nowadays nnue developers are moving to higher values like [0.8] or even [0.9] which means the emphasis becomes on the score of moves. Oh well, Chris can tell you more. And so the last word on Benjamin data has not been said.

I took a look at Berserk on GitHub yesterday, I think it’s a strong engine. From what I could tell he is using Lambda=0.0, all result, no evaluation.

Quote :
In general, it would be very kind if you could make Rebel a bit more user-friendly, the way you did with Pro Deo: more settings, multiPV (especially), experiments with big data (although NNUE is a use of big data...). Otherwise, I am quite happy with Rebel already and it has become my engine of choice.
Back to top Go down
matejst

matejst


Posts : 612
Join date : 2020-11-26

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptySun Jan 30, 2022 5:38 pm

Dietrich Kappe experimented with the ratio. He found that a mix of results and evaluation gave better results for his nets. Dark Horse and The White Rose are based on the same data, but Dark Horse is only evaluation, while The White Rose is some eval + some results, but I don't know the ratio. TWR is stronger, although I prefer DH's evaluation.

Berserk's evaluation seems to be the weakest part of the engine, and the author is working on a different net architecture right now. Its search is outstanding -- it is almost as fast as Koivisto and, although I didn't test these engines recently, Berserk seemed to be better tactically. In general, Jay Honnold seems to be a very competent engineer, making very fast progress.
Back to top Go down
matejst

matejst


Posts : 612
Join date : 2020-11-26

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptySun Jan 30, 2022 5:41 pm

In general, a lot can be found at the OpenBench discord channel, where several authors share their findings. It's here:

https://discord.com/invite/HNXsgfS
Back to top Go down
Admin
Admin
Admin


Posts : 2522
Join date : 2020-11-17
Location : Netherlands

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptySun Jan 30, 2022 8:12 pm

matejst wrote:
Berserk's evaluation seems to be the weakest part of the engine, and the author is working on a different net architecture right now. Its search is outstanding -- it is almost as fast as Koivisto and, although I didn't test these engines recently, Berserk seemed to be better tactically. In general, Jay Honnold seems to be a very competent engineer, making very fast progress.

Both Koivisto and Berserk need a good NN, the Berserk NN is based on game results only which hardly creates a better NN than HCE. Once they do that they will become competitive with Komodo.

Dio likes this post

Back to top Go down
http://rebel13.nl/
Dio




Posts : 214
Join date : 2021-08-28

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptySun Jan 30, 2022 9:02 pm

We from CEGT have started first tests with Rebel 14.1 for the 40/4 list and the results so far are looking very good. We like the possibility of using additional nets from Chris Whittington. Because Chris has already participated in online tournaments with his own engine, we would like to test this engine with his nets, provided that it will be publicly available in the near future.
Back to top Go down
Admin
Admin
Admin


Posts : 2522
Join date : 2020-11-17
Location : Netherlands

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptyMon Jan 31, 2022 12:19 am

Dio wrote:
We from CEGT have started first tests with Rebel 14.1 for the 40/4 list and the results so far are looking very good. We like the possibility of using additional nets from Chris Whittington. Because Chris has already participated in online tournaments with his own engine, we would like to test this engine with his nets, provided that it will be publicly available in the near future.

It's not for me answer the question, I am only curious if you are testing the AVX2 or SSE version.

I found the link and so far the results look good indeed.

https://cegt.forumieren.com/t1669-testing-rebel-14-1nn

Keep up the good work.

Dio likes this post

Back to top Go down
http://rebel13.nl/
Chris Whittington




Posts : 1254
Join date : 2020-11-17
Location : France

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptyTue Feb 01, 2022 12:43 am

Dio wrote:
We from CEGT have started first tests with Rebel 14.1 for the 40/4 list and the results so far are looking very good. We like the possibility of using additional nets from Chris Whittington. Because Chris has already participated in online tournaments with his own engine, we would like to test this engine with his nets, provided that it will be publicly available in the near future.

Yes and yes. Do you have a GitHub account?
Back to top Go down
Dio




Posts : 214
Join date : 2021-08-28

My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 EmptyTue Feb 01, 2022 11:18 am

Hello Chris, thank you for your answer.
Unfortunately I don't have a github account.
Maybe it would be possible to make a first contact via
Facebook, I found your profile.
I don't want to post my private email address in the forum.
Back to top Go down
Sponsored content





My first NNUE - Page 5 Empty
PostSubject: Re: My first NNUE   My first NNUE - Page 5 Empty

Back to top Go down
 
My first NNUE
Back to top 
Page 5 of 6Go to page : Previous  1, 2, 3, 4, 5, 6  Next
 Similar topics
-
» Ethereal 13 NNUE
» Is NNUE a dead end?
» Rodent NNUE
» Which NNUE development is next?
» glasses for NNUE?

Permissions in this forum:You cannot reply to topics in this forum
ProDeo :: Computer Chess-
Jump to: