Posts : 2528 Join date : 2020-11-17 Location : Netherlands
Subject: The NNUE split programmers are in Wed Dec 02, 2020 7:41 pm
Fact - The Stockfish team took the FREEWARE Sergio-2257 net now in use by Stockfish 12 and as usual hardly gave credit, not for the net, neither for the source code. And because of that the rest of the programmers are facing a big dilemma.
2. Is it worth the risk to receive the eternal wrath of some because they (unjustified) will claim it's steeling from Stockfish?
3. How do programmers feel themselves, doing the same as the Stockfish team, and feel bad about it, just because Stockfish was the first?
4. Several engine programmers already started to create their own NN from scratch, not using the well trained FREEWARE "Sergio" NN's at the cost of a lot of elo as the following examples from the SRL list will show.
Minic with Sergio nets
Code:
The SRL rating list of : Wed Dec 2 16:16:16 2020 EPD : lc1.epd Time : 1000ms Max Time Hash Engine Points Used Time Found Pos Elo Score Score ms Mb Cpu Errors 1 Stockfish 12 341638 11:34:09.7 25383 40000 3416 400000 85.41% 1000 128 1 0 2 Ethereal-NNUE 334327 11:37:45.7 24676 40000 3343 400000 83.58% 1000 128 1 0 3 RubiChess 1.8 NNUE 328244 11:30:21.7 23651 40000 3282 400000 82.06% 1000 128 1 0 4 Minic 2.51 SF12 327917 11:02:37.3 23628 40000 3279 400000 81.98% 1000 128 1 0 5 Minic 2.51 NNUE 325905 11:01:15.1 23441 40000 3259 400000 81.48% 1000 128 1 0 6 Minic 2.50 NNUE 322147 11:06:22.6 22986 40000 3221 400000 80.54% 1000 128 1 0
5. And, how should the rating list people act on this controversial situation, allow any NNUE version?
6. Despite the no doubt fantastic new development the author of Ethereal already stated the NNUE trend is a good reason to lose interest in computer chess, and I can sympathize with that.
I will invite some of the programmers to comment and later create an user poll here and a programmer poll on my website. I will start another thread asking for input for good poll questions.
Mclane, Brendan, matejst and koolaidsa like this post
Chris Whittington
Posts : 1254 Join date : 2020-11-17 Location : France
Subject: Re: The NNUE split programmers are in Wed Dec 02, 2020 8:24 pm
Admin wrote:
Fact - The Stockfish team took the FREEWARE Sergio-2257 net now in use by Stockfish 12 and as usual hardly gave credit, not for the net, neither for the source code. And because of that the rest of the programmers are facing a big dilemma.
2. Is it worth the risk to receive the eternal wrath of some because they (unjustified) will claim it's steeling from Stockfish?
3. How do programmers feel themselves, doing the same as the Stockfish team, and feel bad about it, just because Stockfish was the first?
4. Several engine programmers already started to create their own NN from scratch, not using the well trained FREEWARE "Sergio" NN's at the cost of a lot of elo as the following examples from the SRL list will show.
Minic with Sergio nets
Code:
The SRL rating list of : Wed Dec 2 16:16:16 2020 EPD : lc1.epd Time : 1000ms Max Time Hash Engine Points Used Time Found Pos Elo Score Score ms Mb Cpu Errors 1 Stockfish 12 341638 11:34:09.7 25383 40000 3416 400000 85.41% 1000 128 1 0 2 Ethereal-NNUE 334327 11:37:45.7 24676 40000 3343 400000 83.58% 1000 128 1 0 3 RubiChess 1.8 NNUE 328244 11:30:21.7 23651 40000 3282 400000 82.06% 1000 128 1 0 4 Minic 2.51 SF12 327917 11:02:37.3 23628 40000 3279 400000 81.98% 1000 128 1 0 5 Minic 2.51 NNUE 325905 11:01:15.1 23441 40000 3259 400000 81.48% 1000 128 1 0 6 Minic 2.50 NNUE 322147 11:06:22.6 22986 40000 3221 400000 80.54% 1000 128 1 0
5. And, how should the rating list people act on this controversial situation, allow any NNUE version?
6. Despite the no doubt fantastic new development the author of Ethereal already stated the NNUE trend is a good reason to lose interest in computer chess, and I can sympathize with that.
I will invite some of the programmers to comment and later create an user poll here and a programmer poll on my website. I will start another thread asking for input for good poll questions.
Too many engine programmers have giant egos and mental health problems. Too many end users like gossiping too much. Add the two together and you get talkchess. Put a psychopath in charge and perfect storm for any shite attacks to build. I am seriously out of all that shite and I recommend any other engine programmers to pull their engines out of distribution, keep everything private and just play online and publish interesting games.
Mclane, Brendan and Nezhman like this post
xr_a_y
Posts : 3 Join date : 2020-12-02
Subject: Re: The NNUE split programmers are in Wed Dec 02, 2020 9:01 pm
Hi,
As I'm invited to chat here, maybe it is interesting to repost my "NNUE story" again.
Phase 1 : In mid august, I was stuck at home, and has already tried some NN experiments by myself but very unsuccessfully, so i decided to give a shot at porting the under-construction NNUE thing in Stockfish inside Minic. I was easy, too easy in fact. It first gave 150Elo at least (using best Sergio nets), then at least 100 more after some optimizations I missed. The nets that were used when Minic played even with Komodo or even beat Ethereal or Xiphos was of course some very good Sergio net.
Phase 2 : As I stated in many place, this was only an experiment. Of course anyone was welcome to test MinicNNUE, but I was against using it to replace Minic in rating list or in tourney such as TCEC. I build an own net using Nodchip learner, named "napping nexus" using public SF data. This net is 100Elo weaker than Sergio one at that time.
Phase 3 : I was of course not satisfied with this, so as an effort to do some thing by myself I started merging the trainer from Nodchip repo inside Minic code in order to build pure Minic nets. That was a success and I was able to build such a net based on Minic data only, named "nascent nutrient". This net is at least 100Elo weaker than the previous one. This was still not enough to let MinicNNUE play instead of Minic as "official" version as too much SF code was now introduced inside Minic.
Phase 4 : I discovered Seer chess engine. A very well designed and written modern C++ code with a small and clear NNUE facility. I decided to go for this and removed the 5000+ lines of SF code inside Minic and replaced that by 300 understandable (to me), modifiable, lines from Seer. I also use the Seer pytorch trainer to build a net using Seer architecture (somehow a haflka) using Minic data. This is a good starting point to experiment. This net is named "nefarious nucleus" and is playing at the same level as Minic with standard eval.
Phase 5 : Since then I'm experimenting with other architectures. I use part of the Gary Linscott pytorch learner and adapt it to what I was doing. I'm currently in the way to release a new stronger net.
What's next ?
I'll continue to make standard evaluation evolve because some Minic features, like personalities, level, are based on that. This is possible because I followed dkappe advice to use a NNUE scaling computed on the fly when the program starts so that my search pruning constant remains the same.
That was the story ...
What are my feeling about that ? Well very mixed ones ...
I know very few about NN, and learning this science is a big part of what drove me inside this adventure. This is, in this sense, a good news. I really don't appreciate that some engine author updated their "official" version using SF code and nets based on SF data. But at the end, one should probably don't even care, this is personal choices. I'm happy SF is leaving Nodchip learner for a pytorch code. I'm was a bit surprised SF12 was release so soon without a "stabilized" learner. But the "hype" around NNUE was so high, it was hard to resist. About that, I now feel that giving MinicNNUE to users before SF12 was released was a little disrespectful because of course NNUE was already known by the aficionados but other people not, and SF team deserved to right to officially announce that first. So, sorry if this caused any trouble. I'm no scarred anymore of the "end of diversity" thing. I clearly see that there are a lot of roads to be explored. For instance, Seer author, Connor McMonigle, is experimenting with a lot of stuff right now.
I'll conclude saying what I said before in other places : chess engine world looks like a community to me, able to share and discuss ideas and code, to work together, and to produce high end programs. I've already witness 4 or 5 devs helping me debug Minic a few hours before TCEC submission deadline in decembre 2018. I've seen many great advice and debate on discord or on talkchess. I don't think my feeling about that is going to change soon. So thank you all again for this. Keep calm, and enjoy chess !
Admin, Mclane, matejst and Nezhman like this post
Admin Admin
Posts : 2528 Join date : 2020-11-17 Location : Netherlands
Subject: Re: The NNUE split programmers are in Thu Dec 03, 2020 1:59 am
Vivien, welcome and thanks for sharing your story. I still don't understand why you have given up 386 elo with your new 3.01 version. Why not keep it? And when building your own NN temporarily give your brainchild another name. But that's just me
xr_a_y
Posts : 3 Join date : 2020-12-02
Subject: Re: The NNUE split programmers are in Thu Dec 03, 2020 7:43 am
I guess this is not about Elo. It's about learning, it's about being proud of myself, it's about taking pleasure with what I'm doing. It's also, I think, more respectful as I have already say elsewhere of authors of great engines which were great source of inspiration to me. I understand the point about NNUE being like LMR a revolution that everyone may embrace but it just hurts my feeling to gain 300 free Elo.
And to answer a question posted in the initial post, I don't free pressure from the chess engine community when taking this decision.
Seeing Minic with top engines in some tests was fun, but it was not "real", was not my code at all.
But the adventure is still going on, I choose to go deeper in this technology but this will take time as the other parts of Minic. I didn't come here with Minic being 3000+ instantly. Far from that! It takes 2 years to get Weini at 2200Elo first! and a lot of frustration to not understand how to progress. Minic then went from 1800 to 2800 in a year and then to 3100+. It's already something to be proud of for me.
Admin and matejst like this post
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: The NNUE split programmers are in Thu Dec 03, 2020 4:05 pm
Ed,
Here, the problem seems to be a bit more complicated, because the first nets, and Sergio Vieri's nets thereafter, were created using SF eval. If I understood the whole process right, shogi developers were using SF search in their engines, so one of them, as a proof of concept, created a trainer for chess NNUE nets, based on SF.
Anyway, on OpenBench discord channel I see that authors of several engines are already experimenting far beyond NNUE, with encouraging results so far. Imho, the next step will be to make more efficient nets, combine AB and NN, and, to be honest, I would like them to close their sources, just to see how far their ideas will carry them (before being incorporated in SF).
Ras made an excellent joke calling SF BorgChess, but, the more I think of it, the more I feel it was perhaps not a joke.
Admin likes this post
Admin Admin
Posts : 2528 Join date : 2020-11-17 Location : Netherlands
Subject: Re: The NNUE split programmers are in Fri Dec 04, 2020 9:05 am
I am aware of the SF NNUE history, it's still freeware. But it's good to see authors moving to own created nets and I like the idea they keep their discoveries hidden for Stockfish.
I don't think Ras was joking, suppose I add the latest SF NNUE to ProDeo and gain a couple of hundred elo points. Then I wait for 4-5 months download the latest SF NNUE, release it as only change and cash 30-40-50 elo points. I repeat that every 4-5-6 months. I think this comes close what Ras meant.
Mars likes this post
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: The NNUE split programmers are in Fri Dec 04, 2020 9:59 am
Ed,
A project like SF is not bad per se, but I do think that it should not be allowed to participate in engine competitions like TCEC. The game is simply rigged: on one hand, you have single authors, with limited resources, on the other a team of developers with hundreds of computers. But the main problem remains the criteria: while a conditio sine qua non for single authors is originality of code/and so often ideas, this precondition does not apply to SF. If the TCEC team insist on equal hardware, why not insisting on equal rules of admission?
And what pisses me off (sorry!) are the perpetual accusations of cloning. To be clear, I am against clones, I like difference, originality, but the focus on code and code only allows SF developers to pillage any idea they see fit and to remain in the clear. I would like them so have a paradigm changing idea, but in 15 years of development there were none. What happened with NNUE is a clear example: the use of NN in AB engines was pioneered by Jonathan. Two years after Winter 0.7, StockFish finally started using a kind of net, borrowed from shogi.
So, yes, I would like to know what can SF team do when manna stops falling from heaven. Would they be able to implement an original net adapted to SF instead of just incorporating NNUE? Would they be able of incorporating opening knowledge, use of bases, etc.? Sometimes, experimenting and learning yields more results than repeating the same old stuff over and over again.
While the SF project was a great at his beginning, it has become lethal to the computerchess world. On one hand, it is arid, void of originality and ideas, on the other, it kills motivation, leaves professionals without income, and chess playing software has become the hobby of students. It is not by chance that the last GUI developed before Banksia was twenty years old, and that Rebel had a more comfortable interface that any created thereafter.
Admin likes this post
xr_a_y
Posts : 3 Join date : 2020-12-02
Subject: Re: The NNUE split programmers are in Fri Dec 04, 2020 6:53 pm
Sharing, copying, adapting code, is in my opinion quite a good thing. This allows beginners to have a starting point. This allows authors who agree to develop open-source to give, take and discuss ideas.
Given that the chess community is very happy to share wisdom, ideas, advice, in general, it feels quite logical to me that authors receiving those inputs go open-source in turn and are thus also sharing their work.
Top engines, like Stockfish, is somehow aggregating this knowledge but is also very often creating new things because Stockfish code itself is a very good sandbox for student, researchers, and chess lovers. I see nothing lethal in that. There are still hundreds on other engines you can play with or against, with very different styles, and features.
TCEC, using multiple "leagues" tourney is quite well adapted to that. QL and L4 are already producing very fun games of good quality (already something like 3200Elo on CCRL scale). In L3 and L2 this is already around 3400Elo and even more in L1 and SUFI. So there is no problem, at least for me, to see Stockfish playing here in L1.
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: The NNUE split programmers are in Fri Dec 04, 2020 7:56 pm
Bonjour, Vivien ! Un grand salut de ma part ! J'aurais aimé que l'on puisse discuter en français, mais il faut respecter les autres également, même si cela nous limite (sévèrement parfois) dans l'expression.
Although I respect your opinion, I still stand on my previous opinion: while I am not against open source (I use linux myself and I am very grateful to the open source community; I also publish my texts/articles as public domain or under creative commons licences because I do think knowledge has to be shared), I still see SF project as harmful -- it should be a better world than lethal -- from a certain point, and for the reasons I explained above.
I could be wrong, of course: it is difficult to know how the computerchess world would have evolved without SF, but it is also difficult not to notice a relative impoverishment of products (although it could be just a migration, from PC to phones), and, I feel, of ideas.
Admin Admin
Posts : 2528 Join date : 2020-11-17 Location : Netherlands
Subject: Re: The NNUE split programmers are in Sat Dec 05, 2020 11:09 pm
matejst wrote:
Ed,
A project like SF is not bad per se, but I do think that it should not be allowed to participate in engine competitions like TCEC. The game is simply rigged: on one hand, you have single authors, with limited resources, on the other a team of developers with hundreds of computers. But the main problem remains the criteria: while a conditio sine qua non for single authors is originality of code/and so often ideas, this precondition does not apply to SF. If the TCEC team insist on equal hardware, why not insisting on equal rules of admission?
And what pisses me off (sorry!) are the perpetual accusations of cloning. To be clear, I am against clones, I like difference, originality, but the focus on code and code only allows SF developers to pillage any idea they see fit and to remain in the clear. I would like them so have a paradigm changing idea, but in 15 years of development there were none. What happened with NNUE is a clear example: the use of NN in AB engines was pioneered by Jonathan. Two years after Winter 0.7, StockFish finally started using a kind of net, borrowed from shogi.
So, yes, I would like to know what can SF team do when manna stops falling from heaven. Would they be able to implement an original net adapted to SF instead of just incorporating NNUE? Would they be able of incorporating opening knowledge, use of bases, etc.? Sometimes, experimenting and learning yields more results than repeating the same old stuff over and over again.
While the SF project was a great at his beginning, it has become lethal to the computerchess world. On one hand, it is arid, void of originality and ideas, on the other, it kills motivation, leaves professionals without income, and chess playing software has become the hobby of students. It is not by chance that the last GUI developed before Banksia was twenty years old, and that Rebel had a more comfortable interface that any created thereafter.
I like being pampered but seriously, 1) the Stockfish source code is a great fishing pool for many other programmers and in the end for many (including myself) what counts is playing strength, at the same time 2) I criticize them for not giving proper credit for what you called the manna from heaven. Many programmers take ideas from Stockfish, that is what they think, but in fact it could be just the invention of other programmers, the manna.
Home 
but seriously, 1) the Stockfish source code is a great fishing pool for many other programmers and in the end for many (including myself) what counts is playing strength, at the same time 2) I criticize them for not giving proper credit for what you called the manna from heaven. Many programmers take ideas from Stockfish, that is what they think, but in fact it could be just the invention of other programmers, the manna.