NNUE Research Project

It´s generally known by now similarity testing on moves does not work with NNUE nets. On this page we will try to research if it is not possible using other methods. One method is to calculate the Root-mean-square deviation (or RMS) of the scores instead of moves as after all NNUE is a set of scores. We will present data and the source code for discussion elsewhere.

Let´s start at the beginning of NNUE in the summer of 2020 the starting point of the NNUE revolution when the Stockfish team implemented the Sergio nets. Our first goal is to measure the stability of the RMS of Stockfish NNUE nets. From the Sergio nets we calculate the RMS of the very first 3 nets (july) and the last 3 (september) and compare the RMS with the final SF12 net, see table one. In table two the nets between SF12 and SF13 are compared plus 5 nets after the release of SF13.

....

http://rebel13.nl/home/nnue.html

What about 270620 Gekkehenker? It was the one that really started the craze, with a clear improvement over SF classic.

I know I have tested it, see - http://talkchess.com/forum3/search.php?keywords=henk-2706&terms=all&author=&sc=1&sf=all&sr=posts&sk=t&sd=d&st=0&ch=300&t=0&submit=Search

But I can't find the NNUE no longer on my HD. It's probably on a PC that crashed a couple of months ago.

Admin wrote:

I know I have tested it, see - http://talkchess.com/forum3/search.php?keywords=henk-2706&terms=all&author=&sc=1&sf=all&sr=posts&sk=t&sd=d&st=0&ch=300&t=0&submit=Search

But I can't find the NNUE no longer on my HD. It's probably on a PC that crashed a couple of months ago.

Try here.

Tonight I have tried a stupid experiment. I have an unreleased Rodent with some code from Toga NNUE, allowing me to use Stockfish networks from before the architecture change. The experiment consisted of changing about 10% values in a 256-element array called ft_biases, described as an element of "input feature converter". These changes were guided by Texel tuning on a set little short of 6 million positions. Of course there was no way to expect a good result from butchering well-tuned weights like that, but I was interested in seeing the amount of damage. Turns out it is manageable: in a 100 game match butchered net got 45,5 points. Had it been a net tuned from scratch, I would consider it a success. I wonder what would be the effect of such change on Your similarity testing.

I can test it for you if you want.

Will this make things even more difficult now with your project ?
https://lczero.org/blog/2021/06/the-importance-of-open-data/


(off topic, feel free to move if needed)
And with the current TCEC guidelines how will this effect Stockfish entry which they used LCO generated data to build their latest NNUE files https://wiki.chessdom.org/Rules#Guidelines

It's the same guy who posted this:

https://lczero.org/blog/2021/04/jumping-on-the-nnue-bandwagon/

Look at the date.