Chess Tal development .... What is to be done?

Little bit of advice needed. I've developed some new code but run into a problem. I use two opening suites for testing out improvements, one is a Gambit book from Ed, the other is a balanced openings book, developed by Frank Quisinsky.
Chess System Tal without the new code gets about 3585 Elo using either opening book.
With the new code, the FQ opening book gives a little less, about 3580 Elo, but the Gambit book gives about 3610 Elo.

What is to be done?

I am in the New Code camp all day Chris!

For engine strenght i think you need a balanced opening book..like in my engine testings i use Noomen 3moves special made for eng-eng testings.
Else you just run a book testing and not busy for engine strenght.

Kind regards,
Ipman.

Ipman, welcome.

Chris Whittington wrote:

Little bit of advice needed. I've developed some new code but run into a problem. I use two opening suites for testing out improvements, one is a Gambit book from Ed, the other is a balanced openings book, developed by Frank Quisinsky.
Chess System Tal without the new code gets about 3585 Elo using either opening book.
With the new code, the FQ opening book gives a little less, about 3580 Elo, but the Gambit book gives about 3610 Elo.

What is to be done?

Alternative - http://rebel13.nl/noomen/

Merge the pgns into one.

Noomen is good.

Just a suggestion. In the past, I have primarily tested nets for Lc0, including for Albert Silver, Dietrich Kappe and Mark Jordan (Leelenstein).

For these tests I have used, among others, the NICE 2 test (>84000 test positions) from Ed.
I modified the test so that it was running without a search, i.e. only the NN was tested.

The results scaled very well with the general playing strength. If the goal is only to test the quality
of an NNUE, this test might be very helpful and relatively quick to perform. For NNUE tests, I think a thinking time of about 1000 ms is reasonable.

Translated with www.DeepL.com/Translator (free version)

How nice that you were using NICE

The decision which net became the final one at Fat Fritz 1.0 (Chessbase) or Leelenstein 15.0 was made based on this test.

I used this test so often that 60 GB of my small SSD were occupied by log files in the meantime. :-)

Dio wrote:

The decision which net became the final one at Fat Fritz 1.0 (Chessbase) or Leelenstein 15.0 was made based on this test.

Surprise there, did not expect the util would be used for top engines.

Dio wrote:

I used this test so often that 60 GB of my small SSD were occupied by log files in the meantime. :-)

Don't get me started

Chris Whittington wrote:

Little bit of advice needed. I've developed some new code but run into a problem. I use two opening suites for testing out improvements, one is a Gambit book from Ed, the other is a balanced openings book, developed by Frank Quisinsky.
Chess System Tal without the new code gets about 3585 Elo using either opening book.
With the new code, the FQ opening book gives a little less, about 3580 Elo, but the Gambit book gives about 3610 Elo.

What is to be done?

Did you want your program to be especially good at sacrificial/gambit play, or to be good at everything?

If you want it to be good at everything, you might have to accept that your new code is not as good as you were hoping for.

Use gambit book