Openings used in CCRL

I was interested the opening choices of the CCRL rating list by checking the score of the first move out of book. I ran 3 tests:

Score < -0.50
Score < -0.75
Score < -1.00

For some engines a score of  -0.50 is already problemetic, let alone -1.00

Code:

PGN              : pgn\ccrl-4040-commented.[1181333].pgn
Games            : 1.181.333
PGN errors       : 0
Margin           : -1.00
Suspect openings : 2.152 (0.18%)
Lost games       : 897 (0.07%)

Code:

PGN              : pgn\ccrl-4040-commented.[1181333].pgn
Games            : 1.181.333
PGN errors       : 0
Margin           : -0.75
Suspect openings : 5.879 (0.50%)
Lost games       : 2.484 (0.21%)

Code:

PGN              : pgn\ccrl-4040-commented.[1181333].pgn
Games            : 1.181.333
PGN errors       : 0
Margin           : -0.50
Suspect openings : 18.871 (1.60%)
Lost games       : 7.621 (0.65%)

Detailed:

http://rebel13.nl/text/ccrl-40-15=1.00.txt
http://rebel13.nl/text/ccrl-40-15=0.75.txt
http://rebel13.nl/text/ccrl-40-15=0.50.txt

Note the crazy scores of Giraffe, the forerunner of Alpha Zero.

Admin wrote:

I was interested the opening choices of the CCRL rating list by checking the score of the first move out of book. I ran 3 tests:

Score < -0.50
Score < -0.75
Score < -1.00

For some engines a score of  -0.50 is already problemetic, let alone -1.00

Code:

PGN              : pgn\ccrl-4040-commented.[1181333].pgn
Games            : 1.181.333
PGN errors       : 0
Margin           : -1.00
Suspect openings : 2.152 (0.18%)
Lost games       : 897 (0.07%)

Code:

PGN              : pgn\ccrl-4040-commented.[1181333].pgn
Games            : 1.181.333
PGN errors       : 0
Margin           : -0.75
Suspect openings : 5.879 (0.50%)
Lost games       : 2.484 (0.21%)

Code:

PGN              : pgn\ccrl-4040-commented.[1181333].pgn
Games            : 1.181.333
PGN errors       : 0
Margin           : -0.50
Suspect openings : 18.871 (1.60%)
Lost games       : 7.621 (0.65%)

Detailed:

http://rebel13.nl/text/ccrl-40-15=1.00.txt
http://rebel13.nl/text/ccrl-40-15=0.75.txt
http://rebel13.nl/text/ccrl-40-15=0.50.txt

Note the crazy scores of Giraffe, the forerunner of Alpha Zero.

I agree. I never understood the logic of forcing a bad opening. I understand that bad openings happen. But that should not be the goal!

mwyoung wrote:

Admin wrote:

I was interested the opening choices of the CCRL rating list by checking the score of the first move out of book. I ran 3 tests:

Score < -0.50
Score < -0.75
Score < -1.00

For some engines a score of  -0.50 is already problemetic, let alone -1.00

Code:

PGN              : pgn\ccrl-4040-commented.[1181333].pgn
Games            : 1.181.333
PGN errors       : 0
Margin           : -1.00
Suspect openings : 2.152 (0.18%)
Lost games       : 897 (0.07%)

Code:

PGN              : pgn\ccrl-4040-commented.[1181333].pgn
Games            : 1.181.333
PGN errors       : 0
Margin           : -0.75
Suspect openings : 5.879 (0.50%)
Lost games       : 2.484 (0.21%)

Code:

PGN              : pgn\ccrl-4040-commented.[1181333].pgn
Games            : 1.181.333
PGN errors       : 0
Margin           : -0.50
Suspect openings : 18.871 (1.60%)
Lost games       : 7.621 (0.65%)

Detailed:

http://rebel13.nl/text/ccrl-40-15=1.00.txt
http://rebel13.nl/text/ccrl-40-15=0.75.txt
http://rebel13.nl/text/ccrl-40-15=0.50.txt

Note the crazy scores of Giraffe, the forerunner of Alpha Zero.

I agree. I never understood the logic of forcing a bad opening. I understand that bad openings happen. But that should not be the goal!

The CEGT stats are even higher.

Code:

CCRL Suspect openings : 1.6% (see above)
CCRL Lost games       : 0.65%

CEGT Suspect openings : 2.75%
CEGT Lost games       : 1:12%

mwyoung wrote:

I agree. I never understood the logic of forcing a bad opening. I understand that bad openings happen. But that should not be the goal!

In the case of TCEC, Uri Blass believes that if computers were allowed to choose their own openings, 95% of the games would be a draw (link). If he's right, then without imbalanced openings a chess computer tournament would have too many draws.

It looks as though computer chess might be approaching "death by draw". If so, the term "chess computer" might come to be replaced by "chess calculator": it will be seen as something that just gives you a correct answer.

TheSelfImprover wrote:

mwyoung wrote:

I agree. I never understood the logic of forcing a bad opening. I understand that bad openings happen. But that should not be the goal!

In the case of TCEC, Uri Blass believes that if computers were allowed to choose their own openings, 95% of the games would be a draw (link). If he's right, then without imbalanced openings a chess computer tournament would have too many draws.

It looks as though computer chess might be approaching "death by draw". If so, the term "chess computer" might come to be replaced by "chess calculator": it will be seen as something that just gives you a correct answer.

You are correct. But that is a natural outcome of two strong engines and equal strength. Do you really think that todays engine play perfect chess, or just sloid chess.

What do you think will happen if I play today Stockfish against the best engines 5 years from now. Trust me it will not be all draws.

Draws are not a bad thing. A bad thing is making a 4000 rated computer play a opening it knows is bad. Just to force wins!. This only gives us bias results!

Here is my current match Stockfish vs Dragon played at 90m+30s. After 17 games we have 17 draws!. And that is a good thing. Because the engines are both about equal in strength, and both are playing very good chess. And I did not bias the results with bad openings. The wins in this match will come! But the best engine will earn the win, and the results will show the better chess engine. And that is the point of testing.

My view is that we shouldn't depend on engine evaluation scores to select openings for other engines to be tested on. The engines are the ones being tested (by us) and their 'opinions' on openings could be flawed and not completely trustworthy. The opening is the weakest phase of non-NN engines' play, anyway.

For example, if EngineX doesn't like so-and-so opening [this could be something like the King's Indian], do we exclude that opening from testing? What if EngineY excels with the same opening? What if EngineY is the only entity that thrives playing this opening? Do we punish EngineY for being better, by excluding an opening because other engines think it's too weak and can't handle it?

Such an opening selection process has to be fallacious and therefore needs to be avoided if we're going to have fair openings for our testing. It also means that the test suite/book opening selection would require much diversity and fine tuning, which should normally not be expected to be a quick or trivial process, but rather painstaking.  In other words, not something that's arbitrarily slapped together.

mwyoung wrote:

TheSelfImprover wrote:

mwyoung wrote:

I agree. I never understood the logic of forcing a bad opening. I understand that bad openings happen. But that should not be the goal!

In the case of TCEC, Uri Blass believes that if computers were allowed to choose their own openings, 95% of the games would be a draw (link). If he's right, then without imbalanced openings a chess computer tournament would have too many draws.

It looks as though computer chess might be approaching "death by draw". If so, the term "chess computer" might come to be replaced by "chess calculator": it will be seen as something that just gives you a correct answer.

You are correct. But that is a natural outcome of two strong engines and equal strength. Do you really think that todays engine play perfect chess, or just sloid chess.

What do you think will happen if I play today Stockfish against the best engines 5 years from now. Trust me it will not be all draws.

Draws are not a bad thing. A bad thing is making a 4000 rated computer play a opening it knows is bad. Just to force wins!. This only gives us bias results!

Here is my current match Stockfish vs Dragon played at 90m+30s. After 17 games we have 17 draws!. And that is a good thing. Because the engines are both about equal in strength, and both are playing very good chess. And I did not bias the results with bad openings. The wins in this match will come! But the best engine will earn the win, and the results will show the better chess engine. And that is the point of testing.

You may be right but I am not sure if 5 years from now the top engine is going to beat dragon or stockfish in this time control.

Note that I think that in order not to bias the results the best thing to do is to play with no opening book from the initial position or if you use a book for the first moves it can be for every engine based on the analysis of the same engine(and I believe that they are not deterministic with 32 cores so you can get different moves and not always the same moves again and again).

I do not say that you use bad opening but I understand that the first moves in the games are not the engine choices and we have no way to be sure that they are correct because we did not solve chess.

mwyoung wrote:

TheSelfImprover wrote:

mwyoung wrote:

I agree. I never understood the logic of forcing a bad opening. I understand that bad openings happen. But that should not be the goal!

In the case of TCEC, Uri Blass believes that if computers were allowed to choose their own openings, 95% of the games would be a draw (link). If he's right, then without imbalanced openings a chess computer tournament would have too many draws.

It looks as though computer chess might be approaching "death by draw". If so, the term "chess computer" might come to be replaced by "chess calculator": it will be seen as something that just gives you a correct answer.

You are correct. But that is a natural outcome of two strong engines and equal strength. Do you really think that todays engine play perfect chess, or just sloid chess.

What do you think will happen if I play today Stockfish against the best engines 5 years from now. Trust me it will not be all draws.

Draws are not a bad thing. A bad thing is making a 4000 rated computer play a opening it knows is bad. Just to force wins!. This only gives us bias results!

Here is my current match Stockfish vs Dragon played at 90m+30s. After 17 games we have 17 draws!. And that is a good thing. Because the engines are both about equal in strength, and both are playing very good chess. And I did not bias the results with bad openings. The wins in this match will come! But the best engine will earn the win, and the results will show the better chess engine. And that is the point of testing.

+1

I have always felt this was also the way to go. It also prevents these events from offering/using openings that one engine may play better, even if this was not a deliberate goal. This topic reminds me of many years ago when some complained of Mchess having a "cooked book". Funny how when you get older, your mind starts to wander. Now, where did I leave that glass of scotch.....

mwyoung wrote:

Draws are not a bad thing. A bad thing is making a 4000 rated computer play a opening it knows is bad. Just to force wins!. This only gives us bias results!

Depends on what you´re going for. The other day I was checking in a Philidor line if it is well playable for black. Yeah, Stockfish didn't like the position with black. But I made it (and Lc0 etc.) play the black side against Benjamin, Fritz, some older Komodo version, Hiarcs. The latter engines with white. Just to see how to defend this against really strong oponents.

So, yes -- they thought the opneing was bad from their point of view but for me as a chess player it was important to know if it was really *that* bad (answer: nope, one can play this if you know the right ideas).

Robert Flesher wrote:

mwyoung wrote:

TheSelfImprover wrote:

mwyoung wrote:

I agree. I never understood the logic of forcing a bad opening. I understand that bad openings happen. But that should not be the goal!

In the case of TCEC, Uri Blass believes that if computers were allowed to choose their own openings, 95% of the games would be a draw (link). If he's right, then without imbalanced openings a chess computer tournament would have too many draws.

It looks as though computer chess might be approaching "death by draw". If so, the term "chess computer" might come to be replaced by "chess calculator": it will be seen as something that just gives you a correct answer.

You are correct. But that is a natural outcome of two strong engines and equal strength. Do you really think that todays engine play perfect chess, or just sloid chess.

What do you think will happen if I play today Stockfish against the best engines 5 years from now. Trust me it will not be all draws.

Draws are not a bad thing. A bad thing is making a 4000 rated computer play a opening it knows is bad. Just to force wins!. This only gives us bias results!

Here is my current match Stockfish vs Dragon played at 90m+30s. After 17 games we have 17 draws!. And that is a good thing. Because the engines are both about equal in strength, and both are playing very good chess. And I did not bias the results with bad openings. The wins in this match will come! But the best engine will earn the win, and the results will show the better chess engine. And that is the point of testing.

+1

I have always felt this was also the way to go. It also prevents these events from offering/using openings that one engine may play better, even if this was not a deliberate goal. This topic reminds me of many years ago when some complained of Mchess having a "cooked book". Funny how when you get older, your mind starts to wander. Now, where did I leave that glass of scotch.....

Responding primarily to the bolded part - openings that distinguish certain engines over others should be exactly the kind of openings we'd want included! That's the whole point of testing (engines).

We're not testing openings, but the engines (!) that play them  - so strange people could lose sight of that..

Nezhman wrote:

Robert Flesher wrote:

mwyoung wrote:

TheSelfImprover wrote:

mwyoung wrote:

I agree. I never understood the logic of forcing a bad opening. I understand that bad openings happen. But that should not be the goal!

In the case of TCEC, Uri Blass believes that if computers were allowed to choose their own openings, 95% of the games would be a draw (link). If he's right, then without imbalanced openings a chess computer tournament would have too many draws.

It looks as though computer chess might be approaching "death by draw". If so, the term "chess computer" might come to be replaced by "chess calculator": it will be seen as something that just gives you a correct answer.

You are correct. But that is a natural outcome of two strong engines and equal strength. Do you really think that todays engine play perfect chess, or just sloid chess.

What do you think will happen if I play today Stockfish against the best engines 5 years from now. Trust me it will not be all draws.

Draws are not a bad thing. A bad thing is making a 4000 rated computer play a opening it knows is bad. Just to force wins!. This only gives us bias results!

Here is my current match Stockfish vs Dragon played at 90m+30s. After 17 games we have 17 draws!. And that is a good thing. Because the engines are both about equal in strength, and both are playing very good chess. And I did not bias the results with bad openings. The wins in this match will come! But the best engine will earn the win, and the results will show the better chess engine. And that is the point of testing.

+1

I have always felt this was also the way to go. It also prevents these events from offering/using openings that one engine may play better, even if this was not a deliberate goal. This topic reminds me of many years ago when some complained of Mchess having a "cooked book". Funny how when you get older, your mind starts to wander. Now, where did I leave that glass of scotch.....

Responding primarily to the bolded part - openings that distinguish certain engines over others should be exactly the kind of openings we'd want included! That's the whole point of testing (engines).

We're not testing openings, but the engines (!) that play them  - so strange people could lose sight of that..

As a chess player, I see it differently, and that is fine! You are testing engines that play chess. Let us not forget, they play chess. Opening theory is one of the key areas that defines chess. To not attempt to use the best openings is simply testing them at playing not the best possible chess. IMHO Although, that is fine as well! Cheers

Correct. My job is not to force wins!

I also look at it as a chess player, one who wants to see a whole diverse and inclusive spectrum of openings being used in testing - all that constitutes common chess practice: good, solid, but also risky, double-edged, gambits; closed, open games, even the starting position. The ability to handle inferior positions is just as important as playing superior, equal, or drawish ones. I also like a good mixture of shallow and deeper lines.

The moment we start saying that we don't want such-and-such openings tested, then we're doing testing a disservice by creating a gap in our testing. Imagine a map with uncharted territory on it.

As long as we have wide and well-diversified books, we should be on the right path as testers.

Hi there,

CCRL used also FEOBOS databases.
Same for CEGT after my information.

For FEOBOS, 10 engines analyzed the end position from balanced positions (3 moves after ECO codes formed). With other words the 10 engines build the opening book in team work with billions of n/s and a 6-Core machine. Each engines analzyed with 6 cores and 4.1Ghz 1-minute the complete opening database.

With an Excel and around 16 Million of formulas we analyzed the material and build a ranking system. So the complete database with 41.614 is sorted with the own ranking system, best position on place 1 and so one.

Information on FEOBOS can be found on my website, the complete documantation and the complete project files.

I think for eng-eng matches we need balanced positions. With a contempt system we developed we sorted out the positions engines produced a fast 3-fold.

Looking in detail give more information as words:
For FCP Tourney-2020 I used the own FEOBOS database!

Remis stats:
Short draw games below 20 moves:
129x after 41.000 games =  0,315% (games replayed)

Short draw games below 30 moves:
525x after 41.000 games =  1,281% (games not replayed)

Short draw games below 40 moves:
1.488x after 41.000 games =  3,630% (games not replayed)

And all of the 500 ECO codes played, and all engines played with Contempt = 0 !!
That are TOP-stats for an opening book!!

The reason is Contempt = 3 factor from FEOBOS databases.

Best
Frank

I think the topic is over since FEOBOS is available. Because I never saw better draw stats in computer chess tournaments or the book isn't balanced, or not all of the 500 ECO codes are inside. The next two FCP Tourney-2020 will be play with the TOP-10.000 positions in our ranking systems as opening book for Shredder GUI. With high-quality balanced openings the TOP-80 in the World should play the tournaments.

If you have any questions about the FEOBOS project ... running around 4-years, please!
With my bad English I try to give the answers.
Again FEOBOS is tuned by engines in self work. We created only the Excel files and sorted out bad opening material via results generated by Excel.

Nezhman wrote:

I also look at it as a chess player, one who wants to see a whole diverse and inclusive spectrum of openings being used in testing - all that constitutes common chess practice: good, solid, but also risky, double-edged, gambits; closed, open games, even the starting position. The ability to handle inferior positions is just as important as playing superior, equal, or drawish ones. I also like a good mixture of shallow and deeper lines.

The moment we start saying that we don't want such-and-such openings tested, then we're doing testing a disservice by creating a gap in our testing. Imagine a map with uncharted territory on it.

As long as we have wide and well-diversified books, we should be on the right path as testers.

Nez, you seem to contradict yourself, first you stated, " We're not testing openings, but the engines (!) that play them", after my response, you state, " I also look at it as a chess player, one who wants to see a whole diverse and inclusive spectrum of openings being used in testing - all that constitutes common chess practice: good, solid, but also risky, double-edged, gambits; closed, open games, even the starting position. The ability to handle inferior positions is just as important as playing superior, equal, or drawish ones. I also like a good mixture of shallow and deeper lines."..............
Therein, we see the problem. The two statements do not agree with each other. However, this is also fine! Observations, nothing more, keep up the great work! Cheers

Hi Robert,

no, no ..
FEOBOS told me, which are the balanced lines from A00-E99 at first. So I have the database and many rarely played balanced opening systems are inside the database.

Now, we have 870 chess players, played more as 30 games against each other with an ELO average from more as 2.400 (without blitz, simultan games) in the last 12 years. I manipulated for FCP Tourney-2020 the probably to play curent GM-theory from the last 12 years. I added the GM statistics from 86.371 games into the FEOBOS 20.1 database with Contempt 3 in step 1 but told the book ... please play only the balanced moves, not the other moves comes from the GM statistics. In step 2 I manipulated the probably a bit that the other openings, rarely played comes rarely on the board ... but comes. For FCP Tourney-2020 we have a mix between 88% curent opening theory and 12% older opening theory or rarely played moves today. That all isn't esay, because the Shredder GUI book settings are not 100% perfect. I can work with statistics and priorities.

3 moves after ECO formed is for eng-eng enough or we have long and boring lines before engines start to fight.

Example 1: If E96 formed after 9 moves the book played E96 with a depth from 12 moves. Is C84 formed after 6 moves the book play C84 with a depth from 9 moves.

B01 formed after 1 move. Min. FEOBOS started with 5 moves. In case of B01 = 4 moves after the ECO code formed.

Example 2: If I build for B01 a database with a book depth from 8 moves, most of positions in database are B01 and later most of eng-eng games are B01. And the analyze time for the end positions by 10 engines need around 3 years, only if I do that for B01. If I analysed all the 500 ECO codes with 10 engines 4 moves after ECO codes formed one system need not 15 months with 1 minute analyze time for 1 position ... 1 system need 59 months analyze time. Not possible to do that for me alone.

The system I am working here is very tricky.

Main interest I have in a bug free database of opening positions for eng-eng testing. To test engines with not balanced material isn't good because in this case I manipulated the final result and the Elo I produced can't be correct. I need all of the 500 ECO codes because I need it for playing style stats.

My English is unfortunately not good enough to explain the FEOBOS system in English. I wrote a 60 sites German documentation with all the ideas for the projects and the work we do here.

Note: I am also a chess player, like to play vs. many engines on DGT board. That's what I have the most fun ... WASP on DGT-Pi ... I can set the Elo to 2350 and Wasp is playing with 2350 Elo without to play blunders ... knps is the secret here. On DGT Pi I have the FEOBOS database ... very important. Wasp try to simulate with aggressive pawns the human style, so I can simulate openings, human style and can play vs. 1.500-2700 Elo if I like. More nobody need I think for self playing.

That's are the main ideas for FEOBOS. Strong chess players find in FCP Tourney-2020 material from all the popular openings and can looking what the best chess programs in the World like to play. A GM told me that he found for two of his favorite king indian lines fantastic material, produced with FCP Tourney-2020. So the tournament isn't not only a competition for engines, it build basics for A00-E99 opening theory, and I can build better basics for playing styles of engines.

FEOBOS is great material for programmers.
To study fast lost games after balanced positions = find blunders, own engine like to play!

If I am working for an opening book for engine x or y I would like to do that with complete other ideas. You can see in Excel table by Klaus Wlotzka all the A00-E99 lines, which are good not good for computer chess (to many draws is for us not good).

Here we created special statistics in Excel! This stats I need for special opening books for engines!!

I think all what we do here is not to 100% perfect but we try to make it perfect with a lot of ideas because main interest are not Elo of engine x or y. Main interest is to create material to openings and to looking in playing styles of engines. Elo is most boring, not the other things I am speaking before.

Best
Frank

Robert Flesher wrote:

Nezhman wrote:

I also look at it as a chess player, one who wants to see a whole diverse and inclusive spectrum of openings being used in testing - all that constitutes common chess practice: good, solid, but also risky, double-edged, gambits; closed, open games, even the starting position. The ability to handle inferior positions is just as important as playing superior, equal, or drawish ones. I also like a good mixture of shallow and deeper lines.

The moment we start saying that we don't want such-and-such openings tested, then we're doing testing a disservice by creating a gap in our testing. Imagine a map with uncharted territory on it.

As long as we have wide and well-diversified books, we should be on the right path as testers.

Nez, you seem to contradict yourself, first you stated, " We're not testing openings, but the engines (!) that play them", after my response, you state, " I also look at it as a chess player, one who wants to see a whole diverse and inclusive spectrum of openings being used in testing - all that constitutes common chess practice: good, solid, but also risky, double-edged, gambits; closed, open games, even the starting position. The ability to handle inferior positions is just as important as playing superior, equal, or drawish ones. I also like a good mixture of shallow and deeper lines."..............
Therein, we see the problem. The two statements do not agree with each other. However, this is also fine! Observations, nothing more, keep up the great work! Cheers

I see no contradiction. In a nutshell: if we don't use a sufficiently varied pool of openings, then we're not properly testing the engines (the higher goal).

The intermediate goal is to pick a selection of openings that's inclusive enough, and leaves no gaps in the testing of the engines abilities.

I'll give you another analogy. Imagine someone who's testing engines, but without including the Q-pawn (d4) openings. That would be an improper and very incomplete test of engine abilities (strength). That's what I'm talking about, as something to avoid. It's an extreme example, but you should get the idea.

Robert Flesher wrote:

This topic reminds me of many years ago when some complained of Mchess having a "cooked book".

There was good evidence that Mchess was "booked up" to beat other chess programs. I don't have proof, and I'm not going to defend this case, but the evidence I saw was that the Mchess opening book was set up specifically to get winning positions against the exact moves that rival chess programs played.

The evidence was good, and the source was good IMO.

TheSelfImprover wrote:

Robert Flesher wrote:

This topic reminds me of many years ago when some complained of Mchess having a "cooked book".

There was good evidence that Mchess was "booked up" to beat other chess programs. I don't have proof, and I'm not going to defend this case, but the evidence I saw was that the Mchess opening book was set up specifically to get winning positions against the exact moves that rival chess programs played.

The evidence was good, and the source was good IMO.

You are correct. This was not uncommon back in the day. Some programs even encoded known testing positions. To achieve better testing and review results. Results and reviews were everything if you wanted to make money in the computer chess market. And there were no free chess programs like today. And some people were not above doing some not so honest things to achieve the results and reviews they needed.

And some companies just out right lied in their advertising.

Yes, unfortunately ...
Stronger engines are a long time not compatible to UCI.
Only with Gandalf and WB Nimzo from Millennium 2000 we can do a bit to make Winboard stronger. We need the power of stronger engines for the user-community. The reason we developed Arena and at first Martin added the winboard protocoll. Same for PGN Standards and so one. Later the UCI protocol from Millennium / Stefan Meyer-Kahlen, Dieter Buerssner (Yace) and Rudolf Huber (SOS).

But we can't say that other companies do nothing. Winboard adapter from ChessBase, Young-Talents CD from Chessbase (but with amateur engines compatible the own protocol only). Mathias Feist do a lot to make the Winboard adapter stronger ... many available versions. Convekta do a lot that all of the Winboard engines runs fine. Lex Loep (Lokasoft) do also a lot. A long time really bad that Hiarcs, Junior, Fritz, ChessTiger aren't UCI compitble ... years later Hiarcs, Junior and Tiger are UCI compatible ... in times the protocols are not longer to stopped and of course ... to late!!

Same for commercial engines. Often to read how strong new commercial engines are and in the reality -100 Elo.

And commercial programmer are very "self-opinionated".
Christopher Theron in TalkChess ... never free engines will be stronger as a commercial engines. I wrote him, what should a programmer, working alone, do against the possible power of houndrets engine programmers working for free ... nothing! The free software will be overran the handfull commercial programs ... in strength and possibilitys.

The secret of all is the combination from all. Commercial software are all the time important, same for the free software. A good mix is the secret but often the commercials forgot it.

MChess was the perfect mix between the three game phases, opening, midgame and endgame. In all of the game phases with the same level. Same for Hiarcs in the beginning of stronger programs.

Robert Flesher wrote:

I also look at it as a chess player, one who wants to see a whole diverse and inclusive spectrum of openings being used in testing - all that constitutes common chess practice: good, solid, but also risky, double-edged, gambits; closed, open games, even the starting position.

Nezhman wrote:

I'll give you another analogy. Imagine someone who's testing engines, but without including the Q-pawn (d4) openings. That would be an improper and very incomplete test of engine abilities (strength). That's what I'm talking about, as something to avoid. It's an extreme example, but you should get the idea.

Robert, Nez -- I´ve got a question to you both. I understand very well that people try to test engines in all kinds of positions, which is obviously perfectly fine. But for some (well, me ) it maybe even more interesting to know how well engine X plays position Y compared to the rest of the pack. So... aren´t you not interested in that kind of question as well?

Look, I for example love to play the Dutch opening (no pun intended in this forum). Which engine can help me to analyse games and find new, good ideas -- to be played against other humans -- best? The usual rating lists give only partially an answer to that question. Sure, "check SFxx" is the first idea. But why not Benjamin 1.0 or Lc0 with some weird net is better for my purpose? And, assuming that my opponents won´t find the deep answers suggested by SF, perhaps moves and plans by lower rated engines are the ones to prepare against?

I´m no longer interested in some absurdly strong engine alone. Yeah, Cfish and Lc0 with a recent net are of course great but then interesting and most humal like play is still the holy grail for me. Unfortunately no rating list and most tests don´t answer that question (yet).

Hi there,

I am looking in stats to each of the openings. For the complicated Dutch systems best stats for my collected positions comes from SlowChess Blitz. Slow have many understandings for the most Dutch systems.

Generally, Engines produced not many fast lost games, produced many fast won games.

In my opinion the best engines for opening analyzes.
Note: King safty is the most important here, the second is the enigne should be good attacker.

That is the main reason that I am looking so many years for exactly this group of engines because most interesting opening analyzes I got from the attacker with a strong king safty. Hakkapeliitta is very aggressive but build the exception because King safty with many pieces on board is satisfying. King safty from Spark is great for an example.

A rating list will give you an Elo information only, not more and not less. If you like to analyze game material, you can see the strong and weaknesses of engines. We start this work with FCP Tourney-2020 and the Excel analyzes but need a break after all this hard work in the last months. I do that in the past without Excel for my older FCP Rating List. Horror, often I am sitting 3-4 hours on my PC for make the opening book better (FCP Live Book).

For opening analyzes are also engines interesting, like the risky / speculative chess, best example is Fizbo. All what Fizbo produced must be check in detail because Fizbo like to play blunders (same I wrote before to Hakkapeliitta). The problem is that Fizbo do not understand the Analyze-mode. Really very bad. The day will come and an other programmer add the important analyze-mode ... hope so!

A puzzle I like!
Generally, the ideas of opening positions are logical quite different. To give the ultimate hint isn't possible. I collect since over 20 years best B01 positions, dutch positions and king indian positions and know from which engines I can await best analyze material. But often, I am surprised that another engine, never on my list, comes with a genius like idea.

Lc0 is generally not the best program for complicated open openings. To many tactic holes if I compare the results with others. But for closed opening positions the engines seem to be very strong.

Check Stockfish is my last idea, not the first.
Directly after the main-lines I have not the feeling that Stockfish give always the strongest moves. The Stockfish Turbo started a bit later. But all the time I am using Stockfish at last for analyzes and check that, compare the results with my favorite engines.

"Human like" the aggressive pawns ... will you get from Wasp. Wasp with a very strong king safty with many pieces on board try to force the pawn moves, try to open the position as soon as possible. If you like a style an engine like to play aggressive with queen, bishops and knights more aggressive as the others ... try out Schooner. If you like an engine are strong in positional understandings for closed openings try out RubiChess.

Where I have the biggest problems is Booot. Booot is after game stats the perfect Allrounder with a lot of all what is important for openings, but "Booot" don't understand the "analyze mode". Same for "Fizbo". Fizbo sources are free available. I hope that the engine will be in the near feature available with Analyze-mode. This would be really an event for people like opening analyzes.

So my biggest wishes are clear:
Schooner must get the perfect UCI support with analyze-mode. And the same for Booot and Fizbo and the world of engines for opening analzyes will be much more interesting.

Best
Frank

Looking here:
http://www.amateurschach.de/main/_bench.html

Such things I like to make if the day is long and the night ...

Please have a look in the AnMon 5.75 w32 results (around 2.350 Elo).
You can see that all the different engines can produced interesting results.

To looking on the same engine ... day for day or weak for weak
is boring and you miss all the nice things around computer chess and of course around the interesting A00-E99 systems.

What most people thinking ...
The best engine will give always the best line is absolutely wrong.
Not for openings ... maybe today for endgame positions or transposition into endgame ... not for openings or positions with many pieces on board.

Best
Frank

I do such things for many of my favorite opening positions with my favorite collected engines from the past.
Inside the list (link I posted) are the current versions of TOP engines ... not my personal favorites from computer chess histoy.

Mars wrote:

Robert Flesher wrote:

I also look at it as a chess player, one who wants to see a whole diverse and inclusive spectrum of openings being used in testing - all that constitutes common chess practice: good, solid, but also risky, double-edged, gambits; closed, open games, even the starting position.

Nezhman wrote:

I'll give you another analogy. Imagine someone who's testing engines, but without including the Q-pawn (d4) openings. That would be an improper and very incomplete test of engine abilities (strength). That's what I'm talking about, as something to avoid. It's an extreme example, but you should get the idea.

Robert, Nez -- I´ve got a question to you both. I understand very well that people try to test engines in all kinds of positions, which is obviously perfectly fine. But for some (well, me ) it maybe even more interesting to know how well engine X plays position Y compared to the rest of the pack. So... aren´t you not interested in that kind of question as well?

Look, I for example love to play the Dutch opening (no pun intended in this forum). Which engine can help me to analyse games and find new, good ideas -- to be played against other humans -- best? The usual rating lists give only partially an answer to that question. Sure, "check SFxx" is the first idea. But why not Benjamin 1.0 or Lc0 with some weird net is better for my purpose? And, assuming that my opponents won´t find the deep answers suggested by SF, perhaps moves and plans by lower rated engines are the ones to prepare against?

I´m no longer interested in some absurdly strong engine alone. Yeah, Cfish and Lc0 with a recent net are of course great but then interesting and most humal like play is still the holy grail for me. Unfortunately no rating list and most tests don´t answer that question (yet).

I like those kinds of tests myself, where I focus on a certain line, or a certain key position, just to see which engines handle those positions very well, picking the right thematic continuation and that sort of thing.

Of course, that would not be a proper *strength* test, but still can offer good educational value, especially so when it comes to some favorite human openings. I've gotten many great insights that way.

Right now, I'm testing the private engine SultanKhan from the Black side of 1.d4 Nc6 (the Bozo-Indian(?), but which can easily transpose to the Nimzowitsch Defense after 2.e4, or the Chigorin after 2.c4 d5) and it looks very playable for a human as well.