Subject: Re: Release Rebel 15x2 Sat May 21, 2022 12:12 am
I am quite ok with the speed, Ed, I don't look for anything better otherwise I would be using SF. I just wanted to help you with details I noticed. I am not a big fan of "fast and furious" engines. Rather slow with a good evaluation.
Admin likes this post
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: Release Rebel 15x2 Sat May 21, 2022 12:19 am
Nevertheless I still think you should add some check extensions to improve the tactical abilities of Rebel since it would fit well with its evaluation.
Now I use Orion (which is even slower) in the background to check tactics. It is a good combo, but I would prefer to use solely Rebel.
pohl4711
Posts : 160 Join date : 2022-03-01 Location : Berlin
Subject: Re: Release Rebel 15x2 Sun May 22, 2022 3:24 am
(Perhaps you have to clear your browsercache or reload the website)
Apologies for an off-topic question, but, if there's a quick answer, how did Stockfish get so far clear at the top?
As top engines become more and more difficult to beat, that ought to be almost impossible.
Stockfish is a community project, lots of possible ideas as candidate improvements are contributed by people. Where 2 people cooperate 1+1 is already 3 and sometimes 4.
TheSelfImprover and matejst like this post
Admin Admin
Posts : 2608 Join date : 2020-11-17 Location : Netherlands
Subject: Re: Release Rebel 15x2 Mon May 23, 2022 9:45 am
RANK Â ENGINE Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â GAMES Â POINTS Â 1 Â 2 Â 3 Â 4 Â 5 Â 6 Â 7 Â 8 Â 9 Â 10 11 12 1. Â BERSERK 8.5.1 64-BIT Â Â Â Â Â Â Â Â Â Â Â Â 4 Â Â 3.0 Â Â * Â = Â . Â . Â . Â . Â . Â . Â . Â = Â 1 Â 1 2. Â SLOWCHESS BLITZ 2.8 64-BIT Â Â Â Â Â Â Â Â Â 4 Â Â 3.0 Â Â = Â * Â . Â . Â . Â . Â . Â = Â . Â . Â 1 Â 1 3. Â RUBICHESS 20220223 64-BIT Â Â Â Â Â Â Â Â Â Â 5 Â Â 3.0 Â Â . Â . Â * Â = Â = Â . Â = Â . Â = Â . Â . Â 1 4. Â STOCKFISH 15 64-BIT Â Â Â Â Â Â Â Â Â Â Â Â Â 4 Â Â 2.5 Â Â . Â . Â = Â * Â = Â = Â . Â . Â . Â . Â . Â 1 5. Â KOIVISTO 8.0 64-BIT Â Â Â Â Â Â Â Â Â Â Â Â Â 4 Â Â 2.5 Â Â . Â . Â = Â = Â * Â . Â = Â . Â . Â . Â . Â 1 6. Â SEER 2.5.0 64-BIT Â Â Â Â Â Â Â Â Â Â Â Â Â Â 4 Â Â 2.5 Â Â . Â . Â . Â = Â . Â * Â = Â . Â 1 Â = Â . Â . 7. Â REBEL 15X2 64-BIT Â Â Â Â Â Â Â Â Â Â Â Â Â Â 4 Â Â 2.0 Â Â . Â . Â = Â . Â = Â = Â * Â = Â . Â . Â . Â . 8. Â IGEL 3.0.5 64-BIT Â Â Â Â Â Â Â Â Â Â Â Â Â Â 4 Â Â 2.0 Â Â . Â = Â . Â . Â . Â . Â = Â * Â = Â = Â . Â . 9. Â ARASAN 23.3 64-BIT Â Â Â Â Â Â Â Â Â Â Â Â Â 4 Â Â 2.0 Â Â . Â . Â = Â . Â . Â 0 Â . Â = Â * Â . Â 1 Â . 10. Â ROFCHADE 3.0 64-BIT Â Â Â Â Â Â Â Â Â Â Â Â Â 4 Â Â 2.0 Â Â = Â . Â . Â . Â . Â = Â . Â = Â . Â * Â = Â . 11. Â FIRE 8.2 64-BIT Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 4 Â Â 0.5 Â Â 0 Â 0 Â . Â . Â . Â . Â . Â . Â 0 Â = Â * Â . 12. Â BOOOT 7.0-BETA 64-BIT Â Â Â Â Â Â Â Â Â Â Â Â 5 Â Â 0.0 Â Â 0 Â 0 Â 0 Â 0 Â 0 Â . Â . Â . Â . Â . Â . Â * Total games = 25
4 draws so far
adminx, matejst, Damir Desevac, Dio and Eelco like this post
Admin Admin
Posts : 2608 Join date : 2020-11-17 Location : Netherlands
Subject: Re: Release Rebel 15x2 Tue May 24, 2022 2:24 am
RANK  ENGINE                   GAMES  POINTS 1  2  3  4  5  6  7  8  9  10 11 12 1.  STOCKFISH 15 64-BIT              7   5.0   *  =  .  1  =  .  =  .  .  1  =  1 2.  SEER 2.5.0 64-BIT               8   5.0   =  *  .  .  =  =  =  1  =  =  .  1 3.  SLOWCHESS BLITZ 2.8 64-BIT          7   4.5   .  .  *  =  .  =  .  =  =  =  1  1 4.  BERSERK 8.5.1 64-BIT             7   4.0   0  .  =  *  .  =  .  =  =  .  1  1 5.  KOIVISTO 8.0 64-BIT              7   4.0   =  =  .  .  *  =  =  .  .  =  =  1 6.  REBEL 15X2 64-BIT               8   4.0   .  =  =  =  =  *  =  .  =  =  =  . 7.  RUBICHESS 20220223 64-BIT           7   4.0   =  =  .  .  =  =  *  =  .  =  .  1 8.  ARASAN 23.3 64-BIT              8   4.0   .  0  =  =  .  .  =  *  =  =  1  = 9.  ROFCHADE 3.0 64-BIT              8   4.0   .  =  =  =  .  =  .  =  *  =  =  = 10. IGEL 3.0.5 64-BIT               8   3.5   0  =  =  .  =  =  =  =  =  *  .  . 11. FIRE 8.2 64-BIT                7   2.0   =  .  0  0  =  =  .  0  =  .  *  . 12. BOOOT 7.0-BETA 64-BIT             8   1.0   0  0  0  0  0  .  0  =  =  .  .  * Total games = 45
8 draws now
Mclane, TheSelfImprover, matejst, Damir Desevac and Dio like this post
Ed, please check the search. It could well be buggy, although I am not sure if is something to do with the GUI.
When I analyze, I sometimes get nonsensical lines, with systematic blunders. It doesn't happen immediately, when I start analyzing, but after some times. In such cases, I also noted that the search slows down.
I thought it was related with the complexity of the position, but I am not sure, I still cannot find which kind of positions triggers this behaviour. Another hypothesis is that has something to do with engine switching -- e.g., today I compare Rebel's lines with SF, and the problem started after that.
Anyway, I think it could have an impact on Rebel's results. In general, its evaluation is outstanding -- in the opening and middlegame, it is at SF's level or very close.
Edit: the endgame play is still much worse than the middlegame.
Admin Admin
Posts : 2608 Join date : 2020-11-17 Location : Netherlands
Suddenly, Rebel 15x2 needs 3mn to achieve depth 14, while it is usually almost instantaneous. It could be the GUI, I will check.
It's true, the Toga search is not good with checks and pawn promotions that end up with checks causing search explosions. I will try my own check system, fixed that problem in the 80's.
matejst likes this post
Admin Admin
Posts : 2608 Join date : 2020-11-17 Location : Netherlands
Subject: Re: Release Rebel 15x2 Wed Jun 01, 2022 7:34 am
Subject: Re: Release Rebel 15x2 Fri Jun 03, 2022 10:21 am
Meanwhile... Did Rebel lost the "Benjamin style"? I still use Marvin with the Benjamin78 NN, and while it seems weaker in the endings, and sometimes too optimistic in the early middlegame (the openings are a topic that should be treated apart), when I compare it with Rebel15x2, its style of play seems much more active, more aggressive than the style of Rebel15x2 (and Rebel 15), which became somehow a bit drawish.
I think that it is due in part because of the inadequacies of the Toga search -- it simply misses tactical shots. I wrote several time here that Rebel 15 misses mates seen not only by other engines, but "felt" by me, if not outright calculated (although only once, when it was a direct, forced mate in six, with checks along the way). Marvin/Benjamin found this particular mate right away.
Then, there is another thing. Short story long, to try to explain myself. I also tried two engines whose nets were based mainly on ending positions: Seer and SCB. Their analysis in the opening are... let's say correct, but when the position is not straightforward, Rebel is way better (in all versions). Despite this disadvantage in openings, and while not being ultra-fast, both engines fare well on different rating lists and in the TCEC competition. I attribute their good results -- following John Stanback remark at CCC -- to their outstanding game in simple positions and endings. Having tested SCB a bit more (Seer has an unpleasant bug), I can attest that it is really the case.
So, I was thinking. Was it not possible to safeguard the Benjamin aggressive style in openings, and add lots of ending knowledge to improve his results (and fix the Toga search). Could a selective approach to data/training retain the Rebel idiosyncrasies, which made it not only so valuable for human analysis, but also so original? Can Rebel remain more than "just another NN engine", like it was at his beginnings?
Finally, a disclaimer. All my remarks are made without knowledge of the way Rebel was done. It is just an impression, and I could be utterly wrong.
Ed, just to confirm. I think now that the reason for Rebel's lack of aggressiveness is because it simply does not find tactical shots. Like I wrote before, I noticed it when analyzing, but then I decided to test both Marvin/Rebel and Rebel 15x2 with the "Tactics for intermediate players" base (exercises I do myself) and Rebel 15x2 had difficulties finding solutions longer than mate in 4, while Marvin/Rebel found them immediately.
You can try for yourself with problems suited for chess engines, I believe the problem would become clear. I believe it has an important impact on the results of Rebel.
matejst
Posts : 612 Join date : 2020-11-26
Subject: Re: Release Rebel 15x2 Wed Jun 08, 2022 11:33 am
Endgames -- while Rebel 15x2 is better in simple positions, it still lacks behind the first versions of CSTal NN in endgame analysis. I worked on rook endgames and transition to pawn endgames yesterday, and analyzed the positions with both engines. Despite the handicap of not having TBs support implemented, CSTal still plays endings clearly better than Rebel 15x2. Rebel somehow still lacks specialized endgame knowledge. Let's have a look at this position:
[puzzle] [SetUp "1"] [FEN "8/3b2k1/8/3B4/8/r7/5RPP/6K1 b - -"]
SlowChess, even without use of TBs solves it easily. So, this knowledge can be trained, and specialized ending knowledge would probably improve drastically the results of the engine. Why not add TB positions to the training data, like Connor Macmonigle did?
Recapitulation.
Rebel's search misses tactical shots and not only probably alters drastically the style of the engine, but influences its results vastly. With a similar net, Rebel 14 was 100 Elos weaker than Marvin/Rebel. Solving the problems with the search will probably not yield so much Elos, but could well hugely improve the results.
Rebel still lacks technical, robust endgame knowledge. It is not only about bishops of different colors -- there, the situation is much better -- but it seems it is simply not trained on enough ending positions.
Dio
Posts : 222 Join date : 2021-08-28
Subject: Re: Release Rebel 15x2 Sun Jun 12, 2022 7:50 pm
it took a while until the games of Rebel 15x2 were evaluated, because our webmaster was very busy, the official first entry of Rebel 15x2 in the 40/4 CEGT list. More results will follow...
Admin, Mclane, adminx and Ipmanchess like this post
Chris Whittington
Posts : 1254 Join date : 2020-11-17 Location : France
Subject: Re: Release Rebel 15x2 Sun Jun 12, 2022 8:12 pm
matejst wrote:
Endgames -- while Rebel 15x2 is better in simple positions, it still lacks behind the first versions of CSTal NN in endgame analysis. I worked on rook endgames and transition to pawn endgames yesterday, and analyzed the positions with both engines. Despite the handicap of not having TBs support implemented, CSTal still plays endings clearly better than Rebel 15x2. Rebel somehow still lacks specialized endgame knowledge. Let's have a look at this position:
[puzzle] [SetUp "1"] [FEN "8/3b2k1/8/3B4/8/r7/5RPP/6K1 b - -"]
SlowChess, even without use of TBs solves it easily. So, this knowledge can be trained, and specialized ending knowledge would probably improve drastically the results of the engine. Why not add TB positions to the training data, like Connor Macmonigle did?
Recapitulation.
Rebel's search misses tactical shots and not only probably alters drastically the style of the engine, but influences its results vastly. With a similar net, Rebel 14 was 100 Elos weaker than Marvin/Rebel. Solving the problems with the search will probably not yield so much Elos, but could well hugely improve the results.
Rebel still lacks technical, robust endgame knowledge. It is not only about bishops of different colors -- there, the situation is much better -- but it seems it is simply not trained on enough ending positions.
Berserk dev. version has recently added some simple endgame knowledge, by-passing the NNUE eval for these cases.
It's all "known" ideas, usually found in HCE engines in one form or another, typically Stockfish and Ethereal, probably many others. Possibly overkill to use it with a NNUE engine, because (theoretically) the NNUE ought to know these things anyway. I guess if one's engine screwed up an ending in a tournament, one might be tempted to program it in, or it might allow for slightly earlier termination when generating training games. Dunno. Anyway, I have no plans. I suspect Ed might have though.
adminx and Dio like this post
Admin Admin
Posts : 2608 Join date : 2020-11-17 Location : Netherlands
Subject: Re: Release Rebel 15x2 Sun Jun 12, 2022 11:58 pm
Dio wrote:
it took a while until the games of Rebel 15x2 were evaluated, because our webmaster was very busy, the official first entry of Rebel 15x2 in the 40/4 CEGT list. More results will follow...