pohl4711
![pohl4711](https://2img.net/u/4215/48/12/73/avatars/110-74.jpg)
Posts : 111 Join date : 2022-03-01 Location : Berlin
![SPCC: Rebel 16.2 Evalcorrect-parameter test Empty](https://2img.net/i/empty.gif) | Subject: SPCC: Rebel 16.2 Evalcorrect-parameter test Wed Feb 22, 2023 8:06 am | |
| Experimental testrun of Rebel 16.2 with different values of the Evalcorrect UCI-parameter. This option can be used to change the playing style of the engine. The default value is 202. Increasing the value should increase the engine aggressiveness. A 10000 games RoundRobin tournament was played. 60sec+600ms thinking-time, singlethread, no ponder, no bases, my UHO_2022_8mvs_+120_+129 openings were used. https://www.sp-cc.de/experiments.htm - Code:
-
Program Elo + - Games Score Av.Op. Draws
1 Rebel 16.2 default : 3617 6 6 4000 52.4% 3600 56.1% 2 Rebel 16.2 Ec=256 : 3601 6 6 4000 49.5% 3604 56.2% 3 Rebel 16.2 Ec=300 : 3600 6 6 4000 49.4% 3604 57.2% 4 Rebel 16.2 Ec=500 : 3600 6 6 4000 49.3% 3604 55.9% 5 Rebel 16.2 Ec=400 : 3599 6 6 4000 49.3% 3604 57.0%
Games : 10000 (finished) White Wins : 4190 (41.9 %) Black Wins : 162 (1.6 %) Draws : 5648 (56.5 %)
Below the Engines Aggressiveness Scoring (EAS), calculated with my EAS-Tool (V5.21): - Code:
-
bad avg.win Rank EAS-Score sacs shorts draws moves Engine/player ------------------------------------------------------------------- 1 75453 10.06% 20.43% 20.36% 79 Rebel 16.2 default 2 72790 09.25% 18.51% 20.45% 81 Rebel 16.2 Ec=400 3 71327 09.36% 16.14% 19.71% 81 Rebel 16.2 Ec=500 4 70839 10.50% 18.20% 20.60% 80 Rebel 16.2 Ec=256 5 61260 09.23% 16.55% 21.16% 82 Rebel 16.2 Ec=300 ------------------------------------------------------------------- *** Average length of all won games: 80 moves
Conclusions: The Evalcorrect-parameter seems quite meaningless. As you can see, the strength and the aggressiveness of Rebel are nearly identical with all Evalcorrect-values and a higher Evalcorrect-value seems to lower the aggressiveness instead of increasing it... | |
|
Chris Whittington
Posts : 1254 Join date : 2020-11-17 Location : France
![SPCC: Rebel 16.2 Evalcorrect-parameter test Empty](https://2img.net/i/empty.gif) | Subject: Re: SPCC: Rebel 16.2 Evalcorrect-parameter test Wed Feb 22, 2023 5:05 pm | |
| - pohl4711 wrote:
- Experimental testrun of Rebel 16.2 with different values of the Evalcorrect UCI-parameter. This option can be used to change the playing style of the engine. The default value is 202. Increasing the value should increase the engine aggressiveness.
A 10000 games RoundRobin tournament was played. 60sec+600ms thinking-time, singlethread, no ponder, no bases, my UHO_2022_8mvs_+120_+129 openings were used.
https://www.sp-cc.de/experiments.htm
- Code:
-
Program Elo + - Games Score Av.Op. Draws
1 Rebel 16.2 default : 3617 6 6 4000 52.4% 3600 56.1% 2 Rebel 16.2 Ec=256 : 3601 6 6 4000 49.5% 3604 56.2% 3 Rebel 16.2 Ec=300 : 3600 6 6 4000 49.4% 3604 57.2% 4 Rebel 16.2 Ec=500 : 3600 6 6 4000 49.3% 3604 55.9% 5 Rebel 16.2 Ec=400 : 3599 6 6 4000 49.3% 3604 57.0%
Games : 10000 (finished) White Wins : 4190 (41.9 %) Black Wins : 162 (1.6 %) Draws : 5648 (56.5 %)
Below the Engines Aggressiveness Scoring (EAS), calculated with my EAS-Tool (V5.21):
- Code:
-
bad avg.win Rank EAS-Score sacs shorts draws moves Engine/player ------------------------------------------------------------------- 1 75453 10.06% 20.43% 20.36% 79 Rebel 16.2 default 2 72790 09.25% 18.51% 20.45% 81 Rebel 16.2 Ec=400 3 71327 09.36% 16.14% 19.71% 81 Rebel 16.2 Ec=500 4 70839 10.50% 18.20% 20.60% 80 Rebel 16.2 Ec=256 5 61260 09.23% 16.55% 21.16% 82 Rebel 16.2 Ec=300 ------------------------------------------------------------------- *** Average length of all won games: 80 moves
Conclusions: The Evalcorrect-parameter seems quite meaningless. As you can see, the strength and the aggressiveness of Rebel are nearly identical with all Evalcorrect-values and a higher Evalcorrect-value seems to lower the aggressiveness instead of increasing it... Purpose of eval correct was to try to bring the NNUE eval into accord with the hand coded search/pruning/extension parameters. HCS was basically originally “tuned” to the CSTal HCE-evaluation, whereas the range and scale of the NNUE eval is a little indeterminate, especially when part turned on game result. Whenever I do a new net, evalcorrect gets retuned. Current value is a multiplier of 202, I think. Whatever delivered most Elo. If it is increased or decreased it basically is changing against what the HC-Search think a pawn is worth. Curiously, I found total engine reaction to eval correct to be quite “dull”, big changes didn’t seem to make a lot of difference. I guess if you increase it a lot, then the evals will begin to swamp the search parameter values (with unknown behaviour) and if decreased a lot, then the evals will themselves disappear in search prune/extension effect (more unknown behaviour). Your finding of a little less aggression with increasing EC is useful and also intuitively in accord. A more useful adjust would be to be able to change the relationship between the base material of a position and the NNUE idea of the positional component. CSTal dev already has this feature, although my (limited) testing didn’t find anything better than the vanilla setting. Obviously it’s neither simple, nor “accurate”, extracting a realistic psnl and a mtrl component out of a NNUE eval. | |
|