ProDeo
Would you like to react to this message? Create an account in a few clicks or log in to continue.
ProDeo

Computer Chess
 
HomeHome  CalendarCalendar  Latest imagesLatest images  FAQFAQ  SearchSearch  MemberlistMemberlist  UsergroupsUsergroups  RegisterRegister  Log in  

 

 News about Rebel or Chess System Tal ?

Go down 
+19
Eelco
Holofernes
TheSelfImprover
Damir Desevac
mwyoung
Wolfgang
texium
Alex-Microsmeta
pohl4711
Peter Berger
peter
gerhard
Werner
matejst
Chris Whittington
Ipmanchess
Mclane
Admin
Dio
23 posters
Go to page : Previous  1, 2, 3, 4, 5, 6, 7, 8  Next
AuthorMessage
Chris Whittington




Posts : 1254
Join date : 2020-11-17
Location : France

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyTue Jan 03, 2023 3:42 pm

Dio wrote:
there were first tests by CEGT. The GUI used was the Banksia GUI. This GUI has no timemargin setting. In 100 games (TC: 4 Minutes + 2 Seconds) against Ethereal 13.75 there were 16 time losses.

How do we deal with this problem?

We could simply take the time controls values that the GUI sends to the engine and factor them down by 0.95 or something and work from those.

Or Ed suggested some time ago that the engine always insists on maintaining a time reservoir of 1000ms or similar.

Or we add a UserOption to do ether of the above. Nothing seems very satisfactory.
Back to top Go down
Dio




Posts : 213
Join date : 2021-08-28

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyTue Jan 03, 2023 3:46 pm

This is exactly what I would suggest. Rebel 16.0 did not have these problems, but it only used about 70 to 75 percent of the thinking time.

I would ideally use only 95 percent of the thinking time. Then you have a certain "security" not to lose by time, the "elo loss" due to these 5 percent should be max. 1 to 2 elo.

Admin likes this post

Back to top Go down
Damir Desevac

Damir Desevac


Posts : 322
Join date : 2020-11-27
Age : 43
Location : Denmark

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyTue Jan 03, 2023 5:28 pm

Just take a look at Stockfish code and add Move Overhead, Slow Mover and Minimum Thinking Time and Rebel will be fine.. Smile Smile
Back to top Go down
Admin
Admin
Admin


Posts : 2571
Join date : 2020-11-17
Location : Netherlands

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyTue Jan 03, 2023 5:43 pm

Dio wrote:
This is exactly what I would suggest. Rebel 16.0 did not have these problems, but it only used about 70 to 75 percent of the thinking time.

I would ideally use only 95 percent of the thinking time. Then you have a certain "security" not to lose by time, the "elo loss" due to these 5 percent should be max. 1 to 2 elo.

That's not a bad suggestion at all. But it remains odd the engine is victimized by the GUI.

Dio likes this post

Back to top Go down
http://rebel13.nl/
Chris Whittington




Posts : 1254
Join date : 2020-11-17
Location : France

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyTue Jan 03, 2023 5:50 pm

Damir Desevac wrote:
Just take a look at Stockfish code and add Move Overhead, Slow Mover and Minimum Thinking Time and Rebel will be fine.. Smile Smile

Move overhead and minimum thinking time are fairly self explanatory but WTF is slow mover?
Back to top Go down
Ipmanchess




Posts : 42
Join date : 2022-06-08

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyTue Jan 03, 2023 5:51 pm

Slow Mover type spin default 100 min 10 max 1000
Lower values will make Stockfish take less time in games, higher values will make it think longer.
Back to top Go down
Chris Whittington




Posts : 1254
Join date : 2020-11-17
Location : France

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyTue Jan 03, 2023 5:58 pm

Admin wrote:
Dio wrote:
This is exactly what I would suggest. Rebel 16.0 did not have these problems, but it only used about 70 to 75 percent of the thinking time.

I would ideally use only 95 percent of the thinking time. Then you have a certain "security" not to lose by time, the "elo loss" due to these 5 percent should be max. 1 to 2 elo.

That's not a bad suggestion at all. But it remains odd the engine is victimized by the GUI.

Worse, victimised by lowest common denominator GUI.
Actually, any GUI without timemargin is badly designed and unsuitable for engine-engine tournament testing.
We should publicise which GUIs are well designed and which not.

Well-designed, suitable for engine testing:
Cutechess

Bad designed, unsuitable for engine testing:
Fritz, Banksia

Back to top Go down
Chris Whittington




Posts : 1254
Join date : 2020-11-17
Location : France

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyTue Jan 03, 2023 6:10 pm

Ipmanchess wrote:
Slow Mover type spin default 100 min 10 max 1000
Lower values will make Stockfish take less time in games, higher values will make it think longer.

User options are also a possible solution.

I’ve left my version on test while I’m out, but it begins to look like it will be very difficult to deal with odd rogue cases. For example, when engine fires up a timeout, it needs to unwind itself back to the root (possibly wait for other threads to unwind) and then send bestmove e2e4 or whatever. Unwinding takes time. If if if there’s high concurrency and/or windows performing itself elsewhere, then the unwinding time can find itself being stretched out by CPU time slicing between threads, won’t happen always, but can happen badly sometimes which is enough to give random spurious time forfeits. We don’t know when this might happen and if we want to safely stop it, then we need a higher than normally necessary overhead to apply to ALL moves. We’re into bad lowest common denominator situation.

I’m minded to stick with 250ms timemargin and give user options to modify various time control parameters as mentioned in another post (SF method).
Back to top Go down
Admin
Admin
Admin


Posts : 2571
Join date : 2020-11-17
Location : Netherlands

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyTue Jan 03, 2023 9:09 pm

It's just that programmers apparently as easy sheep have agreed to fix problems created by others (GUI, Windows, other stuff running in the background) at the cost of hard-fought elo, see my (and Chris) arguments at Talkchess. A chess GUI should be tolerant. It should either have a builtin tolerance or offer a move-overhead option at the cost of the GUI.

Back to top Go down
http://rebel13.nl/
Eelco

Eelco


Posts : 232
Join date : 2021-10-08

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyTue Jan 03, 2023 10:33 pm

I only ever used the Shredder GUI for longer time testgames, 5 minutes per game or three minutes per game, with no increment. Never time losses because I think Shredder just gives a bonus that works like some increment. But you have to trust the GUI to do it fairly. Cutechess can test with very small values for Move Overhead (the Stockfish UCI option), because it reacts so quickly. Move Overhead as an UCI option in Stockfish is mainly intended to deal with time lag on servers. Or not so reliable GUIs like in TCEC tournaments. Stockfish lost games in TCEC, on time, because the Move Overhead was tiny as used in Fishtest. It is increased now whenever an official version is made, or a TCEC contender. But Move Overhead (in the engine) is not such a good parameter to use for the internal time management IMO. It is to compensate for the external situations that you have no control over as an engine. Like the clock used by the GUI that may differ from the internal clock, things like that.
Back to top Go down
Admin
Admin
Admin


Posts : 2571
Join date : 2020-11-17
Location : Netherlands

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyWed Jan 04, 2023 12:06 am

Eelco wrote:
Move Overhead as an UCI option in Stockfish is mainly intended to deal with time lag on servers. Or not so reliable GUIs like in TCEC tournaments. Stockfish lost games in TCEC, on time, because the Move Overhead was tiny as used in Fishtest.

Oh really, what a bunch of amateurs.

I am not willing to give in to this kind amateurism, there will be no fix. Patches are for bugs, patches are not for well working engines that lower elo. Let each take care of their shortcomings.

Eelco likes this post

Back to top Go down
http://rebel13.nl/
Eelco

Eelco


Posts : 232
Join date : 2021-10-08

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyWed Jan 04, 2023 4:35 am

Time losses in TCEC because of a difference of maybe 100 msecs was of course humiliating. I do not remember the exact difference, or when it was. I do think it happened. Elo increase at Cutechess millions of game testing by more aggressive timing, how much of that remains in CEGT? I have no idea but a few Elo points does not sound strange. 16 losses hurt.

(Edit: If those losses are counted of course, but I think they are not counted because it is not fair to other engines that do not play Rebel, they get no free points...? It was a long time ago from the time losses but if you want I can look it up Ed. Here is a Fishcooking (Marco's Google mailing list where most of the informal discussions were) discussion just as an example, between Ivan and Ronald de Man where Ronald suddenly had time losses too on 6 threads with Cfish, the suggestion was to increase slightly Move Overhead but Ivan also says here for TCEC Stockfish had to use Move Overhead 1000 miliseconds (this was also after the incident) and that this is no regression at those time controls. But of course it should be possible to prevent a loss on time that is caused this way (lagging GUI or network lag) by other means (than Move Overhead). Having a time reserve of one second will have maybe some impact if it is a large portion of total time but it becomes less of an impact in CEGT timecontrols. Buf if there is a possible time lag of a second every move, then you can move instantly but you will still lose a second per move, so then a time reserve of one second is not going to be enough. One second per move seems very large, but I'm not sure what the implementation of Move Overhead exactly was. I'm sure somebody here knows it and can tell you, but you guys maybe don't want to look at the code.) (Edit: I do think Move Overhead is meant as possible loss per move? So every move this is calculated in. As far as I can see, but you have to look at the code, it is not written up elsewhere, at least I could not find this, clearly. If the time lag is this large, the time losses also become more understandable.)

Quote :
Or Ed suggested some time ago that the engine always insists on maintaining a time reservoir of 1000ms or similar.

I do not remember what it was in Fruit but this seems basic if you do not have it yet and should take care of simple timepressure.
Back to top Go down
Dio




Posts : 213
Join date : 2021-08-28

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyWed Jan 04, 2023 9:00 am

@Elco: a moveoverhead of 1000 (ms) refers to the whole game, not to a single move.
Back to top Go down
Eelco

Eelco


Posts : 232
Join date : 2021-10-08

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyWed Jan 04, 2023 9:16 am

Quote :

@Elco: a moveoverhead of 1000 (ms) refers to the whole game, not to a single move

Thanks Dio! That sounds a lot more doable then, to implement something for that, without Elo loss. If the programmers want to, that is. I did think it was for the whole game originally, and then for Stockfish to lose on time in TCEC , if it had just reserved another second total, somewhere in its timemanagement, it would have been safe. I do think that was embarrassing.
Back to top Go down
Chris Whittington




Posts : 1254
Join date : 2020-11-17
Location : France

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyWed Jan 04, 2023 11:17 am

A problem with time reservoirs and time margins is they are absolute values. Absolute values can’t work for both short and long time controls. How do you make a time reservoir for game in 1 second with 20 second increments? And so on. Of course, you can, by tweaking this and that and testing, but it’s not what an engine programmer needs or wants to spend his time on. It remains a problem of contradiction between gui time and engine time by interface and communication delays over which the engine has no control or knowledge. Thus it’s a tester and gui problem. Testers should establish that an engine is not deliberately or otherwise egregiously going over time and then use a realistic timemargin. If their GUI has no time margin, then they should ditch it and use one that has.

Overall objective is good chess, establish no cheating and then trust the engine programmer timings. Else we are in the realm of gui terrorism.

Admin likes this post

Back to top Go down
Admin
Admin
Admin


Posts : 2571
Join date : 2020-11-17
Location : Netherlands

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyWed Jan 04, 2023 11:28 am

Chris Whittington wrote:
A problem with time reservoirs and time margins is they are absolute values. Absolute values can’t work for both short and long time controls. How do you make a time reservoir for game in 1 second with 20 second increments? And so on. Of course, you can, by tweaking this and that and testing, but it’s not what an engine programmer needs or wants to spend his time on. It remains a problem of contradiction between gui time and engine time by interface and communication delays over which the engine has no control or knowledge. Thus it’s a tester and gui problem. Testers should establish that an engine is not deliberately or otherwise egregiously going over time and then use a realistic timemargin. If their GUI has no time margin, then they should ditch it and use one that has.

Overall objective is good chess, establish no cheating and then trust the engine programmer timings. Else we are in the realm of gui terrorism.

That.
Back to top Go down
http://rebel13.nl/
Ray




Posts : 38
Join date : 2020-11-26

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyWed Jan 04, 2023 1:30 pm

Has Rebel 16.1 been confirmed to be able to play from the 324 Chess324 positions ? The idea is that every engine that plays standard chess can do so, although I think there was at least one engine that required a fix. I guess I can try it and see.
Back to top Go down
Chris Whittington




Posts : 1254
Join date : 2020-11-17
Location : France

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyWed Jan 04, 2023 3:22 pm

Ray wrote:
Has Rebel 16.1 been confirmed to be able to play from the 324 Chess324 positions ?  The idea is that every engine that plays standard chess can do so, although I think there was at least one engine that required a fix. I guess I can try it and see.

There's no special code to account for any special positions. Nor any special positions in the train files. If it can read the EPD's or PGN's then it will be able to play.
Back to top Go down
Ray




Posts : 38
Join date : 2020-11-26

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyThu Jan 05, 2023 12:54 am

Chris Whittington wrote:
Ray wrote:
Has Rebel 16.1 been confirmed to be able to play from the 324 Chess324 positions ?  The idea is that every engine that plays standard chess can do so, although I think there was at least one engine that required a fix. I guess I can try it and see.

There's no special code to account for any special positions. Nor any special positions in the train files.  If it can read the EPD's or PGN's then it will be able to play.

It plays but loses 1 game in 5 on time so is unsuitable for my tournament. CuteChessGUI, concurrency 10, 500ms margin. Blitz 2'+1". All the other engines have played over 5,000 games without a single time loss.


Last edited by Ray on Thu Jan 05, 2023 12:56 am; edited 1 time in total

Mclane likes this post

Back to top Go down
Mclane

Mclane


Posts : 3011
Join date : 2020-11-17
Age : 57
Location : United States of Europe, Germany, Ruhr area

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyThu Jan 05, 2023 12:56 am

Time controls are broken

Ray likes this post

Back to top Go down
http://www.thorstenczub.de
Chris Whittington




Posts : 1254
Join date : 2020-11-17
Location : France

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyThu Jan 05, 2023 1:11 am

Ray wrote:
Chris Whittington wrote:
Ray wrote:
Has Rebel 16.1 been confirmed to be able to play from the 324 Chess324 positions ?  The idea is that every engine that plays standard chess can do so, although I think there was at least one engine that required a fix. I guess I can try it and see.

There's no special code to account for any special positions. Nor any special positions in the train files.  If it can read the EPD's or PGN's then it will be able to play.

It plays but loses 1 game in 5 on time so is unsuitable for my tournament. CuteChessGUI, concurrency 10, 500ms margin. Blitz 2'+1".  All the other engines have played over 5,000 games without a single time loss.

Weird.

Blitz 2'+1" is 120+1 in cutechess-ese?? (120 secs for game, 1000ms increment?)

How many cores?

If I test myself at those settings, do I need a special book for the 324?

Ah! Thought .... do you test play direct off the opening positions? My tests are mostly of book lines (UHO eg) starting 8 or more moves deep.
Back to top Go down
Ray




Posts : 38
Join date : 2020-11-26

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyThu Jan 05, 2023 2:58 am

120 for the game +1 second per move, so Fischer time control. Plus 500ms margin. Yes, 324 plays direct from a starting position. No book moves, like FRC in that respect (only). Concurrency of 10 on a 10 core machine that supports 20 threads but I always limit myself to the number of real cores. So plenty of resource to handle any background windows processes. Nothing else running at the same time bar anything that Windows may be doing. Had it on my second machine also, same but 12 threads on a 12/24 machine. A pity because I think it was performing very strongly. I guess I could set the margin to 1000ms, but the point is no other engine that I use requires that.
Back to top Go down
Admin
Admin
Admin


Posts : 2571
Join date : 2020-11-17
Location : Netherlands

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyThu Jan 05, 2023 7:49 am

Mclane wrote:
Time controls are broken

In Arena?

Please tell me more.
Back to top Go down
http://rebel13.nl/
Admin
Admin
Admin


Posts : 2571
Join date : 2020-11-17
Location : Netherlands

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyThu Jan 05, 2023 9:17 am

Ray wrote:
120 for the game +1 second per move, so Fischer time control. Plus 500ms margin.  Yes, 324 plays direct from a starting position. No book moves, like FRC in that respect (only). Concurrency of 10 on a 10 core machine that supports 20 threads but I always limit myself to the number of real cores. So plenty of resource to handle any background windows processes. Nothing else running at the same time bar anything that Windows may be doing. Had it on my second machine also, same but 12 threads on a 12/24 machine. A pity because I think it was performing very strongly.

I did a run with your 120+1 time control, timemargin=1000, using 5 threads. It ran on a busy PC, training a new net taking 6 threads, thus in total 11 of the 12 were in use. And no single forfeit.

Code:
Engine                 Depth       Time   Games     Moves  Average Forfeit  Book Depth     MIDG   EARLY    ENDG    LATE
rebel-16.1             24.89    4:30:57     101      6849    2.37     0      891  8.82    22.68 | 23.56 | 25.29 | 29.00
rebel-16a              23.27    4:10:34     101      6859    2.19     0      899  8.90    21.28 | 22.15 | 23.44 | 27.17

So to play in your tournament I have to give up my perfect working NNUE development system where I can train a net and test it at the same time and surrender to faulty system that somehow slipped in during the years. A time margin is NORMAL, see Stefan he even uses 2 seconds.  I am not going to bow down to a system that lowers my development time with 50%.

Ray wrote:
I guess I could set the margin to 1000ms, but the point is no other engine that I use requires that.

Let's put that into perspective, game in 120+1 with an average game length of (say) 80 moves means 200 seconds per game. Your timemargin is 500 (I am glad you use it) but what means half a second in comparison with a game that last 200 seconds?

I did the same 120+1 run on my other Ryzen, it has 24 threads, 12 threads in use for training a new net, 10 threads for cutechess, timemargin 1000ms. In total 22/24 threads in use, no single time forfeit.

Here is an advice, you have a 20 thread PC and only use 10 threads. Try 18 threads with a timemargin=1000 and see what happens, it might speed-up your testing tremendously.
Back to top Go down
http://rebel13.nl/
Ray




Posts : 38
Join date : 2020-11-26

News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 EmptyThu Jan 05, 2023 9:31 am

OK so Pohl uses 2 seconds, that explains why no forfeits for him. I don't like that myself, but each to their own.

I'll increase to 1000ms if that is what is needed. It is a tiny amount of time as you say, and a sensible compromise.

Admin likes this post

Back to top Go down
Sponsored content





News about Rebel or Chess System Tal ? - Page 7 Empty
PostSubject: Re: News about Rebel or Chess System Tal ?   News about Rebel or Chess System Tal ? - Page 7 Empty

Back to top Go down
 
News about Rebel or Chess System Tal ?
Back to top 
Page 7 of 8Go to page : Previous  1, 2, 3, 4, 5, 6, 7, 8  Next
 Similar topics
-
» Using Rebel 15X2-AVX2 in Infinity Chess too complicated for me.
» New Release Announcement Rebel-14-Chess-System-Tal [NNUE]
» Is stockfish on old hardware stronger then genius, rebel, chess tiger, ….
» Intuitive Chess Match: Rebel Century CCRL Rated 2542 vs Lc0 Neural Network Play Only (0 Calculation)
» News

Permissions in this forum:You cannot reply to topics in this forum
ProDeo :: Computer Chess-
Jump to: