questions about lc0

mwyoung wrote:

In your Chessable project. Make sure you are using as needed Chessbase 17 Deep Position analysis feature. This search can often correct mistakes in Stockfish's analysis. Finding correct moves and better evaluations. That Stockfish alone can not understand with just a long analysis search running for hours, or days alone.

Well... It's very time consuming - I've been working on position myself - There are no blunders.
I cannot do this test my current hardware when I only have Leela installed, but I go back and forth while I add explanations to suggested variations. Also, this deep analysis prevents from me really working the position.
I can share with you some of my analysis files if you want to see the quality and run this test.

Btw, Which AMD Thread Ripper do people use in this forum ?
The variety and pricing jumps are sick ....  64 cores the new Thread-Ripper also the 3090X and 3970X are still super super expensive.
I see that you relate no just look at number of cores (8 16 32)
So which type of Threat-Ripper is considered fastest then (and more affordable) best for Stockfish evaluation ?
Can you specify the exact type ?

My configuration of my PC.

CPU: AMD Threadripper ( which one ?)


I did your test  ( I could manage to upload the foto so I made an instagram account designed for this forum ... sorry my low tech abilities - that was my solution)


https://www.instagram.com/p/CmtmAv7I763/?utm_source=ig_web_copy_link


This is my ms: 59811

What is a good normal Ms for long hours ?
I guess it cannot drop below 130 to get a high quality analysis ?
What is your number ?

The build pictured in the posting is built with a 2950x.

But what makes sense for the best one for you to buy. If the budget allows it. Go with the 5900 series Theadrippers.

The best way to know is build some PC designs. With PC part picker. And see the price for the complete system.

And there are other options other then a Threadripper CPU.
Build some PCs with a high end desktop CPU. That will work for you also in some PC builds.



.

What do you say then about my intel 12290 miserable almost 60 k Ms
It's pretty low, isn't it ? (cannot reach precise evaluation too quickly and too steady)

mwyoung wrote:

The build pictured in the posting is built with a 2950x.

But what makes sense for the best one for you to buy. If the budget allows it. Go with the 5900 series Theadrippers.

The best way to know is build some PC designs. With PC part picker. And see the price for the complete system.

And there are other options other then a Threadripper CPU.
Build some PCs with a high end desktop CPU. That will work for you also in some PC builds.



.

It does not even appear there !!
The max they have is 3990X https://www.instagram.com/p/Cmtvzj4IrW-/ and its alone costs 8184$ - I have to dismiss it from start! I don't have these sums of money at this point. :-(

Btw,
Mwyoung, you have no idea how appreciate your time to discuss and explain this stuff to me !
I am really grateful for your help !  - This bench mark test I have no idea about it existence!

This is the list of Chessbase providers of pcs for rent (mainly Stockfish) - do you think the MS they claim to have isn't the real MS ? (The idea isn't to defame anyone! - just to be clear) - The idea is to learn this issue which is so important to chess players!

The list of Chessbase users who offer service and their claimed MS
https://www.instagram.com/p/CmtxY2FIhBW/

Sinemora wrote:

Btw,
Mwyoung, you have no idea how appreciate your time to discuss and explain this stuff to me !
I am really grateful for your help !  - This bench mark test I have no idea about it existence!

This is the list of Chessbase providers of pcs for rent (mainly Stockfish) - do you think the MS they claim to have isn't the real MS ? (The idea isn't to defame anyone! - just to be clear) - The idea is to learn this issue which is so important to chess players!

The list of Chessbase users who offer service and their claimed MS
https://www.instagram.com/p/CmtxY2FIhBW/

It does not matter if their NPS is true or not. That is not the issue.

The issue is no matter what number they are claiming. The number is meaningless for the renter.

And the issue is not from the people renting.

I will say again, as I have already said in this post. NPS between 2 computers is irrelevant. Do not be fooled! That is not how you measure the speed differences of the computers.

Here is what the vast majority of people really think when seeing the NPS numbers you posted.

Chessify running Stockfish 15.1 nodes per second is 364Mnps vs the renters home computer running Stockfish 15.1 is 10 Mnps. And they think Wow! Chessify computer is over 30 times faster then my home computer. I am renting the Chessify computer for some really strong analysis.

As most people think you measure the speed of the two systems by the NPS.  

To know the true speed difference between the Chessify computer, and your home computer running Stockfish 15.1. You will need to do some test positions on both computers, and measure the time to depth for those test positions.

And the Chessify computer will not be 30x faster, or 20x faster or 10x faster or 5x faster and it may not even be 2x faster when running Stockfish.



.

Sinemora wrote:

What do you say then about my intel 12290 miserable almost 60 k Ms
It's pretty low, isn't it ? (cannot reach precise evaluation too quickly and too steady)

Do a benchmark on your computer with stockfish, and give what stockfish you are doing the test with for the benchmark test. Meaning version number of Stockfish as that matters, and the Settings used for the benchmark test.

Post the results, and I can give you an answer.

mwyoung wrote:

Sinemora wrote:

Btw,
Mwyoung, you have no idea how appreciate your time to discuss and explain this stuff to me !
I am really grateful for your help !  - This bench mark test I have no idea about it existence!

This is the list of Chessbase providers of pcs for rent (mainly Stockfish) - do you think the MS they claim to have isn't the real MS ? (The idea isn't to defame anyone! - just to be clear) - The idea is to learn this issue which is so important to chess players!

The list of Chessbase users who offer service and their claimed MS
https://www.instagram.com/p/CmtxY2FIhBW/

It does not matter if their NPS is true or not. That is not the issue.

The issue is no matter what number they are claiming. The number is meaningless for the renter.

And the issue is not from the people renting.

I will say again, as I have already said in this post. NPS between 2 computers is irrelevant. Do not be fooled! That is not how you measure the speed differences of the computers.

Here is what the vast majority of people really think when seeing the NPS numbers you posted.

Chessify running Stockfish 15.1 nodes per second is 364Mnps vs the renters home computer running Stockfish 15.1 is 10 Mnps. And they think Wow! Chessify computer is over 30 times faster then my home computer. I am renting the Chessify computer for some really strong analysis.

As most people think you measure the speed of the two systems by the NPS.  

To know the true speed difference between the Chessify computer, and your home computer running Stockfish 15.1. You will need to do some test positions on both computers, and measure the time to depth for those test positions.

And the Chessify computer will not be 30x faster, or 20x faster or 10x faster or 5x faster and it may not even be 2x faster when running Stockfish.



.

I am not sure if time to depth is the correct test.
I think time to find the right move in some positions when it does not find it in a very short time is the correct test.

The problem with nodes is that the search with smp is not efficient.
For example 128 threads may search 128M nodes at the same time that 1 thread search 1M nodes but it is not equivalent to being 128 times faster because
different threads may search the same nodes or uesless nodes.

Inspite of it I find it hard to believe that 364M nodes is even not equivalent to being twice faster than 10M nodes and
I am not sure if the same depth means the same thing for stockfish with different machines.

The best way to test computer speed is fixed depth matches with single core engines and concurrency equal to the max number of threads.

Ozymandias wrote:

The best way to test computer speed is fixed depth matches with single core engines and concurrency equal to the max number of threads.

And I have done all those test! Check them out here on Pro Deo!

Uri Blass wrote:

mwyoung wrote:

Sinemora wrote:

Btw,
Mwyoung, you have no idea how appreciate your time to discuss and explain this stuff to me !
I am really grateful for your help !  - This bench mark test I have no idea about it existence!

This is the list of Chessbase providers of pcs for rent (mainly Stockfish) - do you think the MS they claim to have isn't the real MS ? (The idea isn't to defame anyone! - just to be clear) - The idea is to learn this issue which is so important to chess players!

The list of Chessbase users who offer service and their claimed MS
https://www.instagram.com/p/CmtxY2FIhBW/

It does not matter if their NPS is true or not. That is not the issue.

The issue is no matter what number they are claiming. The number is meaningless for the renter.

And the issue is not from the people renting.

I will say again, as I have already said in this post. NPS between 2 computers is irrelevant. Do not be fooled! That is not how you measure the speed differences of the computers.

Here is what the vast majority of people really think when seeing the NPS numbers you posted.

Chessify running Stockfish 15.1 nodes per second is 364Mnps vs the renters home computer running Stockfish 15.1 is 10 Mnps. And they think Wow! Chessify computer is over 30 times faster then my home computer. I am renting the Chessify computer for some really strong analysis.

As most people think you measure the speed of the two systems by the NPS.  

To know the true speed difference between the Chessify computer, and your home computer running Stockfish 15.1. You will need to do some test positions on both computers, and measure the time to depth for those test positions.

And the Chessify computer will not be 30x faster, or 20x faster or 10x faster or 5x faster and it may not even be 2x faster when running Stockfish.



.

I am not sure if time to depth is the correct test.
I think time to find the right move in some positions when it does not find it in a very short time is the correct test.

The problem with nodes is that the search with smp is not efficient.
For example 128 threads may search 128M nodes at the same time that 1 thread search 1M nodes but it is not equivalent to being 128 times faster because
different threads may search the same nodes or uesless nodes.

Inspite of it I find it hard to believe that 364M nodes is even not equivalent to being twice faster than 10M nodes and
I am not sure if the same depth means the same thing for stockfish with different machines.

Here is my advice.

Almost everyone has a computer with more then 1 CPU core.

Everyone in the world can discover for themselves how Stockfish behaves as you add threads.

I suggest you do so before wasting your money!

Stockfish has a built in benchmark. That can be used in many ways to do some quick test in many ways.

mwyoung wrote:

Ozymandias wrote:

The best way to test computer speed is fixed depth matches with single core engines and concurrency equal to the max number of threads.

And I have done all those test! Check them out here on Pro Deo!

Over half a billion games worth of testing?

Ozymandias wrote:

mwyoung wrote:

Ozymandias wrote:

The best way to test computer speed is fixed depth matches with single core engines and concurrency equal to the max number of threads.

And I have done all those test! Check them out here on Pro Deo!

Over half a billion games worth of testing?

Then I would just reject every thing I said. And start renting some fast NPS systems for some really good and powerful analysis.

Or if your smart. Test it for yourself!

I don't understand your answer. My question was a "yes or no" one, which option did you choose?

Well guys,
I cannot run these tests (simply out of time). There are more capable guys than me to run multiple tests...
At the end of the day, I want to best possible tool to create the best opening repertoires according to my knowledge and chess understanding. For example, That is what I am creating now against 1.e4 with an updated Petroff Repertoire
I need rely on other capable guys (who understand about computers, Chess engines and hardware way more than I do) and rely on their advice as for What hardware to use  (with correct parameters)

I don't mind sharing my analysis with guys like Mwyoung or others here (as Betatesters) I don't know people's actual interest in opening theory per see ( or chess at all) or just applications (they are more like Engines oriented AI investigators ) to examine the quality of my analysis.

Actually I don't really get the very point of your question(s). I share your interest in opening theory and digging into lines with the ideas of computer engines, other people and myself.

As someone pointed out somewhere else the difference in hardware performance has become nearly irrelevant when it is about this hobby, as we are clearly experiencing right now what used to be called "diminishing returns" and was wildly debated back some time. If you are able to rely on your own judgement at certain points, which I assume you can as a Fide master, and you run your whatever computer on a position for an hour, don't expect something completely different if you run it for 24 hours - this won't happen often enough to make the additional money spent on the faster computer worthwhile IMHO.

Hardware will still matter if you reduce thinking time +a lot+ and run completely automated tests on opening books ( Ed has done some of this, so you can read about it here somewhere), but for how I imagine your project to be from what you wrote, save the money or spend it for poor people instead would be my advice.

Peter Berger wrote:

Actually I don't really get the very point of your question(s). I share your interest in opening theory and digging into lines with the ideas of computer engines, other people and myself.

As someone pointed out somewhere else the difference in hardware performance has become nearly irrelevant when it is about this hobby, as we are clearly experiencing right now what used to be called "diminishing returns" and was wildly debated back some time. If you are able to rely on your own judgement at certain points, which I assume you can as a Fide master, and you run your whatever computer on a position for an hour, don't expect something completely different if you run it for 24 hours - this won't happen often enough to make the additional money spent on the faster computer worthwhile IMHO.

Hardware will still matter if you reduce thinking time +a lot+ and run completely automated tests on opening books ( Ed has done some of this, so you can read about it here somewhere), but for how I imagine your project to be from what you wrote, save the money or spend it for poor people instead would be my advice.

This is the great blessing that Stockfish has given the chess world. It is true that Stockfish scales like dog shit.

So where is the blessing....

Because despite this Stockfish is still the best chess engine. And you get great analysis with Stockfish on very weak hardware. To the point of my phone running the current Stockfish is stronger then every chess engine running other then Dragon and Leela. And is even better then any stockfish before stockfish nnue. When the other chess engines are running on my Threadripper vs my phone.

You do not need massive hardware anymore with Stockfish.

Just look at my crazy results in my testings. Where Stockfish played at 3m+2s against Lc0 and Dragon playing Stockfish with 40 moves in 2hours.

Chess users are drunk on nodes per second. Without ever testing what these numbers really mean.

The ultimate chess computer! 25 years ago, till today.

Sinemora wrote:

One note from about the Strength of LeelaCzero.
Obviously in terms of total objective strength Stockfish has won the battle. However, for the aspiring Chess player (especially below 2100) Leela is super super important !
Stockfish (especially the latest 15.1 is simply so impressively tactically strong) that no human could ever become. My conclusion that Leela can offer very very original solutions (much more "intutiive" and easy to find) and what happens is the following : in 80-85 % of time Leela Evaluations turn out to be "more accurate" at first sight. What I mean by that is that Stockfish takes more more time and high-depth to lower his advantage and actually agree with "Leela" First instincts ...I've been witnessing this tendency over and over again. Sometimes Leela misevaluate tactical shots and king safety and her line is refuted, but in the vast vast majority of the cases, Stockfish corrects his evaluation to Leela's first seconds evaluation. This tendency is quite amazing and it what makes Leela so so important for a human player (especially lower rated club level player) who simply cannot never reach to capacity to justify or understand Stockfish brute force solutions.  It's not about positional, but about practicality !   Leela is much much practical than stockfish even in complicated positions - This is why it is so so important tool loved by higher rated players...  

I did not want to bring this disturbing topic up. But you really need to understand what you are seeing with the Stockfish and Lc0 evaluations you are using.

They are fake evaluations. That can be changed by the engine programmer. And have been changed by the chess engine programmer. And can be changed tomorrow by the chess engine programmer. Without changing the play of the chess engine itself.

And the formula to generate these fake evaluations are not the same for Lc0 and Stockfish.

Lc0 and Stockfish do not use centipawns in evaluating chess positions. And most position for Stockfish. As Stockfish is a hybrid chess engine.

To see Stockfish and Lc0 real evaluations in Chessbase 17. You must activate Show Win percentage. In the engine option window.

mwyoung wrote:

Sinemora wrote:

One note from about the Strength of LeelaCzero.
Obviously in terms of total objective strength Stockfish has won the battle. However, for the aspiring Chess player (especially below 2100) Leela is super super important !
Stockfish (especially the latest 15.1 is simply so impressively tactically strong) that no human could ever become. My conclusion that Leela can offer very very original solutions (much more "intutiive" and easy to find) and what happens is the following : in 80-85 % of time Leela Evaluations turn out to be "more accurate" at first sight. What I mean by that is that Stockfish takes more more time and high-depth to lower his advantage and actually agree with "Leela" First instincts ...I've been witnessing this tendency over and over again. Sometimes Leela misevaluate tactical shots and king safety and her line is refuted, but in the vast vast majority of the cases, Stockfish corrects his evaluation to Leela's first seconds evaluation. This tendency is quite amazing and it what makes Leela so so important for a human player (especially lower rated club level player) who simply cannot never reach to capacity to justify or understand Stockfish brute force solutions.  It's not about positional, but about practicality !   Leela is much much practical than stockfish even in complicated positions - This is why it is so so important tool loved by higher rated players...  

I did not want to bring this disturbing topic up. But you really need to understand what you are seeing with the Stockfish and Lc0 evaluations you are using.

They are fake evaluations. That can be changed by the engine programmer. And have been changed by the chess engine programmer. And can be changed tomorrow by the chess engine programmer. Without changing the play of the chess engine itself.

And the formula to generate these fake evaluations are not the same for Lc0 and Stockfish.

Lc0 and Stockfish do not use centipawns in evaluating chess positions. And most position for Stockfish. As Stockfish is a hybrid chess engine.

To see Stockfish and Lc0 real evaluations in Chessbase 17. You must activate Show Win percentage. In the engine option window.

I dont want to bring this disturbing topic up, but NN evaluations are actually probabilities (not accurate evaluations), the best you can hope for out of an NN is maybe 90% confidence that the probability is 'accurate'. What you get is a probablity with a confidence rating.

Chris Whittington wrote:

mwyoung wrote:

Sinemora wrote:

One note from about the Strength of LeelaCzero.
Obviously in terms of total objective strength Stockfish has won the battle. However, for the aspiring Chess player (especially below 2100) Leela is super super important !
Stockfish (especially the latest 15.1 is simply so impressively tactically strong) that no human could ever become. My conclusion that Leela can offer very very original solutions (much more "intutiive" and easy to find) and what happens is the following : in 80-85 % of time Leela Evaluations turn out to be "more accurate" at first sight. What I mean by that is that Stockfish takes more more time and high-depth to lower his advantage and actually agree with "Leela" First instincts ...I've been witnessing this tendency over and over again. Sometimes Leela misevaluate tactical shots and king safety and her line is refuted, but in the vast vast majority of the cases, Stockfish corrects his evaluation to Leela's first seconds evaluation. This tendency is quite amazing and it what makes Leela so so important for a human player (especially lower rated club level player) who simply cannot never reach to capacity to justify or understand Stockfish brute force solutions.  It's not about positional, but about practicality !   Leela is much much practical than stockfish even in complicated positions - This is why it is so so important tool loved by higher rated players...  

I did not want to bring this disturbing topic up. But you really need to understand what you are seeing with the Stockfish and Lc0 evaluations you are using.

They are fake evaluations. That can be changed by the engine programmer. And have been changed by the chess engine programmer. And can be changed tomorrow by the chess engine programmer. Without changing the play of the chess engine itself.

And the formula to generate these fake evaluations are not the same for Lc0 and Stockfish.

Lc0 and Stockfish do not use centipawns in evaluating chess positions. And most position for Stockfish. As Stockfish is a hybrid chess engine.



To see Stockfish and Lc0 real evaluations in Chessbase 17. You must activate Show Win percentage. In the engine option window.

I dont want to bring this disturbing topic up, but NN evaluations are actually probabilities (not accurate evaluations), the best you can hope for out of an NN is maybe 90% confidence that the probability is 'accurate'. What you get is a probablity with a confidence rating.

YEP. The point being is the centipawn evals. Are fake, and are just a formula that can be changed and is changed. And more important, the formulas for the 2 engines is not the same.

The real evaluations are statistical probabilities, and have nothing to do with material, space, tempo, or what the pieces are worth or anything else.......

But you can see them as the GUI will display them as W% D% L%

mwyoung wrote:

world-casino-nl.com/aplay

Use this link to build the PC, see if the parts you want will work together, and get the cost of the build, and what size power supply you will need. and links to buy the PC Parts at the cheapest price.

Do not be bound by my build, use PC part picker for your budget, and specs needed for that budget.  

My configuration of my PC.

CPU: AMD Threadripper
GPU: Nvidia RTX 2080ti by MSI
Motherboard: Asus - Stick to this brand of motherboard.
Ram: 64 GB Corsair
SSD: 2TB m.2 Samsung 970 plus.
AIO; Enermax for Threadripper.
Case: Lian Li - Der Bauer High air flow over clocking case.
Fans: 6 Corsair 120mm high air flow, high static pressure fans.
Power supply: 1000 watt gold rated power supply by Corsair.


Stockfish benchmark.

Double Click on the stockfish exe in the folder to generate a dos box.

Type bench (hash) (cpu cores aka threads) (depth of search for the test position)

like this.

bench 1024 4 25

Press enter key.

This will benchmark with 1 GB of hash, using 4 cores, with a search depth of 25.

Be consistent with your search depth. I mostly use a depth of 35 to 40. But the higher the depth the longer the benchmark will take to run. But if your testing for true speed, you want a longer test to get the CPU running as hot as possible. So you can check the speed of your system like when you are doing a long analysis of a position.

Now you can compare a computer's system completion time for different computers with the benchmark time, or the same computers with different hash, and cpu cores when running Stockfish. But just keep the depth consistent when comparing completion times. Completion time of the benchmark is in milliseconds.

Thank you very much, I've been looking for about a week where I can build a PC, I've already read a lot of forums and finally found it.

Tanja_Ratowski wrote:

mwyoung wrote:

https://pcpartpicker.com/

Use this link to build the PC, see if the parts you want will work together, and get the cost of the build, and what size power supply you will need. and links to buy the PC Parts at the cheapest price.

Do not be bound by my build, use PC part picker for your budget, and specs needed for that budget.  

My configuration of my PC.

CPU: AMD Threadripper
GPU: Nvidia RTX 2080ti by MSI
Motherboard: Asus - Stick to this brand of motherboard.
Ram: 64 GB Corsair
SSD: 2TB m.2 Samsung 970 plus.
AIO; Enermax for Threadripper.
Case: Lian Li - Der Bauer High air flow over clocking case.
Fans: 6 Corsair 120mm high air flow, high static pressure fans.
Power supply: 1000 watt gold rated power supply by Corsair.


Stockfish benchmark.

Double Click on the stockfish exe in the folder to generate a dos box.

Type bench (hash) (cpu cores aka threads) (depth of search for the test position)

like this.

bench 1024 4 25

Press enter key.

This will benchmark with 1 GB of hash, using 4 cores, with a search depth of 25.

Be consistent with your search depth. I mostly use a depth of 35 to 40. But the higher the depth the longer the benchmark will take to run. But if your testing for true speed, you want a longer test to get the CPU running as hot as possible. So you can check the speed of your system like when you are doing a long analysis of a position.

Now you can compare a computer's system completion time for different computers with the benchmark time, or the same computers with different hash, and cpu cores when running Stockfish. But just keep the depth consistent when comparing completion times. Completion time of the benchmark is in milliseconds.

Thank you very much, I've been looking for about a week where I can build a PC, I've already read a lot of forums and finally found it.

You are welcome.

And if you are building a system for running Stockfish. Remember what you have read here. On how to properly benchmark a computer for Stockfish. Or you will be like this poor hapless sole. Who built a computer to run Stockfish at maximum nodes per second. But can not buy a win, and is getting owned by Stockfish running on a laptop, and a Raspberry Pi 3 only 50 kns.

"Monster PC hasn't won a single game in over 12 hours!"

The mystery as to why is plain to see. If you ignore NPS like you should. And only look at the Monster PC's time to depth results in the games.

How fast is Stockfish here!?  

https://talkchess.com/forum3/viewtopic.php?f=2&t=81494