dinsdag 1 juni 2021

Noomen Low Draw Rate Testset

Hi all,

Here is a new testset for engine-engine testing: Noomen_LowDrawRate_v2.  This is a selection of TCEC superfinal opening lines (S9 through S20) with the following criteria:

  • Leela Chess Zero book exit of +0.35 or higher
  • Good chance of a 1.5-0.5 result between the strongest engines
As the previous two superfinals (S19 and S20) showed, Leela book exits below +0.30 are a certain draw under TCEC superfinal conditions. To get interesting games it is necessary to give the engines something to play with. Hence the minimum book exit of +0.35.

Using this special book for engine-engine testing will lead to:
  • Exciting games
  • More decisive results  
Enjoy testing!

Download:
Noomen_LowDrawRate_v2

zaterdag 8 mei 2021

The TCEC 21 superfinal book, by GM Matthew Sadler and Jeroen Noomen

I am excited to announce that the TCEC 21 superfinal book has been made in cooperation between chess Grandmaster Matthew Sadler and myself (Jeroen Noomen). Matthew is a well respected Grandmaster and the co-author of the great book "Game Changer", together with WGM Natasha Regan. It was an honour to get Matthew on board and I am very pleased with the result!


Here is the most important information about our joint effort:

  • The TCEC 21 superfinal book was mostly completed in April 2021. Both Matthew and I have chosen 25 lines, which will be played alternately. Matthew's lines are #1, #3, #5 and so forth, my lines are #2, #4, #6 and so forth. The lines are mixed, which means that lower book exits are mixed with higher book exits. I chose to mix the book exits, as the experiment with a low-to-high exit in season 20 led to a lot of draws in the first half of the superfinal.
  • To determine the book exits we used Leela Chess Zero and Stockfish. All lines typically have a LC0 book exit between +0.30 through +0.50, with a few exceptions. Of course the book exits at TCEC will be different, because of much faster hardware and different Leela and Stockfish versions. 
  • As usual, all important openings will be played; just like in the previous three TCEC superfinals there will be a lot more risky opening lines compared to TCEC 16 and before. This is necessary to avoid an excessive number of uninteresting draws. Statistics of the previous two superfinals show that a Leela book exit below +0.30 is an almost 100% certain draw. 
  • The chosen lines typically have a length between 3 moves and 13 moves; there are five lines which are 3 moves long and there are three lines which are 13 moves long. This time some openings will be played more often than you are used to in previous superfinals. 
  • Average move length is 6.84, which is is comparable to the TCEC 20 superfinal.
  • There are sharp lines in the set, as well as positional linesmore complex lines, lines with opposite castling ("SALC", i.e. short and long castling), dubious lines, lines with an imbalance in material and gambits. In my (Jeroen) selection I used a couple of lines of Stefan Pohl's "Unbalanced Human Openings".
  • As I mentioned previously, I now prefer 1-0 1-0 results over two straight forward (i.e. boring) draws. In effect the result is the same: 1-1. But two wins give more excitement. The super solid openings played by today's top Grandmasters are a certain draw in today's engine chess, hence they are not suitable for TCEC superfinal purposes.
  • We have avoided positions that could lead to many exchanges, as well as positions with a symmetrical pawn structure and positions that could lead to easy wins for the side with an advantage. The last aspect is becoming more and more difficult, with strong hardware, a long time control and engines being so strong. Of course risky lines means that the chance of 1-0 1-0 is increasing.
  • What do I expect? 20 wins is the minimum I'd like to see, but of course I hope for more wins. In my humble opinion I think that the TCEC 21 superfinal could be the most exciting superfinal ever!

ECO code distribution
ECO A: 10 lines
ECO B: 16 lines
ECO C: 10 lines
ECO D: 8 lines
ECO E: 6 lines

Move length distribution
3 moves: 5 lines
4 moves: 7 lines
5 moves: 4 lines
6 moves: 12 lines
7 moves: 2 lines
8 moves: 7 lines
9 moves: 5 lines
10 moves: 3 lines

maandag 1 februari 2021

TCEC 20 superfinal opening PGN available

 The TCEC 20 superfinal opening PGN is now available for download:

TCEC 20 opening PGN

Stockfish won the superfinal vs Leela Chess Zero by 53-47. Stockfish won 14 games, Leela won 8 games, while 78 games ended in a draw.

Some interesting statistics about the decisive opening pairs that ended 1.5-0.5:

0.36 / 1.05
0.33 / 0.67
0.40 / 0.71
0.40 / 0.91
0.41 / 0.86
0.43 / 0.98
0.42 / 0.79
0.42 / 0.64
0.49 / 1.07
0.65 / 1.15
0.55 / 0.84
0.43 / 1.34
0.52 / 0.90
0.52 / 1.48

These are the book exit evaluations by Leela (mentioned first) and Stockfish (mentioned second) in the opening lines that ended 1.5-0.5. Conclusion can be similar to the S19 superfinal: Leela book exits lower than +0.40 seem to be an almost certain draw. There were only two exceptions, where Leela blundered a drawish position after the book exits had been lower than+0.40.

zaterdag 9 januari 2021

Opening selection TCEC 20 superfinal

Here are Jeroen Noomen's comments on the opening selection for the TCEC 20 superfinal:


"Here are the most important details regarding my selection of the openings for the TCEC 20 superfinal:

  • At www.twitch.tv/navratil25 two interesting matches between LC0 and Stockfish were played recently. The first match was bookless. It ended in 100 draws, with all games being predictable, boring and without any winning chances. The second match was played with more or less balanced lines. It ended in 2-2 with 96 draws. Again many predictable, boring games, in which the engines had nothing no play for. Thus the conclusion is clear: to keep computer chess interesting and avoid a huge number of uninteresting & boring draws, unbalanced openings with higher book exits are absolutely necessary.
  • The TCEC 20 superfinal book was mostly completed in June 2020, pre NNUE. Compared to all my other superfinal books, there has been a big change: the opening with the lowest book exit will be played first, followed by the next slightly higher book exit and so forth, until line 50, which has the highest book exit. This means it is possible that the same opening will be played two times in a row.
  • To determine the book exits I took the average evaluation of Stockfish 12 (5 minutes on my 8 core Ryzen) and LC0 v26.2 J92-100 (2 minutes on my notebook). Of course the book exits at TCEC will be different, because of much faster hardware and different Leela and Stockfish versions. But all in all we will see a superfinal in which the openings will become riskier as the match progresses.
  • As usual, all important openings will be played; just like in the previous two TCEC superfinals there will be a lot more risky opening lines compared to TCEC 16 and before. However, the TCEC 20 superfinal book will be less extreme than the TCEC 18 superfinal book, which was a bit too much in my opinion.
  • In the TCEC 20 superfinal we will see many openings (exactly the same, or slightly different) that have featured in previous superfinals. 
  • The chosen lines typically have a length between 1 move and 12 moves; there are two lines which are only 1 move (i.e. the first white and black move), there is one line with 14 moves of book and one line with 16 moves of book.
  • Average move length is 6.76 (which is 0.32 higher compared to the TCEC 19 superfinal).
  • There are sharp lines in the set, as well as positional linesmore complex lines, lines with opposite castling ("SALC", i.e. short and long castling), dubious lines, lines with an imbalance in material and gambits.
  • As I mentioned in my article about the TCEC 18 superfinal book, I now prefer 1-0 1-0 results over two straight forward draws. In effect the result is the same: 1-1. But two wins give more excitement. 
  • I have avoided positions that could lead to many exchanges and/or have a high draw rate in correspondence chess (I prefer draw rates of 50% or less), as well as positions with symmetrical pawn structures and positions that could lead to easy wins for the side with an advantage. The last aspect is becoming more and more difficult, with strong hardware, a long time control and engines being so strong. Of course risky lines means that the chance of 1-0 1-0 is increasing.
  • What do I expect? 20 wins is the minimum I'd like to see, but of course I hope for more wins. My personal favourite lines are #3, #12, #13, #27, #42 and  #47. Enjoy the superfinal!

ECO code distribution
ECO A: 12 lines
ECO B: 14 lines
ECO C: 11 lines
ECO D: 4 lines
ECO E: 9 lines

Move length distribution
1 move: 2 lines
2 moves: 2 lines
3 moves: 5 lines
4 moves: 8 lines
5 moves: 3 lines
6 moves: 5 lines
7 moves: 5 lines
8 moves: 4 lines
9 moves: 4 lines
10 moves: 6 lines
>10 moves: 6 lines

zaterdag 17 oktober 2020

TCEC 19 superfinal opening PGN available

Jeroen Noomen has made his TCEC 19 superfinal book available for download. His comments:

"The TCEC 19 superfinal ended Friday, October 16th and was won by Stockfish by 54.5-45.5 vs Leela Chess Zero. In wins 18-9, with 73 draws. Just like the S18 superfinal book, the S19 book was way more risky than S16 and before. Nevertheless, the S19 book also had several balanced lines and theoretically interesting lines. Interestingly, all wins were only in (very) dubious openings. Balanced lines and slightly dubious lines had a 100% draw rate. A sign of the times?

You can download the S19 superfinal book from the following location:

http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=33689

Some interesting statistics:

In the Leela wins, the book exit evaluations by Leela were: 0.43 / 0.54 / 0.50 / 0.46 / 0.66 / 0.48 / 0.56 / 0.52 / 0.63

I.e. the lowest book exit that led to a win was 0.43 (which is quite high by Leela standards). All other white Leela games that had lower exits ended in a draw.

In the SF wins, the book exit evaluations by SF were: 1.14 / 0.99 / 0.77 / 0.63 / 0.98 / 1.39 / 1.06 / 1.19 / 1.14 / 1.20 / 0.70 / 0.66 / 1.38 / 1.05 / 0.91 / 1.27 / 1.00 / 0.85  (note: with contempt=24)

I.e. the lowest book exit that led to a win was 0.63. All other white SF games that had lower exits ended in a draw.

Conclusions:

1. All balanced lines, slightly unbalanced lines and slightly dubious lines ended in draws => the draw rate for such lines is 100%;
2. The ONLY wins in this superfinal were scored in (very) dubious lines.

zaterdag 19 september 2020

Opening selection TCEC 19 superfinal

 Here are Jeroen Noomen's comments on the opening selection for the TCEC 19 superfinal:


"Here are the most important details regarding my selection of the openings for the TCEC 19 superfinal:
  • After the TCEC 18 superfinal two bonus events were played with short, balanced openings. The result was predictable: more than 90% draws and a lot of games where evaluations stayed close to 0.00 with not much happening. Thus the conclusion is clear: to keep computer chess (and TCEC!) interesting, unbalanced openings with higher book exits are absolutely necessary.
  • The TCEC 19 superfinal book was mostly completed in April 2020. After seeing the TCEC 18 superfinal in which there were a couple of busted lines, I decided to do a thorough quick check with Stockfish and Leela on the TCEC 19 superfinal book. This led to the removal of a couple of busted lines on July 3rd, which were replaced by more proper lines. Normally I don't change books anymore when they are finished, but because of the TCEC 18 superfinal experience I decided to act and make reparations. 
  • As usual, all important openings will be played; just like in the TCEC 18 superfinal there will be a lot more risky opening lines compared to TCEC 16 and before. However, the TCEC 19 superfinal book will be less extreme than the TCEC 18 superfinal book, which was a bit too much in my opinion.
  • The chosen lines typically have a length between 1 move and 13 moves; there are four lines which are only 1 move (i.e. the first white and black move), there is one line with 14 moves of book. Of the 1 move lines one is rather topical, the other three are unbalanced.
  • A few hints: in this superfinal we will see the return of the Grünfeld Indian, an opening that always causes me headaches, as most lines are too drawish for TCEC superfinal purposes. Of course the King's gambit is present as well. 
  • Average move length is 6.44 (which is 0.6 lower compared to the TCEC 18 superfinal)
  • There are sharp lines in the set, as well as positional linesmore complex lines, lines with opposite castling ("SALC", i.e. short and long castling), dubious lines, lines with an imbalance in material and gambits; my personal favourite lines are #12, #17, #20, #28, #36 and #48 through #50.
  • As TCEC is also followed by strong grandmasters and we also see TCEC games mentioned at theoretical websites as https://www.chesspublishing.com/content/, I now also prefer to include some theoretically important lines, to see if the engines can produce interesting novelties. 
  • The complexity of a position and the book exit evaluation are the most important selection criterions. Book exits have been checked with classical Stockfish (development version, contempt = 0) on a 16 core Ryzen system and Leela Chess Zero v0.26.0, with the net that played in the TCEC 18 superfinal. As mentioned above, book exits are significantly higher compared to season 16 and before, to keep the games interesting and avoid boring draws. Nevertheless, there are also some lines with a lower book exit, especially in the theoretically interesting lines.
  • As I mentioned in my article about the TCEC 18 superfinal book, I now prefer 1-0 1-0 results over two straight forward draws. In effect the result is the same: 1-1. But two wins give more excitement. 
  • I have avoided positions that could lead to many exchanges and/or have a high draw rate in correspondence chess (I prefer draw rates of 50% or less), as well as positions with symmetrical pawn structures and positions that could lead to easy wins for the side with an advantage. The last aspect is becoming more and more difficult, with strong hardware, a long time control and engines being so strong. Of course risky lines means that the chance of 1-0 1-0 is increasing.
  • What do I expect? Frankly speaking I don't know, now that Stockfish also 'went NN'. So I am a bit cautious: 20 wins is the minimum I'd like to see. In any case: enjoy the superfinal!

ECO code distribution
ECO A: 11 lines
ECO B: 14 lines
ECO C: 11 lines
ECO D: 4 lines
ECO E10 lines

Move length distribution
1 move: 4 lines
2 moves: 1 line
3 moves: 2 lines
4 moves: 10 lines
5 moves: 6 lines
6 moves: 5 lines
7 moves: 4 lines
8 moves: 6 lines
9 moves: 3 lines
10 moves: 3 lines
>10 moves: 6 lines

donderdag 2 juli 2020

TCEC 18 superfinal opening PGN available

Jeroen Noomen's TCEC 18 superfinal opening PGN is available for download:

TCEC 18_Noomen Select

The opening set is different from all other superfinal books on several points:
  • Much higher book exits
  • Lots of very risky lines (increasing the chance of a double 1-0 or 0-1 result)
  • Most of the lines are offbeat

    Thus if you like a 'bloody' engine match, you will like this opening set!