donderdag 12 augustus 2021

Noomen Low Draw Rate Testset (version 3)

Hi all,

Below you can download Jeroen Noomen's "Low Draw Rate Testset" - version 3. Added are the most interesting lines from the TCEC 21 superfinal. A couple of lines by GM Matthew Sadler were selected as well, but these were altered slightly to make them unique.

All lines in this set have a Leela Chess Zero book exit of +0.35 or higher. This is necessary to get interesting games and a reasonable amount of wins. In engine-engine matches both engines have to defend the weaker line, in the return they can try to turn the advantage into a win.

Currently balanced lines and/or top GM lines are not an option anymore for the best engines. Selecting such lines will lead to a result of (close to) 100% draws. And the games will be uninteresting as well. To avoid this, opening lines have to be unbalanced.

Download link:

Noomen Low Draw Rate Testset v3

vrijdag 6 augustus 2021

TCEC 21 superfinal opening PGN available

The TCEC 21 superfinal finished on Saturday, August 7th. Stockfish prolonged its title by beating Leela Chess Zero by 56-44. Stockfish won 19 games, Leela won 7 games and 74 games ended in a draw.

22 lines ended decisive, i.e. in a 1.5-0.5 score. Only 2 lines ended with two wins. 

Via the link below you can download the TCEC 21 superfinal opening PGN. The odd numbered lines were selected by GM Matthew Sadler. The even numbered lines were selected by Jeroen Noomen. 

TCEC 21 superfinal PGN

dinsdag 1 juni 2021

Noomen Low Draw Rate Testset

Hi all,

Here is a new testset for engine-engine testing: Noomen_LowDrawRate_v2.  This is a selection of TCEC superfinal opening lines (S9 through S20) with the following criteria:

  • Leela Chess Zero book exit of +0.35 or higher
  • Good chance of a 1.5-0.5 result between the strongest engines
As the previous two superfinals (S19 and S20) showed, Leela book exits below +0.30 are a certain draw under TCEC superfinal conditions. To get interesting games it is necessary to give the engines something to play with. Hence the minimum book exit of +0.35.

Using this special book for engine-engine testing will lead to:
  • Exciting games
  • More decisive results  
Enjoy testing!

Download:
Noomen_LowDrawRate_v2

zaterdag 8 mei 2021

The TCEC 21 superfinal book, by GM Matthew Sadler and Jeroen Noomen

I am excited to announce that the TCEC 21 superfinal book has been made in cooperation between chess Grandmaster Matthew Sadler and myself (Jeroen Noomen). Matthew is a well respected Grandmaster and the co-author of the great book "Game Changer", together with WGM Natasha Regan. It was an honour to get Matthew on board and I am very pleased with the result!


Here is the most important information about our joint effort:

  • The TCEC 21 superfinal book was mostly completed in April 2021. Both Matthew and I have chosen 25 lines, which will be played alternately. Matthew's lines are #1, #3, #5 and so forth, my lines are #2, #4, #6 and so forth. The lines are mixed, which means that lower book exits are mixed with higher book exits. I chose to mix the book exits, as the experiment with a low-to-high exit in season 20 led to a lot of draws in the first half of the superfinal.
  • To determine the book exits we used Leela Chess Zero and Stockfish. All lines typically have a LC0 book exit between +0.30 through +0.50, with a few exceptions. Of course the book exits at TCEC will be different, because of much faster hardware and different Leela and Stockfish versions. 
  • As usual, all important openings will be played; just like in the previous three TCEC superfinals there will be a lot more risky opening lines compared to TCEC 16 and before. This is necessary to avoid an excessive number of uninteresting draws. Statistics of the previous two superfinals show that a Leela book exit below +0.30 is an almost 100% certain draw. 
  • The chosen lines typically have a length between 3 moves and 13 moves; there are five lines which are 3 moves long and there are three lines which are 13 moves long. This time some openings will be played more often than you are used to in previous superfinals. 
  • Average move length is 6.84, which is is comparable to the TCEC 20 superfinal.
  • There are sharp lines in the set, as well as positional linesmore complex lines, lines with opposite castling ("SALC", i.e. short and long castling), dubious lines, lines with an imbalance in material and gambits. In my (Jeroen) selection I used a couple of lines of Stefan Pohl's "Unbalanced Human Openings".
  • As I mentioned previously, I now prefer 1-0 1-0 results over two straight forward (i.e. boring) draws. In effect the result is the same: 1-1. But two wins give more excitement. The super solid openings played by today's top Grandmasters are a certain draw in today's engine chess, hence they are not suitable for TCEC superfinal purposes.
  • We have avoided positions that could lead to many exchanges, as well as positions with a symmetrical pawn structure and positions that could lead to easy wins for the side with an advantage. The last aspect is becoming more and more difficult, with strong hardware, a long time control and engines being so strong. Of course risky lines means that the chance of 1-0 1-0 is increasing.
  • What do I expect? 20 wins is the minimum I'd like to see, but of course I hope for more wins. In my humble opinion I think that the TCEC 21 superfinal could be the most exciting superfinal ever!

ECO code distribution
ECO A: 10 lines
ECO B: 16 lines
ECO C: 10 lines
ECO D: 8 lines
ECO E: 6 lines

Move length distribution
3 moves: 5 lines
4 moves: 7 lines
5 moves: 4 lines
6 moves: 12 lines
7 moves: 2 lines
8 moves: 7 lines
9 moves: 5 lines
10 moves: 3 lines

maandag 1 februari 2021

TCEC 20 superfinal opening PGN available

 The TCEC 20 superfinal opening PGN is now available for download:

TCEC 20 opening PGN

Stockfish won the superfinal vs Leela Chess Zero by 53-47. Stockfish won 14 games, Leela won 8 games, while 78 games ended in a draw.

Some interesting statistics about the decisive opening pairs that ended 1.5-0.5:

0.36 / 1.05
0.33 / 0.67
0.40 / 0.71
0.40 / 0.91
0.41 / 0.86
0.43 / 0.98
0.42 / 0.79
0.42 / 0.64
0.49 / 1.07
0.65 / 1.15
0.55 / 0.84
0.43 / 1.34
0.52 / 0.90
0.52 / 1.48

These are the book exit evaluations by Leela (mentioned first) and Stockfish (mentioned second) in the opening lines that ended 1.5-0.5. Conclusion can be similar to the S19 superfinal: Leela book exits lower than +0.40 seem to be an almost certain draw. There were only two exceptions, where Leela blundered a drawish position after the book exits had been lower than+0.40.

zaterdag 9 januari 2021

Opening selection TCEC 20 superfinal

Here are Jeroen Noomen's comments on the opening selection for the TCEC 20 superfinal:


"Here are the most important details regarding my selection of the openings for the TCEC 20 superfinal:

  • At www.twitch.tv/navratil25 two interesting matches between LC0 and Stockfish were played recently. The first match was bookless. It ended in 100 draws, with all games being predictable, boring and without any winning chances. The second match was played with more or less balanced lines. It ended in 2-2 with 96 draws. Again many predictable, boring games, in which the engines had nothing no play for. Thus the conclusion is clear: to keep computer chess interesting and avoid a huge number of uninteresting & boring draws, unbalanced openings with higher book exits are absolutely necessary.
  • The TCEC 20 superfinal book was mostly completed in June 2020, pre NNUE. Compared to all my other superfinal books, there has been a big change: the opening with the lowest book exit will be played first, followed by the next slightly higher book exit and so forth, until line 50, which has the highest book exit. This means it is possible that the same opening will be played two times in a row.
  • To determine the book exits I took the average evaluation of Stockfish 12 (5 minutes on my 8 core Ryzen) and LC0 v26.2 J92-100 (2 minutes on my notebook). Of course the book exits at TCEC will be different, because of much faster hardware and different Leela and Stockfish versions. But all in all we will see a superfinal in which the openings will become riskier as the match progresses.
  • As usual, all important openings will be played; just like in the previous two TCEC superfinals there will be a lot more risky opening lines compared to TCEC 16 and before. However, the TCEC 20 superfinal book will be less extreme than the TCEC 18 superfinal book, which was a bit too much in my opinion.
  • In the TCEC 20 superfinal we will see many openings (exactly the same, or slightly different) that have featured in previous superfinals. 
  • The chosen lines typically have a length between 1 move and 12 moves; there are two lines which are only 1 move (i.e. the first white and black move), there is one line with 14 moves of book and one line with 16 moves of book.
  • Average move length is 6.76 (which is 0.32 higher compared to the TCEC 19 superfinal).
  • There are sharp lines in the set, as well as positional linesmore complex lines, lines with opposite castling ("SALC", i.e. short and long castling), dubious lines, lines with an imbalance in material and gambits.
  • As I mentioned in my article about the TCEC 18 superfinal book, I now prefer 1-0 1-0 results over two straight forward draws. In effect the result is the same: 1-1. But two wins give more excitement. 
  • I have avoided positions that could lead to many exchanges and/or have a high draw rate in correspondence chess (I prefer draw rates of 50% or less), as well as positions with symmetrical pawn structures and positions that could lead to easy wins for the side with an advantage. The last aspect is becoming more and more difficult, with strong hardware, a long time control and engines being so strong. Of course risky lines means that the chance of 1-0 1-0 is increasing.
  • What do I expect? 20 wins is the minimum I'd like to see, but of course I hope for more wins. My personal favourite lines are #3, #12, #13, #27, #42 and  #47. Enjoy the superfinal!

ECO code distribution
ECO A: 12 lines
ECO B: 14 lines
ECO C: 11 lines
ECO D: 4 lines
ECO E: 9 lines

Move length distribution
1 move: 2 lines
2 moves: 2 lines
3 moves: 5 lines
4 moves: 8 lines
5 moves: 3 lines
6 moves: 5 lines
7 moves: 5 lines
8 moves: 4 lines
9 moves: 4 lines
10 moves: 6 lines
>10 moves: 6 lines

zaterdag 17 oktober 2020

TCEC 19 superfinal opening PGN available

Jeroen Noomen has made his TCEC 19 superfinal book available for download. His comments:

"The TCEC 19 superfinal ended Friday, October 16th and was won by Stockfish by 54.5-45.5 vs Leela Chess Zero. In wins 18-9, with 73 draws. Just like the S18 superfinal book, the S19 book was way more risky than S16 and before. Nevertheless, the S19 book also had several balanced lines and theoretically interesting lines. Interestingly, all wins were only in (very) dubious openings. Balanced lines and slightly dubious lines had a 100% draw rate. A sign of the times?

You can download the S19 superfinal book from the following location:

http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=33689

Some interesting statistics:

In the Leela wins, the book exit evaluations by Leela were: 0.43 / 0.54 / 0.50 / 0.46 / 0.66 / 0.48 / 0.56 / 0.52 / 0.63

I.e. the lowest book exit that led to a win was 0.43 (which is quite high by Leela standards). All other white Leela games that had lower exits ended in a draw.

In the SF wins, the book exit evaluations by SF were: 1.14 / 0.99 / 0.77 / 0.63 / 0.98 / 1.39 / 1.06 / 1.19 / 1.14 / 1.20 / 0.70 / 0.66 / 1.38 / 1.05 / 0.91 / 1.27 / 1.00 / 0.85  (note: with contempt=24)

I.e. the lowest book exit that led to a win was 0.63. All other white SF games that had lower exits ended in a draw.

Conclusions:

1. All balanced lines, slightly unbalanced lines and slightly dubious lines ended in draws => the draw rate for such lines is 100%;
2. The ONLY wins in this superfinal were scored in (very) dubious lines.