The Week in Chess

Wednesday, April 30, 2014

Stockfish 14042817 x64 vs. Stockfish Clones Tournament 2014-04-29

Finally, I had the intentional chance to conduct a test between the latest and strongest Stockfish chess engine against its latest clone challengers. The active clone challengers chosen are Stockfish RDMF, Stockfish IpmanSE3, Stockfish Rockwood and Stockfish owl. Unfortunately, all the Stockfish Rockwood compiles did not work with any of my computers, so it is not possible to include it. Other clones were not considered because they were too old or no longer maintained.

The test was done on single computer with AMD CPU 4 cores but only 2 cores were allocated so that it would be possible to have 2 instances of the tournament to  finish quickly. It was a 4 engine-100 round matches in the fast 1 minute base + 1 second increment time control. Additional information could be found in the results below.

After less than 24 hours of continuous games, a winner emerge.

And the winner is... Stockfish O.W.L. compilation!

This is the same compilation which was despised and ridiculed by some people as it surprisingly became the compilation to put the Stockfish chess engine NUMBER ONE first as compared with the Chess Engines Rating Lists all over the world. The owl compilation was taken out in the last Owl Chess Engines Rating List and replaced it with the official Stockfish compilation from www.abrok.eu/stockfish to remove doubts that Stockfish is really the number 1 chess engine as of this date.

As stated in the previous post, the Stockfish owl version will only be used for personal fun and to do battle with the other Stockfish clones. This clone competition will remind the Stockfish team that somewhere, there are Stockfish clones that will always be stronger than the original if it could provide a quick update after the release of the latest commit. IpmanSE3 and RDMF clones in the test were dated April 13 and April 14 respectively so their true strength were not updated, but the O.W.L. version managed to compile immediately just hours after the last patch. This may be an endless cycle as the original Stockfish will get stronger, the clones must update quickly to maintain the lead. It maybe the reason why there is no one Stockfish clone that ever managed to grab the spotlight for a long time unless one super smart guy is above the rest, like what happened with the Ippolit family of chess engines.

Here is the result of the Stockfish clones tournament:

Stockfish 14042817 x64 vs. Stockfish Clones Tournament - 2014-04-29
RankEngineScoreStStStSt
1Stockfish 14042817 x64 owl160.0/300· ·· ·· ··19-14-6718-11-7121-13-66
2Stockfish_14042817_x64 153.0/30014-19-67· ·· ·· ··14-9-7720-14-66
3Stockfish 210414 RDMF 145.0/30011-18-719-14-77· ·· ·· ··17-15-68
4Stockfish IpmanSE3 x64 142.0/30013-21-6614-20-6615-17-68· ·· ·· ··

600 games played / Tournament finished

Tournament start: 2014.04.29, 01:31:40
Latest update: 2014.04.30, 00:42:19
Level: Blitz 1/1
Hardware: AMD Phenom(tm) II X4 945 Processor with 1.8 GB Memory
Operating system: Windows 7 Ultimate Professional Service Pack 1 (Build 7601) 64 bit
Table created with: Arena 3.5

Download the Stockfish clones tournament PGN games here.
.

Sunday, April 27, 2014

Stockfish 14042609 x64 vs. Houdini 4 Pro x64 - Gauntlet Match, 200 Rounds 1M1S

 Two days ago, I performed a test with Stockfish 14042408 vs. Houdini 4 Pro at 10 seconds base + 500 milliseconds increment time control, to determine whether Houdini is still the Blitz King of Chess Engines and at the same time to calibrate the ELO strength of Stockfish using the factory compile from the authors instead of the O.W.L. compile. Today I tested the latest compile of Stockfish 14042609 x64  released yesterday, April 26 at the normal 1 minute base + 1 seconds increment longer time control to reinforce the calibration of the ELO strength. Other than the time control difference, the same tournament conditions were used including the 200 round robin matches.

And the result?...  An amazing statistically rare identical score of 112-88 or 56%-44% in favor of Stockfish! The win-loss-draw figures are however different which are 63-39-98 with this current match and 66-42-92 with the previous match, but the sum scores are the same. In the gauntlet matches of 100 round robin games for the Owl Rating List publication last April 21, 2014, where Stockfish 14042120 was the main star, the score with Stockfish vs. Houdini was 56.5% vs. 43.5% which is statistically very close with the other two test results. There may not be significant improvement of the released patches of Stockfish since April 21, 2014.

I can declare with confidence that the ELO rating of Stockfish is stable and accurate within the Owl Computer Chess Engines Rating List tournament conditions.

Here is the gauntlet match statistics:
Stockfish 14042609 x64 vs. Houdini 4 Pro x64 - Match 200R 1M1S
RankEngineScoreStHoS-B
1Stockfish_14042609_x64112.0/200· ·· ·· ··63-39-98 9856.00 
2Houdini 4 Pro x64 88.0/20039-63-98· ·· ·· ·· 9856.00 


200 games played / Tournament finished

Tournament start: 2014.04.26, 07:35:21
Latest update: 2014.04.26, 23:11:49
Level: Blitz 1/1
Hardware: AMD Phenom(tm) II X4 945 Processor with 1.8 GB Memory
Operating system: Windows 7 Ultimate Professional Service Pack 1 (Build 7601) 64 bit
Table created with: Arena 3.5

Download the gauntlet match PGN games here.

Saturday, April 26, 2014

Protector 1.6.0 x64 - Gauntlet Matches, 100 Rounds

Protector 1.6.0 x64 is a UCI chess engine by +Raimund Heid released last April 21, 0214.

Protector 1.6.0 scored 42.47% with 480 wins, 766 losses and 654 draws against the selection of top 19 chess engines in the 100 round gauntlet matches. It earned 2889 ELO rating points with 38 ELO points increase and placed 10th rank in the Top Chess Engines Selection list above Shredder 12 and Hannibal 1.4b.


Rank Engine ELO Raw Games Score% Points Win Loss Draw Chg
1 Stockfish 14042120 x64 3190 256 100 88.50 88.5 78 1 21 0
2 Houdini 4 Pro x64 3162 216 100 84.50 84.5 73 4 23 0
3 Gull 3 x64 3116 188 100 82.50 82.5 68 3 29 0
4 Komodo TCEC x64 3133 173 100 80.00 80.0 67 7 26 -1
5 Strelka 5.7 x64 3047 103 100 71.50 71.5 56 13 31 -1
6 Rybka 4.1 x64 3002 97 100 71.00 71.0 54 12 34 0
7 Fire 3.0 x64 3036 97 100 72.00 72.0 52 8 40 0
8 Critter 1.6a x64 3064 48 100 65.00 65.0 44 14 42 -2
9 Equinox 2.02 x64 3022 46 100 64.00 64.0 47 19 34 -1
10 Shredder 12 x64 2881 -33 100 53.50 53.5 30 23 47 0
11 Deep Hiarcs 14 2867 -41 100 52.00 52.0 35 31 34 1
12 Protector 1.6.0 x64 2889 -55 1900 42.47 807.0 480 766 654 2889
13 Spike 1.4 2849 -59 100 49.50 49.5 29 30 41 0
14 Naum 4.2 x64 2829 -93 100 44.50 44.5 26 37 37 -1
15 Hannibal 1.4b x64 2880 -96 100 44.00 44.0 24 36 40 -1
16 Senpai 1.0 x64 2828 -134 100 38.50 38.5 18 41 41 -1
17 Sjeng 2010 2792 -147 100 37.00 37.0 19 45 36 0
18 Junior 13.8.04 x64 2783 -157 100 36.50 36.5 22 49 29 0
19 Murka 3 x64 2762 -170 100 33.50 33.5 14 47 39 0
20 DiscoCheck 5.2 x64 2759 -240 100 25.00 25.0 10 60 30 -2
.
Download the gauntlet matches PGN games here.

Owl Computer Chess Engines Rating List #105

The 105th Owl Computer Chess Engines Rating List released, 04/26/2014.

View the full rating list here.

Friday, April 25, 2014

Stockfish 14042408 x64 vs. Houdini 4 Pro x64 - Fast Blitz Match 10secs + 500ms

It's time once again to test the latest of Stockfish with version 14042408 released yesterday, April 24, 2014. The release was a series of 4 patches applied in 1 day such as:
   - Remove RookOn7th and merge values into psqt
   - Remove penalty for knight when few enemy pawns
   - Shuffle movepicker score
   - Correctly apply previous patch

The purpose of the test is to check for strength consistency, no regression and duel with Houdini in very short time control of sub-second per move match where Houdini is considered supreme by chess engine experts. The tournament conditions are indicated in the match statistics below in a 200 round combat

After more than 5 hours of continuous fight, Stockfish 14042408 emerged as the victor where it scored 112 points out of 200 games or 56% vs 44% result. Other testers may have different results, but this test showed that Houdini is no longer the undisputed king in fast Blitz match. Today, based on the match results, I declare Stockfish as the new Chess Engine Blitz King in the Owl Chess Engines Rating List.

Here is the match statistics:


Stockfish 14042408 x64 vs. Houdini 4 Pro x64 - Match 200R 10secs + 500ms
RankEngineScoreStHoS-B
1Stockfish_14042408_x64112.0/200· ·· ·· ··66-42-92 9856.00 
2Houdini 4 Pro x64 88.0/20042-66-92· ·· ·· ·· 9856.00 


200 games played / Tournament finished

Tournament start: 2014.04.24, 04:48:20
Latest update: 2014.04.24, 11:11:13
Level: Blitz 0:10/0.5
Hardware: AMD Phenom(tm) II X4 945 Processor with 1.8 GB Memory
Operating system: Windows 7 Ultimate Professional Service Pack 1 (Build 7601) 64 bit
Table created with: Arena 3.5

Download the match result PGN games here:

Thursday, April 24, 2014

Stockfish 14042120 x64 - Gauntlet Matches, 100 Rounds

Stockfish 14042120 x64 is a developmental UCI chess engine by +Marco Costalba+Joona Kiiski+Tord Romstad released last April 21, 2014 .

Stockfish 14042120 scored 80.82% with 1279 wins, 108 losses and 513 draws against the top 19 selection of computer chess engines in the 100 round gauntlet matches.  It defeated all the chess engines convincingly, retaining the number 1 spot of the strongest chess engines in the Owl Rating List with an ELO rating of 3190. However, The ELO rating difference between Houdini 4 was reduced to 28 ELO points difference which was lower than previous difference of around 50 ELO. The important change in this current rating list is the use of the original Stockfish development version from the authors instead of the compiled unofficial version from O.W.L. 

Some people have found this site to be ridiculous and not worth looking at which was probably caused by the sudden rise of Stockfish to the number 1 spot in this rating list site and the fact the other many popular rating list sites had Houdini as the number 1 for a number of years. The major point which this rating site and the other sites differ is in the usage of the Stockfish version.

If some people found this site laughable, they may almost certainly find the prestigious TCEC site ridiculous too because Houdini is not number 1 there and presently struggling against Stockfish and Komodo in the current TCEC Season 6 live tournament.

From this moment on, only the official Stockfish compilation from the authors will be used in this site, while the O.W.L. version will just be for personal entertainment or maybe to let it battle against the other Stockfish compilations/derivatives.

Rank Engine ELO Raw Games Score% Points Win Loss Draw Chg
1 Stockfish 14042120 x64 3190 234 1900 80.82 1535.5 1279 108 513 3190
2 Houdini 4 Pro x64 3162 195 100 43.50 43.5 20 33 47 -2
3 Komodo TCEC x64 3134 183 100 42.00 42.0 23 39 38 -1
4 Gull 3 x64 3116 150 100 36.00 36.0 13 41 46 0
5 Fire 3.0 x64 3036 131 100 32.50 32.5 7 42 51 0
6 Critter 1.6a x64 3066 129 100 32.50 32.5 9 44 47 -1
7 Equinox 2.02 x64 3023 111 100 30.00 30.0 8 48 44 1
8 Rybka 4.1 x64 3002 42 100 23.50 23.5 11 64 25 -2
9 Strelka 5.7 x64 3048 39 100 21.50 21.5 6 63 31 -2
10 Hannibal 1.4b x64 2881 -39 100 13.50 13.5 1 74 25 0
11 Deep Hiarcs 14 2866 -68 100 12.00 12.0 2 78 20 1
12 Senpai 1.0 x64 2829 -88 100 10.00 10.0 0 80 20 0
13 Shredder 12 x64 2881 -92 100 11.00 11.0 3 81 16 -1
14 Spike 1.4 2849 -115 100 9.00 9.0 1 83 16 0
15 Naum 4.2 x64 2830 -115 100 9.00 9.0 1 83 16 0
16 Junior 13.8.04 x64 2783 -123 100 9.00 9.0 2 84 14 0
17 Murka 3 x64 2762 -127 100 8.00 8.0 0 84 16 2
18 Protector 1.5.0 x64 2851 -136 100 8.00 8.0 1 85 14 -1
19 DiscoCheck 5.2 x64 2761 -138 100 7.50 7.5 0 85 15 3
20 Sjeng 2010 2792 -174 100 6.00 6.0 0 88 12 0
.
Download the gauntlet matches games PGN here.

Owl Computer Chess Engines Rating List # 104

The 104th Owl Computer Chess Engines Rating List released, 04/24/2014.

View the full rating list here.

Gaviota 1.0 x64 - Gauntlet Matches, 100 Rounds

The gauntlet matches for Gaviota 1.0 x64 against the selected top chess engines was started a few days ago but was aborted after seeing so many losses by time forfeit at 43.75% or 350 games. The previous version Gaviota 0.86 did not suffer the same behavior. This is the second occurrence of gauntlet matches that was aborted because of excessive losses by time forfeit where Chiron 2.02 had similar performance. A third should be from Arminius 14-01-18 which was tested earlier but was not reported. It should be noted that these engines were contestants in the current TCEC Season 6 Live Tournament and do not have serious problems. The probable cause why these engines do not perform well in the Owl Rating List is because they use the old Winboard Protocol which may not be 100% compatible with the new Arena 3.5 GUI. Most chess engines using Winboard usually gave me extra work to do just to run successfully in Arena tournaments.

This failed Gaviota 1.0 gauntlet matches will not be reflected in the Owl Rating List. 











Gaviota
Rank Engine ELO Games Score% Points Win Loss Draw TF% Ply TF
1 Thinker 54D Inert x64 317 100 93 93 91 5 4 0 0 86
2 Crafty 23.8 x64 53 100 72 72 67 23 10 9 0 64
3 Doch 1.3.4 x64 34 100 71.5 71.5 62 19 19 0 0 50
4 Booot 5.2.0 x64 1 100 67 67 58 24 18 0 0 46
5 Minko Chess 1.3 x64 -5 100 66.5 66.5 57 24 19 0 0 40
6 Cheng 4.0.36a x64 -20 100 64.5 64.5 54 25 21 0 0 35
7 Nemo 1.0.1b x64 -69 100 58 58 45 29 26 0 0 31
8 Gaviota 1.0 x64 -125 800 33.31 266.5 192 459 149 43.75 1 xxx
9 Loop 2007 x64 -186 100 41 41 25 43 32 0 0 7









Total
350
.
Download the gauntlet matches PGN games here.

Chessdom News