Finally, after 3 years of procrastination, I decided to make a Comparative Rating List from the rating list sources which make up the Owl Hybrid Rating List.
It has been stated by some rating list producers that their list should not be compared with the other rating lists because of the different tournament conditions. It is true that the rating lists will never be the same, but I just go ahead to share information that might be interesting to chess engines enthusiasts.
This Comparative Rating List was derived from the rating lists of CCRL, CEGT, FGRL, FCP, IPON and OWL. It contained only the best chess engines version for each family in a single rating list site. The PGN games from each rating list site are cleaned to conform to a naming convention which will produce a harmonious set of engine names to avoid duplicates. Sites like CEGT, CCRL and FGRL have multiple and separate time controlled lists, but these separate lists are merged to make a simpler presentation. From this aggregated lists of the respective producers, the games are filtered to extract only the best engine versions. which were based on their best versions list. For example, the basis for CCRL is the Pure List 40/4 which has more entries than the other 40/40 list. A rating list is then produced for each producer using ORDO 1.2.6 with a common ELO base rating 3090 anchored on Naum 4.6. Incidentally, the 3090 anchor base is arbitrarily set with earlier data from the combined rating lists of Chess Engines and Top Grandmasters. That rating list was anchored on Garry Kasparov with his peak ELO of 2851 and the results showed that the top Chess Engines surpassed the 3000 mark including Naum 4.6. The resulting rating list from each site are then combined to produce a master ELO rating list which will be used to for comparison.
Here are some interesting data from the Comparative Rating List:
- The strongest chess engine in the world is Asm Fish 2016-12-12 which has a peak rating of 3602 from FGRL and 3586 from the overall combined rating. It is 50 ELO stronger than Stockfish 8.
- The highest rating numbers are produced by FGRL and OWL which have a time control of 1 minute or 60 seconds.
- Stockfish rating is 3481 for CCRL at this comparative list while it is 3483 at CCRL site as of May 17, 2017 which is almost on target.
There may be multiple chess engines in one family which is caused by the fact that the rating list producers do not use the same versions with each other. We can see that some sites use older versions and some sites have the more recent versions. Some lists have many entries and others have short ones. We can also see that some who don't like clones/derivatives have actual entries for these in their list.
Well, it is left to the reader to interpret the results. I just hope that they will get information fast when they look the respective rating lists side by side at a glance.
See the link to the full Comparative Rating List here.
Download the detailed comparative rating list in spreadsheet format which shows the rank, rating, games played, points and score percentage. It may be useful to the curious.