out of curiosity if...

[chris]

...someone posts a result (with games) announcing +30 or +50 or +80 elo vs SF15.1 under fishtest conditions (cutechess-cli, 1 thread, hash 16 MB, UHO_XXL_+0.90_+1.19.epd as opening book, tc 10+0.1, etc), what level of hostility can he expect from the chess community ?

[Nemesis]

...someone posts a result (with games) announcing +30 or +50 or +80 elo vs SF15.1 under fishtest conditions (cutechess-cli, 1 thread, hash 16 MB, UHO_XXL_+0.90_+1.19.epd as opening book, tc 10+0.1, etc), what level of hostility can he expect from the chess community ?

You're entited to post your results as you see fit with the conditions you choose. No doubt some will nitpick since it does not have their imprimatur.

[chris]

For example, the test of the new green net stopped after 30k games so I guess at best "only" 15k distinct openings were tested on the 261k openings from the "UHO_XXL_+0.90_+1.19" book. The result (+3 elo) therefore depends only on a fraction of the book. I understand that as 116 hosts were able to participate in this test, some openings may have been tested several times, so probably there were not 15k.

scenario : if someone chooses a short list of openings from this book, especially the openings where he knows his engine will be the most efficient, what is the credibility of his result ?

[sarona]

...someone posts a result (with games) announcing +30 or +50 or +80 elo vs SF15.1 under fishtest conditions (cutechess-cli, 1 thread, hash 16 MB, UHO_XXL_+0.90_+1.19.epd as opening book, tc 10+0.1, etc), what level of hostility can he expect from the chess community ?

If you post said result over at the TC Forum, it is likely you will get a 10+ page thread loaded with insults and lectures as to why your test result is in error (despite the fact that you are using Fishtest parameters and modern hardware).

[chris]

usual recent fishtest conditions :

- cutechess-cli
- 8moves_v3.pgn or UHO_XXL_+0.90_+1.19.epd
- 1 thread, hash 16 MB, tc 10+0.1
- 1 thread, hash 64 MB, tc 60+0.6
- 8 threads, hash 256 MB, tc 30+0.3

Then, combine games from several different hosts ?

[chris]

Based on statistics obtained through expRate, few UHO_XXL_+0.90_+1.19 openings where SF could suffer...

E00-E99 :

Code: Select all

d2d4 g8f6 c2c4 g7g6 b1c3 f8g7 e2e4 d7d6 f2f3 e8g8 c1e3 c7c6 d1d2 a7a6 h2h4 b7b5
d2d4 g8f6 c2c4 e7e6 g2g3 d7d5 f1g2 c7c5 c4d5 e6d5 g1f3 b8c6 e1g1 f8e7 d4c5 e8g8
e2e4 g7g6 c2c4 d7d6 d2d4 g8f6 b1c3 f8g7 f1e2 b8c6 c1e3 e7e5 d4d5 c6b8 f2f3 f6h5
d2d4 g8f6 c2c4 g7g6 b1c3 f8g7 e2e4 d7d6 c1e3 e8g8 f2f3 b8c6 d4d5 c6b8 f1d3 e7e5
d2d4 g7g6 c2c4 g8f6 b1c3 f8g7 e2e4 d7d6 c1e3 e8g8 d1d2 f6g4 e3g5 b8c6 f2f3 g4f6
c2c4 g7g6 d2d4 g8f6 b1c3 f8g7 e2e4 d7d6 f1e2 e8g8 g1f3 b8c6 d4d5 c6b8 h2h3 e7e5
d2d4 g8f6 c2c4 g7g6 b1c3 f8g7 g1f3 e8g8 e2e4 d7d6 f1d3 b8c6 h2h3 e7e5 d4d5 c6b8
d2d4 g8f6 c2c4 g7g6 b1c3 f8g7 e2e4 d7d6 h2h3 e8g8 g1f3 e7e5 d4d5 f6h5 f3h2 h5f6

D00-D99 :

Code: Select all

d2d4 d7d5 c2c4 e7e6 b1c3 f8e7 g1f3 g8f6 c1g5 b8d7 d1c2 e8g8 e1c1 f8e8 c1b1 d7f8
d2d4 g8f6 c2c4 g7g6 b1c3 d7d5 g1f3 f8g7 d1b3 d5c4 b3c4 c8f5 f3g5 e8g8 e2e4 f5c8
d2d4 d7d5 g1f3 g8f6 g2g3 c7c6 f1g2 e7e6 e1g1 f8e7 c2c4 d5c4 f3e5 e8g8 e5c4 f6d5
d2d4 d7d5 g1f3 g8f6 c2c4 c7c6 b1c3 c8f5 d1b3 f5c8 e2e3 e7e6 f1d3 f8d6 e1g1 e8g8
d2d4 d7d5 c2c4 e7e6 g1f3 c7c6 e2e3 g8f6 f1d3 d5c4 d3c4 b7b5 c4d3 b8d7 e1g1 f8d6

C00-C99 :

Code: Select all

e2e4 e7e6 d2d4 d7d5 b1c3 g8f6 e4e5 f6d7 f2f4 c7c5 g1f3 b8c6 c1e3 d8b6 d1d2 b6d8
e2e4 e7e5 g1f3 b8c6 f1c4 f8c5 c2c3 g8f6 d2d3 h7h6 e1g1 e8g8 h2h3 c6a5 c4b5 a5c6

B00-B99 :

Code: Select all

e2e4 c7c5 g1f3 d7d6 b1c3 a7a6 d2d4 c5d4 f3d4 g8f6 c1g5 b8c6 g5c1 c6b8 d4f3 f6g8
e2e4 c7c6 b1c3 d7d5 g1f3 d5e4 c3e4 g8f6 e4f6 g7f6 f1e2 c8g4 d2d4 d8c7 e1g1 b8d7
e2e4 c7c6 d2d4 d7d5 b1c3 d5e4 c3e4 g8f6 e4f6 g7f6 g1f3 d8c7 c2c4 c8g4 f1e2 b8d7
e2e4 c7c6 d2d4 d7d5 b1d2 d5e4 d2e4 g8f6 e4f6 g7f6 g1f3 c8f5 f1d3 f5g4 d3e2 d8c7
e2e4 c7c5 f1c4 e7e6 b1c3 b8c6 g1f3 c6a5 c4e2 d7d6 d2d4 c5d4 f3d4 a5c6 c1e3 g8f6
e2e4 c7c5 g1f3 d7d6 d2d4 c5d4 f3d4 b8c6 c1e3 g8f6 b1c3 e7e6 f1c4 c8d7 d4b5 d7c8
e2e4 d7d6 d2d4 g7g6 b1c3 f8g7 c1e3 g8f6 d1d2 b8c6 h2h3 e7e5 d4e5 c6e5 f2f4 e5c6
e2e4 c7c5 g1f3 g7g6 d2d4 f8g7 b1c3 d8a5 c1e3 b8c6 d1d2 c5d4 f3d4 g8f6 d4b3 a5d8
e2e4 c7c5 g1f3 b8c6 d2d4 c5d4 f3d4 d8b6 d4b3 g8f6 b1c3 d7d6 c1e3 b6d8 f2f3 g7g6
e2e4 g7g6 d2d4 f8g7 b1c3 c7c5 g1f3 c5d4 f3d4 d8b6 d4b3 d7d6 c1e3 b6d8 d1d2 b8c6
e2e4 g7g6 d2d4 f8g7 b1c3 c7c5 g1f3 b8c6 c1e3 d8b6 c3d5 b6d8 d1d2 c5d4 f3d4 g8f6
e2e4 c7c5 g1f3 b8c6 d2d4 c5d4 f3d4 g7g6 b1c3 f8g7 c1e3 g8f6 f1e2 d8a5 d4b3 a5d8
e2e4 c7c6 g1f3 d7d5 b1c3 c8g4 h2h3 g4h5 f1e2 d5e4 c3e4 g8f6 e4f6 g7f6 e1g1 d8c7
e2e4 c7c5 g1f3 d7d6 d2d4 c5d4 f3d4 g8f6 b1c3 a7a6 f1e2 g7g6 c1e3 f8g7 d1d2 h7h5
e2e4 c7c5 g1f3 d7d6 d2d4 c5d4 f3d4 g8f6 b1c3 a7a6 c1g5 d8a5 d1d2 b8c6 d4b3 a5d8
e2e4 g7g6 d2d4 f8g7 b1c3 d7d6 f1e2 g8f6 e2f1 f6g8 c1e3 g8f6 f2f3 c7c6 d1d2 e8g8
e2e4 c7c6 d2d4 d7d5 b1c3 g7g6 h2h3 f8g7 g1f3 g8h6 f1d3 f7f6 d1e2 h6f7 c1e3 e8g8
e2e4 c7c5 b1c3 b8c6 f1b5 c6a5 g1f3 a7a6 b5e2 a5c6 d2d4 c5d4 f3d4 d7d6 c1e3 e7e6
e2e4 c7c5 g1f3 d7d6 d2d4 c5d4 f3d4 g8f6 b1c3 b8c6 c1g5 d8b6 d4b3 e7e6 g5f4 b6d8
e2e4 c7c6 d2d4 d7d5 b1d2 d5e4 d2e4 d8d5 e4c3 d5a5 f1c4 g8f6 c1d2 c8f5 d1e2 e7e6
e2e4 c7c5 g1f3 d7d6 d2d4 c5d4 f3d4 g8f6 b1c3 b8c6 c1g5 e7e6 d1d2 f8e7 e1c1 d8b6
e2e4 d7d6 d2d4 g8f6 b1c3 g7g6 f1e2 f8g7 g1f3 e8g8 c1e3 c7c6 d1d2 f6g4 e3g5 g4f6
e2e4 c7c6 d2d4 d7d5 b1c3 d5e4 c3e4 g8f6 e4f6 g7f6 g1f3 c8g4 f1e2 d8c7 f3h4 g4e2
e2e4 g7g6 d2d4 f8g7 b1c3 c7c6 g1f3 d7d6 c1e3 g8f6 d1d2 d8a5 f1d3 f6g4 e3g5 g4f6
e2e4 c7c5 g1f3 b8c6 d2d4 c5d4 f3d4 g8f6 b1c3 d7d6 c1g5 c8d7 d1d2 d8a5 d4b3 a5d8
e2e4 c7c5 b1c3 b8c6 f1b5 c6d4 b5c4 e7e6 g1f3 d4c6 d2d4 c5d4 f3d4 d7d6 c1e3 g8f6

A00-A99 :

Code: Select all

d2d4 g8f6 c1g5 f6e4 g5f4 g7g6 f2f3 e4f6 e2e4 d7d6 c2c4 f8g7 b1c3 e8g8 d1d2 c7c6
d2d4 f7f5 g1f3 g8f6 c2c4 g7g6 b1c3 f8g7 g2g3 d7d6 f1g2 e8g8 d4d5 e7e5 d5e6 c7c6
d2d4 f7f5 g1f3 g8f6 g2g3 g7g6 f1g2 f8g7 e1g1 e8g8 c2c4 d7d6 b1c3 b8d7 f3g5 d7b8
g1f3 g7g6 d2d4 f8g7 c2c4 f7f5 g2g3 g8f6 f1g2 e8g8 e1g1 d7d6 b1c3 b8c6 d4d5 c6b8
d2d4 f7f5 g1f3 g7g6 g2g3 f8g7 f1g2 g8f6 e1g1 e8g8 f1e1 b8c6 d4d5 c6b8 c2c4 d7d6
d2d4 g8f6 c1g5 f6e4 g5f4 g7g6 f2f3 e4f6 e2e4 d7d6 c2c4 f8g7 b1c3 f6h5 f4e3 e7e5
g1f3 d7d5 g2g3 g8f6 f1g2 b8c6 d2d4 c6b8 c2c4 e7e6 b1c3 f8e7 e1g1 e8g8 d1c2 c7c6
d2d4 d7d6 g1f3 c8g4 e2e4 g8f6 b1c3 c7c6 h2h3 g4h5 d1e2 h5f3 e2f3 e7e6 f1d3 d6d5
d2d4 g8f6 c1f4 c7c5 c2c3 d8a5 e2e3 c5d4 e3d4 d7d5 f1d3 e7e6 g1f3 b8c6 b1d2 a5d8
d2d4 g8f6 g1f3 e7e6 c1f4 c7c5 e2e3 b8c6 b1d2 c5d4 e3d4 d8b6 d2c4 b6d8 c2c3 d7d5
d2d4 g8f6 g1f3 g7g6 g2g3 f6g8 c2c4 f8g7 f1g2 g8f6 e1g1 e8g8 f1e1 d7d6 b1c3 b8c6
g1f3 d7d5 c2c4 c7c6 e2e3 g8f6 b1c3 c8g4 d1b3 g4c8 d2d4 e7e6 f1d3 f8d6 e1g1 b8d7
d2d4 g7g6 c2c4 f8g7 b1c3 d7d6 e2e4 e7e5 g1f3 b8c6 d4d5 c6e7 f1e2 g8f6 f3d2 e8g8
b2b3 g8f6 c1b2 g7g6 e2e4 d7d6 g2g3 f8g7 f1g2 e8g8 d2d4 e7e5 d4e5 f6g4 b1c3 g4e5
d2d4 g8f6 c2c4 b8c6 g1f3 d7d6 d4d5 c6b8 b1c3 e7e5 d1c2 g7g6 e2e4 f8g7 h2h3 e8g8
d2d4 b8c6 c1f4 d7d6 g1f3 g8f6 c2c4 g7g6 d4d5 c6b8 b1c3 f8g7 e2e4 e8g8 h2h3 e7e5

[chris]

warm up (=round 0) : I tried to put SF 15.1 in the best conditions with a selection of 1269 openings from the UHO_XXL_+0.80_+1.09 opening list, low power (8 threads), light hash (256MB) and an ultra short time per move (TC 30+0.3).

Code: Select all

   # PLAYER             :  RATING  ERROR  POINTS  PLAYED   (%)    W     D    L  D(%)  OppAvg  OppN
   1 stockfish 15.1     :      42     11  1384.0    2538  54.5  751  1266  521  49.9       0     1
   2 Eman experience    :       0   ----  1154.0    2538  45.5  521  1266  751  49.9      42     1

White advantage = 193.06 +/- 5.76
Draw rate (equal opponents) = 50.00 % +/- 0.00

Now let's see how many rounds Stockfish can withstand...

[chris]

After learning the 4 openings where Eman lost as white :

Code: Select all

   # PLAYER             :  RATING  ERROR  POINTS  PLAYED   (%)    W     D    L  D(%)  OppAvg  OppN
   1 stockfish 15.1     :      41     11  1381.0    2538  54.4  746  1270  522  50.0       0     1
   2 Eman experience    :       0   ----  1157.0    2538  45.6  522  1270  746  50.0      41     1

White advantage = 193.67 +/- 5.76
Draw rate (equal opponents) = 50.00 % +/- 0.00

[chris]

Currently, Eman is learning the 746 openings where it lost as black :


ps : UHO_XXL_+0.80_+1.09 means that all its openings give a white advantage between +0.80 and +1.09...

[chris]

After learning the 746 openings where Eman lost as black :

Code: Select all

   # PLAYER             :  RATING  ERROR  POINTS  PLAYED   (%)    W     D    L  D(%)  OppAvg  OppN
   1 Eman experience    :       0   ----  1285.0    2538  50.6  520  1530  488  60.3      -5     1
   2 stockfish 15.1     :      -5     11  1253.0    2538  49.4  488  1530  520  60.3       0     1

White advantage = 147.00 +/- 5.37
Draw rate (equal opponents) = 50.00 % +/- 0.00

[chris]

Next step, Eman will learn the 749 openings where it drew as white :

[chris]

After learning the 749 openings where Eman drew as white :

Code: Select all

   # PLAYER             :  RATING  ERROR  POINTS  PLAYED   (%)    W     D    L  D(%)  OppAvg  OppN
   1 Eman experience    :       0   ----  1406.5    2538  55.4  763  1287  488  50.7     -51     1
   2 stockfish 15.1     :     -51     11  1131.5    2538  44.6  488  1287  763  50.7       0     1

White advantage = 192.48 +/- 5.74
Draw rate (equal opponents) = 50.00 % +/- 0.00

[Hurnavich]

Hi, is there a video available on how to undertake this training process? thanks

[chris]

The last step in order to complete the round 1 => Eman will learn the 397 openings where it drew as black :


To Learn these ones it will be harder because, as expected SF15.1 will always start with a white advantage between +0.80 and +1.09 and, even if Eman already managed to draw 397 openings as black (on 1269), on this step it will have to learn a way to win them as black !

ps : On round 0, SF15.1 managed to win 4 openings as black. Since beginning of round 1, Eman only won this one

[chris]

Hi, is there a video available on how to undertake this training process? thanks

I would be surprised if a chess engine trainer rented several camcorders to film his cluster in action...

Here are some tutorials :
Train your engine !
how to create a learning farm
old videos

I plan to share the 2538 games at the end of the round 1 (but not my eman.exp loool). All the 1269 openings come from the UHO_XXL_+0.80_+1.09 opening list. Here i'm only searching how many steps it need to draw or beat Stockfish on one of the fishtest conditions. These 1269 openings (on 200k) were selected by expRate as usual (or openingRate when i want to draw or beat a bin book).