Page 1 of 5

Toolkit to train a net without gensfen nor selfplay

Posted: 23/01/2021, 21:16
by deeds
Hi all ! I plan to offer some private tools to reuse available data in order to train some nets.

First tool => nnue_eval : https://mega.nz/folder/OtRRlSLZ#ZfmDUuDTI7NIRQcdqcX6SA

Enjoy !

Toolkit to train a net without gensfen nor selfplay

Posted: 23/01/2021, 22:41
by IbaiBuR
Thanks a lot for your tool! It is very simple to use and I have always wanted to train a net based on pgns but had failed rescoring with different programs. It is great that you shared this.

Toolkit to train a net without gensfen nor selfplay

Posted: 23/01/2021, 23:27
by hazsan88
Thank you very much for the tool that's shared
then, how to convert Plain Text into Bin :?:

xrf01

Toolkit to train a net without gensfen nor selfplay

Posted: 24/01/2021, 5:12
by deeds
My pleasure !

From PLAIN TEXT format to NNUE BIN format, you can use the (LEARN) CONVERT_BIN command from any NODCHIP executables.

Toolkit to train a net without gensfen nor selfplay

Posted: 24/01/2021, 7:39
by ZamChess
deeds wrote:My pleasure !

From PLAIN TEXT format to NNUE BIN format, you can use the (LEARN) CONVERT_BIN command from any NODCHIP executables.
Where can I find NODCHIP executables ?...

Toolkit to train a net without gensfen nor selfplay

Posted: 24/01/2021, 7:41
by deeds

Toolkit to train a net without gensfen nor selfplay

Posted: 24/01/2021, 14:38
by IbaiBuR
Hello, is it possible to resume rescoring or not? It says 3 days left and I wont be able to have my laptop that time running.

Toolkit to train a net without gensfen nor selfplay

Posted: 24/01/2021, 15:00
by deeds
Hard to say like that ...

Scenario # 1: converting a single large PGN file
The tool is optimized for multitasking so you should split your large PGN file into several small PGN files.

Scenario # 2: converting multiple PGN files simultaneously
It will suffice to move the PGN+PLAIN files that are completed before restarting the process with the remaining PGN files.

There is a backdoor : create a file called override.ini and write the new ending date/time you want (by default the ending date/time is the starting date/time + 1 year). Then you have to move the PGN+PLAIN files that are completed.

Toolkit to train a net without gensfen nor selfplay

Posted: 24/01/2021, 15:44
by IbaiBuR
Ok thanks, I'll see. Another question, you suggest using multiple small pgns at the same time, but that will create several plain files. Is there a way to merge the plain files? Thanks for your attention and kindness.

Toolkit to train a net without gensfen nor selfplay

Posted: 24/01/2021, 15:54
by deeds
yes then you can combine your PLAIN TEXT files by a command like this :
copy /b *_plain.txt big_plain.txt

Toolkit to train a net without gensfen nor selfplay

Posted: 24/01/2021, 15:57
by deeds
Second tool => nnue_extract : https://mega.nz/folder/2ohFXYqY#JTLEKhVjypvyTRI5zFPenQ

ENJOY !

Toolkit to train a net without gensfen nor selfplay

Posted: 24/01/2021, 17:21
by IbaiBuR
Oh ok I had no idea about that. Where do I need to run that command? In the cmd I guess.

Toolkit to train a net without gensfen nor selfplay

Posted: 24/01/2021, 18:46
by deeds
yes into a CMD command console.

or you can use this trick :
stockfish.x86-64-bmi2.halfkp_256x2-32-32.profile-learn.2020-08-30.exe
learn convert_bin check_invalid_fen 1 check_illegal_move 1 output_file_name train_me.bin 01_plain.txt
learn convert_bin check_invalid_fen 1 check_illegal_move 1 output_file_name train_me.bin 02_plain.txt
learn convert_bin check_invalid_fen 1 check_illegal_move 1 output_file_name train_me.bin 03_plain.txt
...
quit

But at the end of this, you keep several PLAIN TEXT files but all their data are stored into only one NNUE BIN file.

Toolkit to train a net without gensfen nor selfplay

Posted: 24/01/2021, 18:56
by IbaiBuR
Ok thats easier for me. Thanks for your help a ton.I wouldnt have realised how to do it on my own.

Toolkit to train a net without gensfen nor selfplay

Posted: 25/01/2021, 9:04
by deeds