Toolkit to train a net without gensfen nor selfplay

deeds · Post by **deeds** » 23/01/2021, 21:16

Hi all ! I plan to offer some private tools to reuse available data in order to train some nets.

First tool => nnue_eval : https://mega.nz/folder/OtRRlSLZ#ZfmDUuDTI7NIRQcdqcX6SA

Enjoy !

Post by **IbaiBuR** » 23/01/2021, 22:41

Thanks a lot for your tool! It is very simple to use and I have always wanted to train a net based on pgns but had failed rescoring with different programs. It is great that you shared this.

hazsan88 · Post by **hazsan88** » 23/01/2021, 23:27

Thank you very much for the tool that's shared
then, how to convert Plain Text into Bin

deeds · Post by **deeds** » 24/01/2021, 5:12

My pleasure !

From PLAIN TEXT format to NNUE BIN format, you can use the (LEARN) CONVERT_BIN command from any NODCHIP executables.

ZamChess · Post by **ZamChess** » 24/01/2021, 7:39

deeds wrote:My pleasure !

From PLAIN TEXT format to NNUE BIN format, you can use the (LEARN) CONVERT_BIN command from any NODCHIP executables.

Where can I find NODCHIP executables ?...

deeds · Post by **deeds** » 24/01/2021, 7:41

https://github.com/nodchip/Stockfish/releases

Post by **IbaiBuR** » 24/01/2021, 14:38

Hello, is it possible to resume rescoring or not? It says 3 days left and I wont be able to have my laptop that time running.

deeds · Post by **deeds** » 24/01/2021, 15:00

Hard to say like that ...

Scenario # 1: converting a single large PGN file
The tool is optimized for multitasking so you should split your large PGN file into several small PGN files.

Scenario # 2: converting multiple PGN files simultaneously
It will suffice to move the PGN+PLAIN files that are completed before restarting the process with the remaining PGN files.

There is a backdoor : create a file called override.ini and write the new ending date/time you want (by default the ending date/time is the starting date/time + 1 year). Then you have to move the PGN+PLAIN files that are completed.

Post by **IbaiBuR** » 24/01/2021, 15:44

Ok thanks, I'll see. Another question, you suggest using multiple small pgns at the same time, but that will create several plain files. Is there a way to merge the plain files? Thanks for your attention and kindness.

deeds · Post by **deeds** » 24/01/2021, 15:54

yes then you can combine your PLAIN TEXT files by a command like this :
copy /b *_plain.txt big_plain.txt

deeds · Post by **deeds** » 24/01/2021, 15:57

Second tool => nnue_extract : https://mega.nz/folder/2ohFXYqY#JTLEKhVjypvyTRI5zFPenQ

ENJOY !

Post by **IbaiBuR** » 24/01/2021, 17:21

Oh ok I had no idea about that. Where do I need to run that command? In the cmd I guess.

deeds · Post by **deeds** » 24/01/2021, 18:46

yes into a CMD command console.

or you can use this trick :
stockfish.x86-64-bmi2.halfkp_256x2-32-32.profile-learn.2020-08-30.exe
learn convert_bin check_invalid_fen 1 check_illegal_move 1 output_file_name train_me.bin 01_plain.txt
learn convert_bin check_invalid_fen 1 check_illegal_move 1 output_file_name train_me.bin 02_plain.txt
learn convert_bin check_invalid_fen 1 check_illegal_move 1 output_file_name train_me.bin 03_plain.txt
...
quit

But at the end of this, you keep several PLAIN TEXT files but all their data are stored into only one NNUE BIN file.

Post by **IbaiBuR** » 24/01/2021, 18:56

Ok thats easier for me. Thanks for your help a ton.I wouldnt have realised how to do it on my own.

deeds · Post by **deeds** » 25/01/2021, 9:04

Third tool => nnue_clean : https://mega.nz/folder/bsp12SpY#RAqR0trXov0XC0pvXvp_-w

ENJOY !

Outskirts CheSS ForuM

Outskirts CheSS ForuM