Compiling Stockfish with gcc on Termux

Archimedes · Post by **Archimedes** » 26/04/2023, 20:37

kramnik wrote: ↑26/04/2023, 19:57 It's kinda trying to teach to a 23 years in service cop how to fire with a gun... Linux was my first system back in '99 amongst the ones still being used. Before - from '87 until '98 - I used KERNAL.

But even an old policeman has to do target practice now and then.

kramnik · Post by **kramnik** » 26/04/2023, 20:39

On another hand: when I compiled BrainLearn the past year I discovered that it creates an experience.bin file under Termux, but when I moved outside it in the Droidfish folder it stopped to create experience.bin. Do you know if installing it as OEX engine it's allowed to create and use experience.bin? Otherwise I can't see the point to port BrainLearn on Android at all, since we already have to remove libraries from it.

Archimedes wrote: ↑26/04/2023, 20:19
kramnik wrote: ↑26/04/2023, 19:57 Are we sure that commenting a library out of nowhere is a wise thing to do?... I guess that malloc was placed there for a reason.
Leave the answer to the compiler if you are not sure.

kramnik · Post by **kramnik** » 26/04/2023, 20:41

Archimedes wrote: ↑26/04/2023, 20:37 But even an old policeman has to do target practice now and then.

Not (yet) old! Just experienced.

I hope janus won't read this thread, he could even be my father...

Archimedes · Post by **Archimedes** » 26/04/2023, 20:54

To shorten the story, below are the changes CECSA makes to the source code to make BrainLearn work on Android.

Code: Select all

# Modifications
if [ "${CECSA_DISTRIBUTION_BUILD,,}" == "true" ]; then
  sed -i 's/-march=native/-march=armv8-a'/ ./Makefile
fi
if [ "${CECSA_EMBEDDED_NETWORK,,}" == "true" ]; then
  sed -i 's/-DNNUE_EMBEDDING_OFF//' ./Makefile
fi
if [ "${CECSA_LIVE_BOOK,,}" == "false" ]; then
  sed -i 's/-DUSE_LIVEBOOK//' ./Makefile
  sed -i 's/-Wl,-Bstatic .* -Wl,-Bdynamic//' ./Makefile
  sed -i 's/-DUSE_LIVEBOOK//' ../jni/Android.mk
  sed -i '0,/include \$(CLEAR_VARS)/s/include \$(CLEAR_VARS)/# include \$(CLEAR_VARS)/' ../jni/Android.mk
  sed -i 's/LOCAL_MODULE := curl/# LOCAL_MODULE := curl/' ../jni/Android.mk
  sed -i 's/LOCAL_SRC_FILES := \/data\/data\/com.termux\/files\/usr\/lib\/libcurl.so/# LOCAL_SRC_FILES := \/data\/data\/com.termux\/files\/usr\/lib\/libcurl.so/' ../jni/Android.mk
  sed -i 's/include \$(PREBUILT_SHARED_LIBRARY)/# include \$(PREBUILT_SHARED_LIBRARY)/' ../jni/Android.mk
  sed -i 's/LOCAL_SHARED_LIBRARIES := curl/# LOCAL_SHARED_LIBRARIES := curl/' ../jni/Android.mk
fi
sed -i 's/map_path("experience"/map_path("\/storage\/emulated\/0\/experience"/g' ./learn.cpp
sed -i 's/map_path("experience-"/map_path("\/storage\/emulated\/0\/experience-"/g' ./learn.cpp
sed -i 's/map_path("experience_new-"/map_path("\/storage\/emulated\/0\/experience_new-"/g' ./learn.cpp
sed -i 's/experience\.bin/\/storage\/emulated\/0\/experience.bin/g' ./learn.cpp
sed -i 's/experience_new\.bin/\/storage\/emulated\/0\/experience_new.bin/g' ./learn.cpp
sed -i 's/#include <mm_malloc.h>/\/\/ #include <mm_malloc.h>'/ ./misc.h
if ! [[ "$2" == "" ]]; then
  sed -i 's/ersion = ".*";/ersion = "'$2'";'/ ./misc.cpp
fi
sed -i 's/#include <sys\/timeb.h>/\/\/ #include <sys\/timeb.h>'/ ./polybook.cpp
sed -i 's/#include <curl\/curl.h>/#include "\/data\/data\/com.termux\/files\/usr\/include\/curl\/curl.h"'/ ./search.cpp
if [[ "$CECSA_COMPILER" == "gcc" ]] && [ "${CECSA_PGO,,}" == "true" ] && [ "${CECSA_COMPILER_GCC_COMMUNITY,,}" == "false" ]; then
  sed -i 's/using namespace std;/#if (__GNUC__ == 10 \&\& __GNUC_MINOR__ == 3 \&\& __GNUC_PATCHLEVEL__ == 0)\n  extern "C" void __gcov_dump();\n#endif\n\nusing namespace std;'/ ./uci.cpp
  sed -i 's/elapsed << endl;/elapsed << endl;\n\n    #if (__GNUC__ == 10 \&\& __GNUC_MINOR__ == 3 \&\& __GNUC_PATCHLEVEL__ == 0)\n      __gcov_dump();\n    #endif'/ ./uci.cpp
fi

kramnik · Post by **kramnik** » 27/04/2023, 12:35

Do you know if installing it as OEX engine it's allowed to create and use experience.bin?

Archimedes wrote: ↑26/04/2023, 20:54 To shorten the story, below are the changes CECSA makes to the source code to make BrainLearn work on Android.

Code: Select all

# Modifications
if [ "${CECSA_DISTRIBUTION_BUILD,,}" == "true" ]; then
  sed -i 's/-march=native/-march=armv8-a'/ ./Makefile
fi
if [ "${CECSA_EMBEDDED_NETWORK,,}" == "true" ]; then
  sed -i 's/-DNNUE_EMBEDDING_OFF//' ./Makefile
fi
if [ "${CECSA_LIVE_BOOK,,}" == "false" ]; then
  sed -i 's/-DUSE_LIVEBOOK//' ./Makefile
  sed -i 's/-Wl,-Bstatic .* -Wl,-Bdynamic//' ./Makefile
  sed -i 's/-DUSE_LIVEBOOK//' ../jni/Android.mk
  sed -i '0,/include \$(CLEAR_VARS)/s/include \$(CLEAR_VARS)/# include \$(CLEAR_VARS)/' ../jni/Android.mk
  sed -i 's/LOCAL_MODULE := curl/# LOCAL_MODULE := curl/' ../jni/Android.mk
  sed -i 's/LOCAL_SRC_FILES := \/data\/data\/com.termux\/files\/usr\/lib\/libcurl.so/# LOCAL_SRC_FILES := \/data\/data\/com.termux\/files\/usr\/lib\/libcurl.so/' ../jni/Android.mk
  sed -i 's/include \$(PREBUILT_SHARED_LIBRARY)/# include \$(PREBUILT_SHARED_LIBRARY)/' ../jni/Android.mk
  sed -i 's/LOCAL_SHARED_LIBRARIES := curl/# LOCAL_SHARED_LIBRARIES := curl/' ../jni/Android.mk
fi
sed -i 's/map_path("experience"/map_path("\/storage\/emulated\/0\/experience"/g' ./learn.cpp
sed -i 's/map_path("experience-"/map_path("\/storage\/emulated\/0\/experience-"/g' ./learn.cpp
sed -i 's/map_path("experience_new-"/map_path("\/storage\/emulated\/0\/experience_new-"/g' ./learn.cpp
sed -i 's/experience\.bin/\/storage\/emulated\/0\/experience.bin/g' ./learn.cpp
sed -i 's/experience_new\.bin/\/storage\/emulated\/0\/experience_new.bin/g' ./learn.cpp
sed -i 's/#include <mm_malloc.h>/\/\/ #include <mm_malloc.h>'/ ./misc.h
if ! [[ "$2" == "" ]]; then
  sed -i 's/ersion = ".*";/ersion = "'$2'";'/ ./misc.cpp
fi
sed -i 's/#include <sys\/timeb.h>/\/\/ #include <sys\/timeb.h>'/ ./polybook.cpp
sed -i 's/#include <curl\/curl.h>/#include "\/data\/data\/com.termux\/files\/usr\/include\/curl\/curl.h"'/ ./search.cpp
if [[ "$CECSA_COMPILER" == "gcc" ]] && [ "${CECSA_PGO,,}" == "true" ] && [ "${CECSA_COMPILER_GCC_COMMUNITY,,}" == "false" ]; then
  sed -i 's/using namespace std;/#if (__GNUC__ == 10 \&\& __GNUC_MINOR__ == 3 \&\& __GNUC_PATCHLEVEL__ == 0)\n  extern "C" void __gcov_dump();\n#endif\n\nusing namespace std;'/ ./uci.cpp
  sed -i 's/elapsed << endl;/elapsed << endl;\n\n    #if (__GNUC__ == 10 \&\& __GNUC_MINOR__ == 3 \&\& __GNUC_PATCHLEVEL__ == 0)\n      __gcov_dump();\n    #endif'/ ./uci.cpp
fi

Archimedes · Post by **Archimedes** » 27/04/2023, 12:52

kramnik wrote: ↑27/04/2023, 12:35 Do you know if installing it as OEX engine it's allowed to create and use experience.bin?

The experience file is created in the /storage/emulated/0 directory. Yes, it works.

angel44e · Post by **angel44e** » 22/06/2023, 7:00

Hi, Archie! I've done these changes in the Makefile (as in your first post on that topic ).And I wish to compile Stockfish by gcc on Termux.All it's done successfully

Code: Select all

~/.../stocktact/src $ make -j build ARCH=armv8 COMP=gcc                                         Default net: nn-cd2ff4716c34.nnue               nn-cd2ff4716c34.nnue available.                 Network validated                                                                               Config:                                         debug: 'no'                                     sanitize: 'none'                                optimize: 'yes'                                 arch: 'armv8'                                   bits: '64'                                      kernel: 'Linux'                                 os: 'Android'                                   prefetch: 'yes'                                 popcnt: 'yes'                                   pext: 'no'                                      sse: 'no'                                       mmx: 'no'                                       sse2: 'no'                                      ssse3: 'no'                                     sse41: 'no'                                     avx2: 'no'                                      avxvnni: 'no'                                   avx512: 'no'                                    vnni256: 'no'                                   vnni512: 'no'                                   neon: 'yes'                                     dotprod: 'no'                                   arm_version: '8'                                target_windows: ''                                                                              Flags:                                          CXX: g++                                        CXXFLAGS:  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE                                           LDFLAGS:   -Wl,--no-as-needed  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE -flto=jobserver -fPIE -pie                                                                                            Testing config sanity. If this fails, try 'make help' ...                                                                                       make ARCH=armv8 COMP=gcc all                    make[1]: Entering directory '/storage/emulated/0/Download/stocktact/src'                        g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o benchmark.o benchmark.cpp               g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o bitbase.o bitbase.cpp                   g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o bitboard.o bitboard.cpp                 g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o endgame.o endgame.cpp                   g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o evaluate.o evaluate.cpp                 g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o main.o main.cpp                         g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o material.o material.cpp                 g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o misc.o misc.cpp                         g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o movegen.o movegen.cpp                   g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o movepick.o movepick.cpp                 g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o pawns.o pawns.cpp                       g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o position.o position.cpp                 g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o psqt.o psqt.cpp                         g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o search.o search.cpp                     g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o thread.o thread.cpp                     g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o timeman.o timeman.cpp                   g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o tt.o tt.cpp                             g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o uci.o uci.cpp                           g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o ucioption.o ucioption.cpp               g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o tune.o tune.cpp                         g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o tbprobe.o syzygy/tbprobe.cpp            g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o evaluate_nnue.o nnue/evaluate_nnue.cpp  g++  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE   -c -o half_ka_v2_hm.o nnue/features/half_ka_v2_hm.cpp                                         search.cpp:76:13: warning: unused variable 'LmrWeight' [-Wunused-const-variable]                  const int LmrWeight[2][6] = {                             ^                                   1 warning generated.                            g++ -o stockfish benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o evaluate_nnue.o half_ka_v2_hm.o   -Wl,--no-as-needed  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE -flto=jobserver -fPIE -pie  make[1]: Leaving directory '/storage/emulated/0/Download/stocktact/src'                         ~/.../stocktact/src $ strip stockfish           ~/.../stocktact/src $

But when I've tried to start in Droidfish it's failed to start.What's wrong with gcc compilation?

Archimedes · Post by **Archimedes** » 22/06/2023, 8:33

angel44e wrote: ↑22/06/2023, 7:00 Hi, Archie! I've done these changes in the Makefile (as in your first post on that topic ).And I wish to compile Stockfish by gcc on Termux.All it's done successfully

These were just quick and dirty changes to compile Stockfish on Android. I also don't even know if these changes still work. I now use my own makefile for this (see CECSA script).

Does the chess engine work under Termux?

Code: Select all

./stockfish bench

angel44e · Post by **angel44e** » 22/06/2023, 8:51

Archimedes wrote: ↑22/06/2023, 8:33
angel44e wrote: ↑22/06/2023, 7:00 Hi, Archie! I've done these changes in the Makefile (as in your first post on that topic ).And I wish to compile Stockfish by gcc on Termux.All it's done successfully
These were just quick and dirty changes to compile Stockfish on Android. I also don't even know if these changes still work. I now use my own makefile for this (see CECSA script).

Does the chess engine work under Termux?
Code: Select all
./stockfish bench

No, permission denied

Code: Select all

^
1 warning generated.
g++ -o stockfish benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o evaluate_nnue.o half_ka_v2_hm.o   -Wl,--no-as-needed  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE -flto=jobserver -fPIE -pie
make[1]: Leaving directory '/storage/emulated/0/Download/stocktact/src'
~/.../stocktact/src $ strip stockfish
~/.../stocktact/src $ ./stockfish bench
bash: ./stockfish: Permission denied
~/.../stocktact/src $

Archimedes · Post by **Archimedes** » 22/06/2023, 12:30

angel44e wrote: ↑22/06/2023, 8:51
Archimedes wrote: ↑22/06/2023, 8:33
angel44e wrote: ↑22/06/2023, 7:00 Hi, Archie! I've done these changes in the Makefile (as in your first post on that topic ).And I wish to compile Stockfish by gcc on Termux.All it's done successfully
These were just quick and dirty changes to compile Stockfish on Android. I also don't even know if these changes still work. I now use my own makefile for this (see CECSA script).

Does the chess engine work under Termux?
Code: Select all
./stockfish bench
No, permission denied
Code: Select all
^
1 warning generated.
g++ -o stockfish benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o evaluate_nnue.o half_ka_v2_hm.o   -Wl,--no-as-needed  -Wall -Wcast-qual -fno-exceptions -std=c++17  -pedantic -Wextra -Wshadow -Wmissing-declarations -DUSE_PTHREADS -DNDEBUG -O3 -march=native -DIS_64BIT -DUSE_POPCNT -DUSE_NEON=8 -flto -fPIE -flto=jobserver -fPIE -pie
make[1]: Leaving directory '/storage/emulated/0/Download/stocktact/src'
~/.../stocktact/src $ strip stockfish
~/.../stocktact/src $ ./stockfish bench
bash: ./stockfish: Permission denied
~/.../stocktact/src $

We already had that.

Please copy the chess engine into the home directory and make it executable or compile the whole thing in the home directory.

Outskirts CheSS ForuM

Outskirts CheSS ForuM