Compiling Stockfish with gcc on Termux

[Archimedes]

Here is a short description, how you can compile Stockfish for arm64-v8a based CPUs with gcc on Termux.

At first, forget the installation of gcc via the repository https://github.com/its-pointless/its-pointless.github.io. It works for Demolito, which is a C based chess engine, but not for Stockfish (and probably other C++ based chess engines).

If you have already installed gcc via the above repository, remove it with:

Code: Select all

apt-get remove gcc-11
apt-get autoremove

Use CCTools for that purpose. A detailed description, how you install the cctool packages under Termux can be found here:
https://cctools.info/index.php/Termux_repository_with_cctools_packages_(gcc_with_fortran_support_etc)

Follow the explanations step by step.

After that, download the Stockfish sources.

Code: Select all

git clone https://github.com/official-stockfish/Stockfish
cd Stockfish/src

Make changes to the makefile, as the following code pieces are shown:

Before:

Code: Select all

ifeq ($(COMP),gcc)
	comp=gcc
	CXX=g++
	CXXFLAGS += -pedantic -Wextra -Wshadow

	ifeq ($(arch),$(filter $(arch),armv7 armv8))
		ifeq ($(OS),Android)
			CXXFLAGS += -m$(bits)
			LDFLAGS += -m$(bits)

After:

Code: Select all

ifeq ($(COMP),gcc)
	comp=gcc
	CXX=g++
	CXXFLAGS += -pedantic -Wextra -Wshadow

	ifeq ($(arch),$(filter $(arch),armv7 armv8))
		ifeq ($(OS),Android)
			CXXFLAGS += 
			LDFLAGS += 

Before:

Code: Select all

### Sometimes gcc is really clang
ifeq ($(COMP),gcc)

After:

Code: Select all

### Sometimes gcc is really clang
ifeq ($(COMP),)

Before:

Code: Select all

### 3.3 Optimization
ifeq ($(optimize),yes)

	CXXFLAGS += -O3

	ifeq ($(comp),gcc)
		ifeq ($(OS), Android)
			CXXFLAGS += -fno-gcse -mthumb -march=armv7-a -mfloat-abi=softfp

After:

Code: Select all

### 3.3 Optimization
ifeq ($(optimize),yes)

	CXXFLAGS += -O3

	ifeq ($(comp),gcc)
		ifeq ($(OS), Android)
			CXXFLAGS += -march=native

Now you can compile Stockfish with gcc and PGO optimization with the following command:

Code: Select all

make -j profile-build ARCH=armv8 COMP=gcc

[kramnik]

Does the produced binary work outside Termux?

Here is a short description, how you can compile Stockfish for arm64-v8a based CPUs with gcc on Termux.

At first, forget the installation of gcc via the repository https://github.com/its-pointless/its-pointless.github.io. It works for Demolito, which is a C based chess engine, but not for Stockfish (and probably other C++ based chess engines).

If you have already installed gcc via the above repository, remove it with:

Code: Select all

apt-get remove gcc-11
apt-get autoremove

Use CCTools for that purpose. A detailed description, how you install the cctool packages under Termux can be found here:
https://cctools.info/index.php/Termux_repository_with_cctools_packages_(gcc_with_fortran_support_etc)

Follow the explanations step by step.

After that, download the Stockfish sources.

Code: Select all

git clone https://github.com/official-stockfish/Stockfish
cd Stockfish/src

Make changes to the makefile, as the following code pieces are shown:

Before:

Code: Select all

ifeq ($(COMP),gcc)
	comp=gcc
	CXX=g++
	CXXFLAGS += -pedantic -Wextra -Wshadow

	ifeq ($(arch),$(filter $(arch),armv7 armv8))
		ifeq ($(OS),Android)
			CXXFLAGS += -m$(bits)
			LDFLAGS += -m$(bits)

After:

Code: Select all

ifeq ($(COMP),gcc)
	comp=gcc
	CXX=g++
	CXXFLAGS += -pedantic -Wextra -Wshadow

	ifeq ($(arch),$(filter $(arch),armv7 armv8))
		ifeq ($(OS),Android)
			CXXFLAGS += 
			LDFLAGS += 

Before:

Code: Select all

### Sometimes gcc is really clang
ifeq ($(COMP),gcc)

After:

Code: Select all

### Sometimes gcc is really clang
ifeq ($(COMP),)

Before:

Code: Select all

### 3.3 Optimization
ifeq ($(optimize),yes)

	CXXFLAGS += -O3

	ifeq ($(comp),gcc)
		ifeq ($(OS), Android)
			CXXFLAGS += -fno-gcse -mthumb -march=armv7-a -mfloat-abi=softfp

After:

Code: Select all

### 3.3 Optimization
ifeq ($(optimize),yes)

	CXXFLAGS += -O3

	ifeq ($(comp),gcc)
		ifeq ($(OS), Android)
			CXXFLAGS += -march=native

Now you can compile Stockfish with gcc and PGO optimization with the following command:

Code: Select all

make -j profile-build ARCH=armv8 COMP=gcc

[Archimedes]

Does the produced binary work outside Termux?

Yes.

[kramnik]

AWESOME, IT WORKS!!

It produces a binary working on Droidfish at the same speed of the others but I got strange warnings:

Code: Select all

bitboard.cpp: In function '_GLOBAL__sub_I__ZN9Stockfish8PopCnt16E':
bitboard.cpp:222:1: warning: 'profdir/#data#data#com.termux#files#home#Stockfish#src#bitboard.gcda' profile count data file not found [-Wmissing-profile]
  222 | } // namespace Stockfish
      | ^
bitbase.cpp: In member function 'std::_Vector_base<Stockfish::(anonymous namespace)::KPKPosition, std::allocator<Stockfish::(anonymous namespace)::KPKPosition> >::_M_allocate(unsigned long)':
bitbase.cpp:172:1: warning: 'profdir/#data#data#com.termux#files#home#Stockfish#src#bitbase.gcda' profile count data file not found [-Wmissing-profile]
  172 | } // namespace Stockfish
      | ^
tune.cpp: In function '_GLOBAL__sub_I__ZN9Stockfish4Tune14update_on_lastE':
tune.cpp:133:1: warning: 'profdir/#data#data#com.termux#files#home#Stockfish#src#tune.gcda' profile count data file not found [-Wmissing-profile]
  133 | } // namespace Stockfish
      | ^
psqt.cpp: In function 'Stockfish::PSQT::init()':
psqt.cpp:131:1: warning: 'profdir/#data#data#com.termux#files#home#Stockfish#src#psqt.gcda' profile count data file not found [-Wmissing-profile]
  131 | } // namespace Stockfish
      | ^
pawns.cpp: In function '_GLOBAL__sub_I__ZN9Stockfish5Pawns5probeERKNS_8PositionE':
pawns.cpp:305:1: warning: 'profdir/#data#data#com.termux#files#home#Stockfish#src#pawns.gcda' profile count data file not found [-Wmissing-profile]
  305 | } // namespace Stockfish
      | ^
movegen.cpp: In function '_GLOBAL__sub_I__ZN9Stockfish8generateILNS_7GenTypeE5EEEPNS_7ExtMoveERKNS_8PositionES3_':
movegen.cpp:276:1: warning: 'profdir/#data#data#com.termux#files#home#Stockfish#src#movegen.gcda' profile count data file not found [-Wmissing-profile]

Does the produced binary work outside Termux?

Yes.

[MichaelM]

AWESOME, IT WORKS!!

It produces a binary working on Droidfish at the same speed of the others but I got strange warnings:

Archimedes and I get also warnings. We discussed this here:
http://outskirts.altervista.org/forum/viewtopic.php?f=40&t=2688

Have you tried the CECSA script? It works fine!

[kramnik]

That's nice! But I prefer go compile by hand, since I work with singular distributions time after time. Nice idea, anyway.

AWESOME, IT WORKS!!

It produces a binary working on Droidfish at the same speed of the others but I got strange warnings:

Archimedes and I get also warnings. We discussed this here:
http://outskirts.altervista.org/forum/viewtopic.php?f=40&t=2688

Have you tried the CECSA script? It works fine!

[MichaelM]

That's nice! But I prefer go compile by hand, since I work with singular distributions time after time. Nice idea,

I also prefer start 'make....' by myself but essentially the script does the same. Just compare the warnings if you are concerned about them

[LittleJoe]

I also get many warnings. It doesn't matter if I use make or the cecsa start script. But the engines are really fast, so let the warnings warn. How cares
And BTW: embedded and non-embedded engines are equally fast. Only one is 46 times larger.

[kramnik]

That's not the problem: the script compiles different engines, while I just have to compile one.

That's nice! But I prefer go compile by hand, since I work with singular distributions time after time. Nice idea,

I also prefer start 'make....' by myself but essentially the script does the same. Just compare the warnings if you are concerned about them

[LittleJoe]

That's not the problem: the script compiles different engines, while I just have to compile one.

That's nice! But I prefer go compile by hand, since I work with singular distributions time after time. Nice idea,

I also prefer start 'make....' by myself but essentially the script does the same. Just compare the warnings if you are concerned about them

No.
It compiles only one, if you do not use 'all'.
And 'all' is mainly used for testing.

[kramnik]

How can I compile 32 bits binaries? make -j profile-build ARCH=x86-32 COMP=gcc gives these errors:

g++: error: unrecognized command-line option '-m32'
g++: error: unrecognized command-line option '-mthumb'
g++: error: unrecognized command-line option '-m32'
g++: error: unrecognized command-line option '-m32'
g++: error: unrecognized command-line option '-mthumb'
g++: error: unrecognized command-line option '-mthumb'
g++: error: unrecognized command-line option '-m32'
g++: error: unrecognized command-line option '-mthumb'
g++: error: unrecognized command-line option '-m32'
g++: error: unrecognized command-line option '-mthumb'

Does the produced binary work outside Termux?

Yes.

[LittleJoe]

How can I compile 32 bits binaries? make -j profile-build ARCH=x86-32 COMP=gcc gives these errors:

g++: error: unrecognized command-line option '-m32'
g++: error: unrecognized command-line option '-mthumb'
g++: error: unrecognized command-line option '-m32'
g++: error: unrecognized command-line option '-m32'
g++: error: unrecognized command-line option '-mthumb'
g++: error: unrecognized command-line option '-mthumb'
g++: error: unrecognized command-line option '-m32'
g++: error: unrecognized command-line option '-mthumb'
g++: error: unrecognized command-line option '-m32'
g++: error: unrecognized command-line option '-mthumb'

Does the produced binary work outside Termux?

Yes.

Doesn't work for me either. I also would like to have engines for my old devices.

[Archimedes]

I also get many warnings. It doesn't matter if I use make or the cecsa start script. But the engines are really fast, so let the warnings warn. How cares
And BTW: embedded and non-embedded engines are equally fast. Only one is 46 times larger.

The warnings occurs during the pgo optimization, when the final, optimized build where created. Looks to me that this is something like a false positive warning message from gcc which can be ignored. If there would be really a problem with a missing profile, we would have a massive decrease with nps which we are don't have. Furthermore all other compiles with gcc (Windows, Linux etc.) would also have problems, because they are using the same steps for pgo optimization provided by the makefile from Stockfish. However, the clang compiler outputs no warning messages. Therefore i disabled this warning message explicit for the gcc compiler with the flag -Wno-missing-profile in the makefiles (will be included in the next update of CECSA).

[kramnik]

Finally compared the binaries:

Benchs of gcc version under Termux on Samsung Galaxy S20 Ultra 5G:

Total time (ms) : 11329
Nodes searched : 7050445
Nodes/second : 622336

Total time (ms) : 11269
Nodes searched : 7050445
Nodes/second : 625649

Benchs of ndk clang version under linux virtual machine:

Total time (ms) : 11582
Nodes searched : 7050445
Nodes/second : 608741

Total time (ms) : 11754
Nodes searched : 7050445
Nodes/second : 599833


I don't think the Termux way worths the effort, especially in view of a possible package removing as it happened for the "pointless" github one, while the method shown here https://chess.massimilianogoi.com/articles/30-09-20-How-To-Compile-Stockfish-And-Stockfish-like-Programs-For-Android/ is likely universal and easier to do, in general.

But I have to do that it's cool as f... to compile Stockfish laid down on the couch by just tapping on my phone. Damn, these objects are really addictive, especially for me being struck by discal hernias.

[kramnik]

I was able to compile BrainLearn on Termux using clang and it worked until it was in that environment, but as soon as I exported it outside Termux and in the Droidfish uci folder it stopped to work... Any idea?

I also get many warnings. It doesn't matter if I use make or the cecsa start script. But the engines are really fast, so let the warnings warn. How cares
And BTW: embedded and non-embedded engines are equally fast. Only one is 46 times larger.

The warnings occurs during the pgo optimization, when the final, optimized build where created. Looks to me that this is something like a false positive warning message from gcc which can be ignored. If there would be really a problem with a missing profile, we would have a massive decrease with nps which we are don't have. Furthermore all other compiles with gcc (Windows, Linux etc.) would also have problems, because they are using the same steps for pgo optimization provided by the makefile from Stockfish. However, the clang compiler outputs no warning messages. Therefore i disabled this warning message explicit for the gcc compiler with the flag -Wno-missing-profile in the makefiles (will be included in the next update of CECSA).