Compiling Stockfish with gcc on Termux

[kramnik]

...
Missing libs have to be statically binded/linked into the exe.
...

Is there any way to do that?

[MichaelM]

...
Missing libs have to be statically binded/linked into the exe.
...

Is there any way to do that?

Archimedes wrote: 'I bet, we need a static version of the curl library.'
I won't bet against him.

[kramnik]

So, it's a matter for the curl developers...

...
Missing libs have to be statically binded/linked into the exe.
...

Is there any way to do that?

Archimedes wrote: 'I bet, we need a static version of the curl library.'
I won't bet against him.

[leonhurricaneX]

...
Missing libs have to be statically binded/linked into the exe.
...

Is there any way to do that?

Archimedes wrote: 'I bet, we need a static version of the curl library.'
I won't bet against him.

Static version of curl is already available in Termux.
Use pkg install libcurl-static

[Archimedes]

Static version of curl is already available in Termux.
Use pkg install libcurl-static

I tried that already with Android NDK. I included libcurl.a and all the other static libraries which libcurl.a depends on as prebuild static libraries, but there was still an error at the end (during the linking process). When linking to libcurl.a only (and nothing else), there was a lot of error messages regarding missing libraries. After adding them too (they are all part of the libcurl-static libraries), the error messages about missing libraries disappeared. But one error message still remains at the end.

For test purposes, i compiled ShashChess again with Android NDK. The shared library libcurl.so is now included in the zip archive. Don't know wether this works and if, this should not be the solution. The file libcurl.so must be in the "current" directory of the engine. For DroidFish this means, it should be copied to the logs directory. On my devices i do not need that (ShashChess, compiled with Clang or Android NDK, works on all of my devices).

ShashChess 19.2:
https://app.box.com/s/rcw8s69126p95vuaodsnjc28sayl8trn

[Archimedes]

...
Missing libs have to be statically binded/linked into the exe.
...

Is there any way to do that?

Embedding the curl library into the executable means, that all the libraries which curl depends on must also be embedded. And there are a few of them. To make the story short, the LDFLAGS in the makefiles from BrainLearn and ShashChess has to be changed as follow:

Before:

Code: Select all

LDFLAGS += -lcurl $(EXTRALDFLAGS)

After:

Code: Select all

LDFLAGS += -Wl,-Bstatic -lcrypto -lcurl -ldl -lnghttp2 -lssh2 -lssl -lz -Wl,-Bdynamic $(EXTRALDFLAGS)

When you get an error during the linking process, it is because you have not installed all static libraries which curl is needing.

You may need the following static libraries for proper compiling with Clang (if not already installed):

Code: Select all

apt-get install libcurl-static
apt-get install libnghttp2-static
apt-get install libssh2-static
apt-get install openssl-static
apt-get install zlib-static

[Archimedes]

The following LDFLAGS are needed for proper compiling BrainLearn and ShashChess with Clang and GCC compiler (GCC needs the search path for finding the necessary libraries). The package ndk-multilib should also be installed.

Code: Select all

LDFLAGS += -Wl,-Bstatic -L/data/data/com.termux/files/usr/aarch64-linux-android/lib -ldl -L/data/data/com.termux/files/usr/lib -lcrypto -lcurl -lnghttp2 -lssh2 -lssl -lz -Wl,-Bdynamic $(EXTRALDFLAGS)

[kramnik]

Still getting this error, making the change below and compiling (BrainLearn) with make -j profile-build ARCH=armv8 COMP=clang:

ld.lld: error: unable to find library -lws2_32
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [Makefile:876: stockfish] Error 1

Add -static-libstdc++ to one of your LDFLAGS for clang and also make sure, you have installed ndk-multilib.

Example:

Code: Select all

ifeq ($(arch),$(filter $(arch),armv7 armv8))
	ifeq ($(OS),Android)
		CXXFLAGS += -m$(bits)
		LDFLAGS += -m$(bits) -static-libstdc++

[leonhurricaneX]

Still getting this error, making the change below and compiling (BrainLearn) with make -j profile-build ARCH=armv8 COMP=clang:

ld.lld: error: unable to find library -lws2_32
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [Makefile:876: stockfish] Error 1

Add -static-libstdc++ to one of your LDFLAGS for clang and also make sure, you have installed ndk-multilib.

Example:

Code: Select all

ifeq ($(arch),$(filter $(arch),armv7 armv8))
	ifeq ($(OS),Android)
		CXXFLAGS += -m$(bits)
		LDFLAGS += -m$(bits) -static-libstdc++

you need to remove -lws2_32 from line 320 in Makefile.

[kramnik]

Still getting error trying to compile 32-bit:

Code: Select all

make -j profile-build ARCH=armv7 COMP=gcc

Code: Select all

make[2]: g++: No such file or directory
make[2]: *** [<builtin>: benchmark.o] Error 127
make[2]: Leaving directory '/data/data/com.termux/files/home/Stockfish/src'
make[1]: *** [Makefile:931: gcc-profile-make] Error 2
make[1]: Leaving directory '/data/data/com.termux/files/home/Stockfish/src'
make: *** [Makefile:786: profile-build] Error 2

[LittleJoe]

Still getting error trying to compile 32-bit:

Code: Select all

make -j profile-build ARCH=armv7 COMP=gcc

Code: Select all

make[2]: g++: No such file or directory
make[2]: *** [<builtin>: benchmark.o] Error 127
make[2]: Leaving directory '/data/data/com.termux/files/home/Stockfish/src'
make[1]: *** [Makefile:931: gcc-profile-make] Error 2
make[1]: Leaving directory '/data/data/com.termux/files/home/Stockfish/src'
make: *** [Makefile:786: profile-build] Error 2

armv7 builds with gcc are difficult or impossible, but NDK works fine:

Code: Select all

make -j build ARCH=armv7-neon COMP=ndk

Maybe also pgo works, but it seems to depend on the clang versions installed in Termux and in the NDK. Just try:

Code: Select all

make -j profile-build ARCH=armv7-neon COMP=ndk

[kramnik]

Before crashing my phone I was able to do it, I followed maybe a post by Archimedes. Now they gave me back the phone from the technical support and it's empty, of course. I need to find the library I installed to compile for 32-bit. I was never able to compile for neon on Termux, anyway. I'm just interested in standard 32-bit compilation.

Still getting error trying to compile 32-bit:

Code: Select all

make -j profile-build ARCH=armv7 COMP=gcc

Code: Select all

make[2]: g++: No such file or directory
make[2]: *** [<builtin>: benchmark.o] Error 127
make[2]: Leaving directory '/data/data/com.termux/files/home/Stockfish/src'
make[1]: *** [Makefile:931: gcc-profile-make] Error 2
make[1]: Leaving directory '/data/data/com.termux/files/home/Stockfish/src'
make: *** [Makefile:786: profile-build] Error 2

armv7 builds with gcc are difficult or impossible, but NDK works fine:

Code: Select all

make -j build ARCH=armv7-neon COMP=ndk

Maybe also pgo works, but it seems to depend on the clang versions installed in Termux and in the NDK. Just try:

Code: Select all

make -j profile-build ARCH=armv7-neon COMP=ndk

[Archimedes]

If you have a 64 bit environment, you are only able to compile for 64 bit devices with gcc and clang. Please correct me, if I'm wrong. For cross compiling you need Android NDK, where you have different methods to do that. One of them is ndk-build. I already described this way.

Here is a short description, how you can do this for Stockfish.

When Android NDK is installed (root directory is in the search path), you need a jni directory (parallel to the src directory) with the following two files, when you want to compile Stockfish for different platforms.

Android.mk:

Code: Select all

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := Stockfish
LOCAL_C_INCLUDES := 
LOCAL_SRC_FILES := $(wildcard ../src/*.cpp ../src/nnue/*.cpp ../src/nnue/features/*.cpp ../src/syzygy/*.cpp)
LOCAL_CFLAGS += -std=c++17 -DNDEBUG -O3 -flto -Wfatal-errors -Wall -Wextra -Wshadow -DNNUE_EMBEDDING_OFF -DUSE_PTHREADS
LOCAL_CPPFLAGS += 

ifeq ($(TARGET_ARCH_ABI), arm64-v8a)
  LOCAL_CFLAGS += -DIS_64BIT -DUSE_NEON=8 -DUSE_POPCNT
else
  ifeq ($(TARGET_ARCH_ABI), armeabi-v7a)
    LOCAL_CFLAGS += -DUSE_NEON=7 -DUSE_POPCNT
  else
    ifeq ($(TARGET_ARCH_ABI), x86)
      LOCAL_CFLAGS += -mssse3 -mfpmath=sse -m32 -DUSE_SSSE3 -DUSE_SSE2 -DUSE_MMX
    else
      ifeq ($(TARGET_ARCH_ABI), x86_64)
        LOCAL_CFLAGS += -msse4.2 -mpopcnt -m64 -DIS_64BIT -DUSE_SSE41 -DUSE_SSSE3 -DUSE_SSE2 -DUSE_MMX -DUSE_POPCNT
      endif
    endif
  endif
endif

LOCAL_LDFLAGS += -flto
include $(BUILD_EXECUTABLE)

Application.mk:

Code: Select all

APP_ABI := 
APP_PLATFORM := android-21
APP_STL := c++_static

Parameters for Application.mk are described here: https://developer.android.com/ndk/guides/application_mk. All you have to do, is, go into the jni directory and call ndk-build. Works on Linux, Mac and Windows. And on Termux for sure.

[kramnik]

Solved. It was a stupid missing.

With

export PATH=$PREFIX/../cctools-toolchain/bin:$PATH

I am able to compile with

make -j profile-build ARCH=armv7 COMP=gcc

(32-bit)

How can I insert $PREFIX/../cctools-toolchain/bin into the PATH without I have to manually enter it any time?

[Archimedes]

Look for .bashrc in your home directory and add the line with the export command to it.