Chess Engine Compiler Script for Android

[superbatil]

removing "-j" and problem solved.

This was also my sneaking suspicion. With CECSA i always use -j 2 for compiling which is fast enough.

I think if you write "-j" without number the default core use are max

[LittleJoe]

Archimedes made some Makefile updates for SF and clones and probably also some other changes in CECSA_220602:

https://sourceforge.net/projects/chess-tools-for-android/files/cecsa_2022.06.02.tar.gz/download

[Archimedes]

Removed Demolito, Fire and Stockfish+. Big thanks goes to MichaelM for the hint, that PGO never worked with GCC in my script. Ignoring the warning messages was a big mistake. Without implementing the function "__gcov_flush()" PGO doesn't work with the GCC compiler.

CECSA now takes care about that and make the necessary changes to the source code (e.g. on Stockfish, changes are necessary in uci.cpp). Stockfish is now around 6 % faster than before.

CECSA 2022.06.05:
https://sourceforge.net/projects/chess-tools-for-android/files/cecsa_2022.06.05.tar.gz/download

[Archimedes]

Replaced __gcov_flush() with __gcov_dump().

CECSA 2022.06.07:
https://sourceforge.net/projects/chess-tools-for-android/files/cecsa_2022.06.07.tar.gz/download

[Archimedes]

Updated Koivisto.

CECSA 2022.06.10:
https://sourceforge.net/projects/chess-tools-for-android/files/cecsa_2022.06.10.tar.gz/download

[Archimedes]

I have now replaced the terrible makefiles from BrainLearn, CorChess, Crystal, Polyfish, ShashChess and Stockfish with my own makefiles in CECSA style. By using the existing files for Android NDK (Android.mk and Application.mk) and the original makefiles, it was easy to create corresponding makefiles for CECSA. I have only used the parameters which are really necessary and profitable. Speed of the resulting binaries should be roughly the same as with the original makefiles. All supported chess engines now have their own makefiles.

CECSA 2022.09.01:
https://sourceforge.net/projects/chess-tools-for-android/files/cecsa_2022.09.01.tar.gz/download

[Archimedes]

PGO can be further optimized with the parameters -fno-peel-loops -fno-tracer (seen in the makefile from Stockfish and already included in the makefiles from CECSA). Which I can confirm after my own tests. Beside that GCC (still using 10.3.0) only requires the following parameters for Stockfish:

Code: Select all

-std=c++17
-O3
-flto
-fno-gcse
-march=native
-DNDEBUG
-DIS_64BIT
-DUSE_NEON=8
-DUSE_POPCNT
-DUSE_PTHREADS

Only a few parameters are needed.

Of course, the makefile for Stockfish can also be used outside of CECSA. Just copy the makefile to the src directory and enter the following commands in Termux (assuming you have a working environment for GCC):

Code: Select all

make net
make profile-build

[Archimedes]

Added a new parameter CECSA_COMPILER_GCC_DISTRIBUTION_BUILD to easily switch between distribution builds (optimized for armv8-a based devices) and non distribution builds (optimized for the host CPU) when using the GCC compiler.

When distributing builds, -march=native is not recommended when using the GCC compiler. In my case, it doesn't matter whether i use -march=native or -march=armv8-a, I always get the same binaries on my smartphone and tablet (because they are armv8-a based devices). But the situation could differ when compiling with -march=native on a newer mobile device. The resulted binaries may not run on older devices.

All makefiles in CECSA are using the parameter -march=native in the GCC section. As it is not very handy to change the parameter in all makefiles whenever i want to distribute builds, I've introduced a new parameter CECSA_COMPILER_GCC_DISTRIBUTION_BUILD in CECSA. When CECSA_COMPILER_GCC_DISTRIBUTION_BUILD is true, the script changes -march=native (if present) to -march=armv8-a in the GCC section of the makefile. When CECSA_COMPILER_GCC_DISTRIBUTION_BUILD is false, nothing happens and GCC will compile with -march=native (as defined in the makefiles).

Default is CECSA_COMPILER_GCC_DISTRIBUTION_BUILD=true. When you only want to compile for yourself, change it to false and GCC will compile with -march=native (recommended for your own mobile device).

CECSA 2022.09.03:
https://sourceforge.net/projects/chess-tools-for-android/files/cecsa_2022.09.03.tar.gz/download

[Archimedes]

Changed the makefiles from BrainLearn, CorChess, Crystal, Polyfish, ShashChess and Stockfish due to changes to the original makefile from Stockfish which introduces a download fallback for the network.

CECSA 2022.09.07:
https://sourceforge.net/projects/chess-tools-for-android/files/cecsa_2022.09.07.tar.gz/download

[Archimedes]

https://sourceforge.net/p/chess-tools-for-android/blog/

CECSA 2022.09.14:
https://sourceforge.net/projects/chess-tools-for-android/files/cecsa_2022.09.14.tar.gz/download

[Archimedes]

https://sourceforge.net/p/chess-tools-for-android/blog/

CECSA 2022.09.24:
https://sourceforge.net/projects/chess-tools-for-android/files/cecsa_2022.09.24.tar.gz/download

[Archimedes]

https://sourceforge.net/p/chess-tools-for-android/blog/

CECSA 2022.09.26
https://sourceforge.net/projects/chess-tools-for-android/files/cecsa_2022.09.26.tar.gz/download

[Archimedes]

Resolving all dependencies when installing a compiler can sometimes be annoying. For this reason, I created a script which installs Android NDK (optional), Clang, GCC and all the necessarry libraries needed by the CECSA script in one go on Termux.

Could be very useful, when installing a new (fresh) installation of Termux. Complete installation takes only a view minutes now.

All you have to do is, start the script from within your home directory of your new installed Termux.

Code: Select all

bash cecsa.sh

As the community solution on
https://github.com/its-pointless/gcc_termux does not work for me, I used CCTools for the installation of the GCC compiler. The speed difference between 10 and 11 is only about 0.6 % on my device (tested with a distribution under Termux), so not much is lost here. If someone tells me how to get the community version to work, I'll be happy to include this variant in the script.

A short description is at the beginning of the installation script.

cecsa.sh:

Code: Select all

# This script installs Android NDK (optional),
# Clang, GCC and all the necessarry libraries
# needed by the CECSA script on Termux.
#
# It is recommended, to use this script only
# for a new installation of Termux.
#
# A full installation with all supported compilers
# including the CECSA script can be done in a
# few steps.
#
# 1. Install Termux from
#
#    https://f-droid.org/en/packages/com.termux/
#
# 2. Allow Termux to access your storage.
#
# 3. Download Android NDK from
#
#    https://github.com/Lzhiyong/termux-ndk
#
#    and download CECSA and this script from
#
#    https://sourceforge.net/projects/chess-tools-for-android/files/
#
#    All three files should be located in your
#    download directory of your mobile device
#    (/storage/emulated/0/Download).
#
# 4. Start Termux and copy this installation
#    script to your home directory.
#
#    cp /storage/emulated/0/Download/cecsa.sh ./
#
#    Start the script with
#
#    bash cecsa.sh
#
#    Answer all questions with "Y" and when
#    finished, restart Termux.
#
# If you only interested in the Clang and GCC
# compiler do not download Android NDK and CECSA
# to your download directory. The script take
# care of this and installs only Clang and GCC.
#
# The script also installs the filemanager mc
# and create a symlink in your home directory
# to your sdcard.

cecsa.sh:
https://sourceforge.net/projects/chess-tools-for-android/files/cecsa.sh/download

[hagtorp]

I'll be happy to include this variant in the script.

When using gcc termux and cecsa, add makefile a compilation flag:

Code: Select all

CXXFLAGS += -D__BIONIC_VERSIONER

CFLAGS += -D__BIONIC_VERSIONER

[Archimedes]

Many Thanks. But it wasn't just that. Only after running setup-patchforgcc I was able to compile Stockfish. But then I found that Stockfish wouldn't start outside of Termux. It only worked after I added -static-libstdc++ to the linker flags (as required by Clang).

After running the installation script (setup-pointless-repo.sh), GCC can be installed with "pkg install gcc-11". After that we have to execute the command "setupgcc-11". And after that we have to execute the command "setup-patchforgcc". Finally we have to add -D__BIONIC_VERSIONER to the CFlags and -static-libstdc++ to the LFlags.

So I will build the comminity solution into the script.