Commit Graph

307 Commits

Author SHA1 Message Date
Tomasz Sobczyk
e4a4f4001f parametrize the name of the training data file generated during pgo 2020-09-13 13:44:19 +02:00
Tomasz Sobczyk
9ee8ce67bf Move removal of generate training data file to profileclean. 2020-09-13 13:42:13 +02:00
Tomasz Sobczyk
4b70f4bf23 Add extra ld flags to the proper variable. 2020-09-13 02:07:29 +02:00
Tomasz Sobczyk
72164ba59c Add missing -fopenmp LDFLAG 2020-09-13 02:06:33 +02:00
Tomasz Sobczyk
fbae6604b1 Remove LEARNCXXFLAGS, LEARNLDFLAGS, BLASDEFINE, BLASCXXFLAGS, BLASLDFLAGS in favor of directly modifying CXXFLAGS and LDFLAGS. 2020-09-13 00:18:01 +02:00
Tomasz Sobczyk
8d1ad6fbf6 Add a makefile option to enable use of BLAS. Default to "no" 2020-09-12 21:16:27 +02:00
Tomasz Sobczyk
1e2fca4040 Move learn target to build target and profile-learn to profile-build. 2020-09-12 16:23:49 +02:00
Joost VandeVondele
8d499e6efa Fix flags for dependency generation (98f24570ab) 2020-09-12 22:25:36 +09:00
Tomasz Sobczyk
98f24570ab Add src to include paths, remove non-standard ".." in includes in learn directory. 2020-09-11 21:53:17 +09:00
Tomasz Sobczyk
a059fa86c4 Move sfen_packer to learn. 2020-09-11 21:53:17 +09:00
Tomasz Sobczyk
683c6146ce Move declarations around and split them. 2020-09-11 21:53:17 +09:00
nodchip
1d00d00241 Removed ENABLE_TEST_CMD macro. 2020-09-09 10:26:42 +09:00
nodchip
a6013557f2 Removed EVAL_NNUE macro. 2020-09-09 10:26:42 +09:00
Tomasz Sobczyk
1482e5215a A second batch of code reorganization. 2020-09-08 20:07:30 +09:00
Joost VandeVondele
6e8f82ad76 Fix small CI failures
1) Only access UCI option if defined
2) disable -Werror for now.
3) disable a few target that don't have _mm_malloc.
4) Add profile-learn target, with small speedup.
5) just test on Linux + gcc (skip macOS, unclear openblas, skip linux+clang, unclear omp/std::filesystem).
2020-09-08 09:14:49 +09:00
Tomasz Sobczyk
e638d66bbe Only add -s flag to the linker if debug=no 2020-09-08 09:10:58 +09:00
Joost VandeVondele
e9e52faae7 Typo fix 2020-09-07 15:21:50 +09:00
Joost VandeVondele
3a06de298b Define BLAS variables in Makefile
makes it a little easier to change the BLAS library used,
doesn't hardcode the mingw headers. Works on Linux with openblas installed.
Should be no change on Windows.
2020-09-07 09:19:31 +09:00
Tomasz Sobczyk
c17f2b15fd General cleanup of learner.cpp. 2020-09-02 23:08:22 +09:00
nodchip
7d6668515c Added -static link option to the learn and profile-learn targets. 2020-08-30 14:54:07 +09:00
nodchip
9f2f31632c Fixed build errors. 2020-08-29 08:17:24 +09:00
nodchip
906c18eb46 Merge branch 'master' of github.com:official-stockfish/Stockfish into nnue-player-merge-2020-08-28
# Conflicts:
#	README.md
#	src/Makefile
#	src/search.cpp
#	src/types.h
#	src/uci.cpp
#	src/ucioption.cpp
2020-08-28 11:26:11 +09:00
Joost VandeVondele
b0b4ca17db Check ARCH=.... variable
to prevent user errors or generating untested code,
check explicitly that the ARCH variable is equivalent to a supported architecture
as listed in `make help`.

To nevertheless compile for an untested target the user can override the internal
variable, passing the undocumented `SUPPORTED_ARCH=true` to make.

closes https://github.com/official-stockfish/Stockfish/pull/3062

No functional change.
2020-08-26 07:07:06 +02:00
mstembera
530fccbf27 Allow for VNNI256 compilation with g++-8
explicitly pass needed -mavx512f -mavx512bw flags

closes https://github.com/official-stockfish/Stockfish/pull/3061

No functional change
2020-08-24 12:43:39 +02:00
mstembera
701b2427bd Support VNNI on 256bit vectors
due to downclocking on current chips (tested up to cascade lake)
supporting avx512 and vnni512, it is better to use avx2 or vnni256
in multithreaded (in particular hyperthreaded) engine use.
In single threaded use, the picture is different.

gcc compilation for vnni256 requires a toolchain for gcc >= 9.

closes https://github.com/official-stockfish/Stockfish/pull/3038

No functional change
2020-08-24 12:03:04 +02:00
George Sobala
e453f09f06 armv8 AArch64 does not require -mfpu=neon
-mpfu is not required on AArch64 / armv8 architecture on Linux and throws an error if present.
This PR has been tested on gcc and clang on Gentoo-64 and Raspian-64 on a Raspberry Pi 4,
as well as with a cross from Ubuntu
(`make clean && make -j build ARCH=armv8         COMP=gcc COMPILER=aarch64-linux-gnu-g++`)

fixes https://github.com/official-stockfish/Stockfish/issues/3056
closes https://github.com/official-stockfish/Stockfish/pull/3059

No functional change
2020-08-24 11:30:55 +02:00
Joost VandeVondele
5f1843c9cb Small trivial cleanups
closes https://github.com/official-stockfish/Stockfish/pull/2801

No functional change
2020-08-23 01:53:41 +02:00
Stéphane Nicolet
3542033342 Instructions to build on older Macintosh
In recent Macs, it is possible to use the Clang compiler provided by Apple
to compile Stockfish out of the box, and this is the method used by default
in our Makefile (the Makefile sets the macosx-version-min=10.14 flag to select
the right libc++ library for the Clang compiler with recent c++17 support).

But it is quite possible to compile and run Stockfish on older Macs! Below
we describe a method to install a recent GNU compiler on these Macs, to get
the c++17 support. We have tested the following procedure to install gcc10 on
machines running Mac OS 10.7, Mac OS 10.9 and Mac OS 10.13:

1) install XCode for your machine.

2) install Apple command-line developer tools for XCode, by typing the following
   command in a Terminal:

```
      sudo xcode-select --install
```

3) go to the Stockfish "src" directory, then try a default build and run Stockfish:

```
      make clean
      make build
      make net
      ./stockfish
```

4) if step 3 worked, congrats! You have a compiler recent enough on your Mac
to compile Stockfish. If not, continue with step 5 to install GNU gcc10 :-)

5) install the MacPorts package manager (https://www.macports.org/install.php),
for instance using the fast method in the "macOS Package (.pkg) Installer"
section of the page.

6) use the "port" command to install the gcc10 package of MacPorts by typing the
following command:

```
    sudo port install gcc10
```

With this step, MacPorts will install the gcc10 compiler under the name "g++-mp-10"
in the /opt/local/bin directory:

```
   which g++-mp-10

   /opt/local/bin/g++-mp-10       <--- answer
```

7) You can now go back to the "src" directory of Stockfish, and try to build
Stockfish by pointing at the right compiler:

```
   make clean
   make build COMP=gcc COMPCXX=/opt/local/bin/g++-mp-10
   make net
   ./stockfish
```

8) Enjoy Stockfish on Macintosh!

See this pull request for further discussion:
https://github.com/official-stockfish/Stockfish/pull/3049

No functional change
2020-08-22 22:37:50 +02:00
Joost VandeVondele
34f67c5722 Explicitly rely on pthreads if possible
allows us to set the needed stacksize on thread creation.

Useful for environments with too small a default stack size (e.g. Alpine Linux with musl).

Passed STC, no regression:

LLR: 2.96 (-2.94,2.94) {-1.25,0.25}
Total: 17816 W: 1344 L: 1275 D: 15197
Ptnml(0-2): 30, 1057, 6682, 1092, 47
https://tests.stockfishchess.org/tests/view/5f402b5587a5c3c63d8f534d

closes https://github.com/official-stockfish/Stockfish/pull/3047

fixes https://github.com/official-stockfish/Stockfish/issues/3041

No functional change.
2020-08-22 11:00:45 +02:00
gsobala
15abcaedc1 Update Makefile for macOS
Changes to deal with compilation (particularly profile-build) on macOS.
(1) The default toolchain has gcc masquerading as clang,
    the previous Makefile was not picking up the required changes
    to the different profiling tools.
(2) The previous Makefile test for gccisclang occurred before
    a potential overwrite of CXX by COMPCXX
(3) llvm-profdata no longer runs as a command on macOS and
    instead is invoked by ``xcrun llvm-profdata``
(4) Needs to support use of true gcc using e.g.
    ``make build ... COMPCXX=g++-10``
(5) enable profile-build in travis for macOS

closes https://github.com/official-stockfish/Stockfish/pull/3043

No functional change
2020-08-21 17:59:21 +02:00
syzygy1
42e8789f0b Expanded support for x86-32 architectures.
add new ARCH targets

x86-32-sse41-popcnt     > x86 32-bit with sse41 and popcnt support
x86-32-sse2             > x86 32-bit with sse2 support
x86-32                  > x86 32-bit generic (with mmx and sse support)

retire x86-32-old (use general-32)

closes https://github.com/official-stockfish/Stockfish/pull/3022

No functional change.
2020-08-18 20:15:07 +02:00
Joost VandeVondele
384d684484 Better error message on missing curl/wget
provide clean error/warning message for missing curl/wget, sha256sum/shasum

fixes https://github.com/official-stockfish/Stockfish/issues/3025

closes https://github.com/official-stockfish/Stockfish/pull/3026

No functional change
2020-08-18 19:57:30 +02:00
Joost VandeVondele
fbae5614eb Fix Makefile typo
remove stray quote, shown with `make help`

No functional change
2020-08-18 08:49:06 +02:00
notruck
65b976439f Support building for Android using NDK
The easiest way to use the NDK in conjunction with this Makefile (tested on linux-x86_64):

1. Download the latest NDK (r21d) from Google from https://developer.android.com/ndk/downloads
2. Place and unzip the NDK in $HOME/ndk folder
3. Export the path variable e.g., `export PATH=$PATH:$HOME/ndk/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/bin`
4. cd to your Stockfish/src dir
5. Issue `make -j ARCH=armv8 COMP=ndk build`  (use `ARCH=armv7` or `ARCH=armv7-neon` for older CPUs)
6. Optionally `make -j ARCH=armv8 COMP=ndk strip`
7. That's all. Enjoy!

Improves support from Raspberry Pi (incomplete?) and compiling on arm in general

closes https://github.com/official-stockfish/Stockfish/pull/3015

fixes https://github.com/official-stockfish/Stockfish/issues/2860

fixes https://github.com/official-stockfish/Stockfish/issues/2641

Support is still fragile as we're missing CI on these targets. Nevertheless tested with:

```bash
  # build crosses from ubuntu 20.04 on x86 to various arch/OS combos
  # tested with suitable packages installed
  # (build-essentials, mingw-w64, g++-arm-linux-gnueabihf, NDK (r21d) from google)

  # cross to Android
  export PATH=$HOME/ndk/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/bin:$PATH
  make clean && make -j build ARCH=armv7         COMP=ndk  && make -j build ARCH=armv7 COMP=ndk strip
  make clean && make -j build ARCH=armv7-neon    COMP=ndk  && make -j build ARCH=armv7-neon COMP=ndk strip
  make clean && make -j build ARCH=armv8         COMP=ndk  && make -j build ARCH=armv8 COMP=ndk strip

  # cross to Raspberry Pi
  make clean && make -j build ARCH=armv7         COMP=gcc COMPILER=arm-linux-gnueabihf-g++
  make clean && make -j build ARCH=armv7-neon    COMP=gcc COMPILER=arm-linux-gnueabihf-g++

  # cross to Windows
  make clean && make -j build ARCH=x86-64-modern COMP=mingw
```

No functional change
2020-08-18 08:19:22 +02:00
Serianol
2fb3f76399 Update Makefile 2020-08-17 20:50:45 +09:00
Joost VandeVondele
ce009ea1aa Verify SHA of downloaded net file
check SHA of the available and downloaded file.

Document the format requirement on the default net.

Also allow curl to make possibly insecure connections, as needed for old curl.

fixes https://github.com/official-stockfish/Stockfish/issues/2998

closes https://github.com/official-stockfish/Stockfish/pull/3000

No functional change.
2020-08-14 16:20:42 +02:00
Stéphane Nicolet
e8ea215a13 Clean-up Makefile help
Do not show the details of the default architecture for a simple "make help"
invocation, as the details are most likely to confuse beginners. Instead we
make it clear which architecture is the default and put an example at the end
of the Makefile as an incentative to use "make help ARCH=blah" to discover
the flags used by the different architectures.

```
    make help
    make help ARCH=x86-64-ssse3
```

Also clean-up and modernize a bit the Makefile examples while at it.

closes https://github.com/official-stockfish/Stockfish/pull/2996

No functional change
2020-08-14 16:18:50 +02:00
Joost VandeVondele
69cfe28f31 Output the SSE2 flag in compiler_info
was missing in the list of outputs, slightly reorder flags.
explicitly add -msse2 if USE_SSE2 (is implicit already, -msse -m64).

closes https://github.com/official-stockfish/Stockfish/pull/2990

No functional change.
2020-08-13 07:41:06 +02:00
mstembera
dd63b98fb0 Add support for VNNI
Adds support for Vector Neural Network Instructions (avx512), as available on Intel Cascade Lake

The _mm512_dpbusd_epi32() intrinsic (vpdpbusd instruction) is taylor made for NNUE.

on a cascade lake CPU (AWS C5.24x.large, gcc 10) NNUE eval is at roughly 78% nps of classical
(single core test)

bench 1024 1 24 default depth:
target 	classical 	NNUE 	ratio
vnni 	2207232 	1725987 	78.20
avx512 	2216789 	1671734 	75.41
avx2 	2194006 	1611263 	73.44
modern 	2185001 	1352469 	61.90

closes https://github.com/official-stockfish/Stockfish/pull/2987

No functional change
2020-08-13 07:39:52 +02:00
Joost VandeVondele
992f549ae7 Restrict avx2 hack to windows target
this workaround is possibly rather a windows & gcc specific problem. See e.g.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412#c25

on Linux with gcc 8 this patch brings roughly a 8% speedup.
However, probably needs some testing in the wild.

includes a workaround for an old msys make (3.81) installation (fixes #2984)

No functional change
2020-08-11 23:35:02 +02:00
nodchip
75b9d6f6b1 Fixed build parameters. 2020-08-11 16:37:47 +09:00
Guy Vreuls
ea6220f381 This commit enables a mixed bench, to improve CI and allow for PGO (profile-build) of the NNUE part of the code.
Joint work gvreuls / vondele

* Download the default NNUE net in AppVeyor
* Download net in travis CI `make net`
* Adjust tests to cover more archs, speedup instrumented testing
* Introduce 'mixed' bench as default, with further options:

classical, NNUE, mixed.

mixed (default) and NNUE require the default net to be present,
which can be obtained with

```
make net
```

Further examples (first is equivalent to `./stockfish bench`):

```
./stockfish bench 16 1 13 default depth mixed
./stockfish bench 16 1 13 default depth classical
./stockfish bench 16 1 13 default depth NNUE
```

The net is now downloaded automatically if needed for `profile-build`
(usual `build` works fine without net present)

PGO gives a nice speedup on fishtest:

passed STC:
LLR: 2.93 (-2.94,2.94) {-0.50,1.50}
Total: 3360 W: 469 L: 343 D: 2548
Ptnml(0-2): 20, 246, 1030, 356, 28
https://tests.stockfishchess.org/tests/view/5f31b5499081672066537569

passed LTC:
LLR: 2.97 (-2.94,2.94) {0.25,1.75}
Total: 8824 W: 609 L: 502 D: 7713
Ptnml(0-2): 8, 430, 3438, 519, 17
https://tests.stockfishchess.org/tests/view/5f31c87b908167206653757c

closes https://github.com/official-stockfish/Stockfish/pull/2931

fixes https://github.com/official-stockfish/Stockfish/issues/2907

requires fishtest updates before commit

Bench: 4290577
2020-08-11 08:17:03 +02:00
mstembera
f46c73040c Fix AVX512 build with older compilers
avoids an intrinsic that is missing in gcc < 10.

For this target, might trigger another gcc bug on windows that
requires up-to-date gcc 8, 9, or 10, or usage of clang.

Fixes https://github.com/official-stockfish/Stockfish/issues/2975

closes https://github.com/official-stockfish/Stockfish/pull/2976

No functional change
2020-08-11 08:17:03 +02:00
Guy Vreuls
4ab8b0b738 Fix parallel LTO issues on Windows
This adds -save-temps to the linker flags when parallel LTO is used on
MinGW/MSYS.

fixes #2977

closes https://github.com/official-stockfish/Stockfish/pull/2978

No functional change.
2020-08-11 08:17:03 +02:00
Fanael Linithien
21df37d7fd Provide vectorized NNUE code for SSE2 and MMX targets
This patch allows old x86 CPUs, from AMD K8 (which the x86-64 baseline
targets) all the way down to the Pentium MMX, to benefit from NNUE with
comparable performance hit versus hand-written eval as on more modern
processors.

NPS of the bench with NNUE enabled on a Pentium III 1.13 GHz (using the
MMX code):
  master: 38951
  this patch: 80586

NPS of the bench with NNUE enabled using baseline x86-64 arch, which is
how linux distros are likely to package stockfish, on a modern CPU
(using the SSE2 code):
  master: 882584
  this patch: 1203945

closes https://github.com/official-stockfish/Stockfish/pull/2956

No functional change.
2020-08-10 19:17:57 +02:00
sf-x
cb0504028e Makefile rework/cleanup
Makefile targets x86-64-sse42, x86-sse3 are removed; x86-64-sse41
is renamed to x86-64-sse41-popcnt (it did enable popcnt).

Makefile variables sse3, sse42, their associated compilation flags
and code in misc.cpp are removed.

closes https://github.com/official-stockfish/Stockfish/pull/2922

No functional change
2020-08-10 14:32:11 +02:00
nodchip
4260ed0c7f Merge branch 'master' of github.com:official-stockfish/Stockfish into nnue-player-merge 2020-08-10 08:52:55 +09:00
nodchip
4f97d3446d Cleaned up source code. 2020-08-10 08:52:34 +09:00
Joost VandeVondele
27b593a944 Fix a data race for NNUE
the stateInfo at the rootPos is no longer read-only, as the NNUE accumulator is part of it.
Threads can thus not share this object and need their own copy.

tested for no regression
https://tests.stockfishchess.org/tests/view/5f3022239081672066536bce
LLR: 2.96 (-2.94,2.94) {-1.50,0.50}
Total: 52800 W: 6843 L: 6802 D: 39155
Ptnml(0-2): 336, 4646, 16399, 4679, 340

closes https://github.com/official-stockfish/Stockfish/pull/2957

fixes https://github.com/official-stockfish/Stockfish/issues/2933

No functional change
2020-08-09 23:51:07 +02:00
Daniel Dugovic
d7a26899a9 Use fallback implementation for C++ aligned_alloc
fixes https://github.com/official-stockfish/Stockfish/issues/2921

closes https://github.com/official-stockfish/Stockfish/pull/2927

No functional change
2020-08-09 17:07:45 +02:00