HCha/Stockfish
1
0
Fork 0
mirror of https://github.com/HChaZZY/Stockfish.git synced 2025-12-22 10:06:26 +08:00
Code Issues Packages Projects Releases Wiki Activity

This commit enables a mixed bench, to improve CI and allow for PGO (profile-build) of the NNUE part of the code.

Browse Source 
Joint work gvreuls / vondele

* Download the default NNUE net in AppVeyor
* Download net in travis CI `make net`
* Adjust tests to cover more archs, speedup instrumented testing
* Introduce 'mixed' bench as default, with further options:

classical, NNUE, mixed.

mixed (default) and NNUE require the default net to be present,
which can be obtained with

```
make net
```

Further examples (first is equivalent to `./stockfish bench`):

```
./stockfish bench 16 1 13 default depth mixed
./stockfish bench 16 1 13 default depth classical
./stockfish bench 16 1 13 default depth NNUE
```

The net is now downloaded automatically if needed for `profile-build`
(usual `build` works fine without net present)

PGO gives a nice speedup on fishtest:

passed STC:
LLR: 2.93 (-2.94,2.94) {-0.50,1.50}
Total: 3360 W: 469 L: 343 D: 2548
Ptnml(0-2): 20, 246, 1030, 356, 28
https://tests.stockfishchess.org/tests/view/5f31b5499081672066537569

passed LTC:
LLR: 2.97 (-2.94,2.94) {0.25,1.75}
Total: 8824 W: 609 L: 502 D: 7713
Ptnml(0-2): 8, 430, 3438, 519, 17
https://tests.stockfishchess.org/tests/view/5f31c87b908167206653757c

closes https://github.com/official-stockfish/Stockfish/pull/2931

fixes https://github.com/official-stockfish/Stockfish/issues/2907

requires fishtest updates before commit

Bench: 4290577
This commit is contained in:
Guy Vreuls
2020-08-07 17:07:46 +02:00
committed by Joost VandeVondele
parent f46c73040c
commit ea6220f381
5 changed files with 51 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
src/benchmark.cpp
View File

@@ -95,8 +95,9 @@ const vector<string> Defaults = {
/// setup_bench() builds a list of UCI commands to be run by bench. There
/// are five parameters: TT size in MB, number of search threads that
/// should be used, the limit value spent for each position, a file name
/// where to look for positions in FEN format and the type of the limit:
/// depth, perft, nodes and movetime (in millisecs).
/// where to look for positions in FEN format, the type of the limit:
/// depth, perft, nodes and movetime (in millisecs), and evaluation type
/// mixed (default), classical, NNUE.
///
/// bench -> search default positions up to depth 13
/// bench 64 1 15 -> search default positions up to depth 15 (TT = 64MB)
@@ -115,6 +116,7 @@ vector<string> setup_bench(const Position& current, istream& is) {
string limit = (is >> token) ? token : "13";
string fenFile = (is >> token) ? token : "default";
string limitType = (is >> token) ? token : "depth";
string evalType = (is >> token) ? token : "mixed";
go = limitType == "eval" ? "eval" : "go " + limitType + " " + limit;
@@ -146,13 +148,20 @@ vector<string> setup_bench(const Position& current, istream& is) {
list.emplace_back("setoption name Hash value " + ttSize);
list.emplace_back("ucinewgame");
size_t posCounter = 0;
for (const string& fen : fens)
if (fen.find("setoption") != string::npos)
list.emplace_back(fen);
else
{
if (evalType == "classical" || (evalType == "mixed" && posCounter % 2 == 0))
list.emplace_back("setoption name Use NNUE value false");
else if (evalType == "NNUE" || (evalType == "mixed" && posCounter % 2 != 0))
list.emplace_back("setoption name Use NNUE value true");
list.emplace_back("position fen " + fen);
list.emplace_back(go);
++posCounter;
}
return list;