HCha/Stockfish
1
0
Fork 0
mirror of https://github.com/HChaZZY/Stockfish.git synced 2025-12-24 11:06:58 +08:00
Code Issues Packages Projects Releases Wiki Activity

Renaming and small changes.

Browse Source
This commit is contained in:
Tomasz Sobczyk
2021-04-19 19:05:37 +02:00
parent ba32bd5d70
commit c2511ffc7b
5 changed files with 56 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
README.md
View File

@@ -138,7 +138,7 @@ Currently, Stockfish has the following UCI options:
### Generating Training Data ### Generating Training Data
To generate training data from the classic eval, use the gensfen command with the setting "Use NNUE" set to "false". The given example is generation in its simplest form. There are more commands. To generate training data from the classic eval, use the generate_training_data command with the setting "Use NNUE" set to "false". The given example is generation in its simplest form. There are more commands.
``` ```
uci uci
@@ -148,7 +148,7 @@ setoption name Threads value X
setoption name Hash value Y setoption name Hash value Y
setoption name SyzygyPath value path setoption name SyzygyPath value path
isready isready
gensfen depth A count B keep_draws 1 eval_limit 32000 generate_training_data depth A count B keep_draws 1 eval_limit 32000
``` ```
- `A` is the searched depth per move, or how far the engine looks forward. This value is an integer. - `A` is the searched depth per move, or how far the engine looks forward. This value is an integer.
@@ -164,9 +164,9 @@ You will also need validation data that is used for loss calculation and accurac
Currently there are 3 training data formats. Two of them are supported directly. Currently there are 3 training data formats. Two of them are supported directly.
- `.bin` - the original training data format. Uses 40 bytes per entry. Is supported directly by the `gensfen` and `learn` commands. - `.bin` - the original training data format. Uses 40 bytes per entry. Is supported directly by the `generate_training_data` command.
- `.plain` - a human readable training data format. This one is not supported directly by the `gensfen` and `learn` commands. It should not be used for data exchange because it's less compact than other formats. It is mostly useful for inspection of the data. - `.plain` - a human readable training data format. This one is not supported directly by the `generate_training_data` command. It should not be used for data exchange because it's less compact than other formats. It is mostly useful for inspection of the data.
- `.binpack` - a compact binary training data format that exploits positions chains to further reduce size. It uses on average between 2 to 3 bytes per entry when generating data with `gensfen`. It is supported directly by `gensfen` and `learn` commands. It is currently the default for the `gensfen` command. A more in depth description can be found [here](docs/binpack.md) - `.binpack` - a compact binary training data format that exploits positions chains to further reduce size. It uses on average between 2 to 3 bytes per entry when generating data with `generate_training_data`. It is supported directly by `generate_training_data` command. It is currently the default for the `generate_training_data` command. A more in depth description can be found [here](docs/binpack.md)
### Conversion between formats. ### Conversion between formats.

36
docs/gensfen.md → docs/generate_training_data.md
View File

@@ -1,34 +1,36 @@
# Gensfen # generate_training_data
`gensfen` command allows generation of training data from self-play in a manner that suits training better than traditional games. It introduces random moves to diversify openings, and fixed depth evaluation. `generate_training_data` command allows generation of training data from self-play in a manner that suits training better than traditional games. It introduces random moves to diversify openings, and fixed depth evaluation.
As all commands in stockfish `gensfen` can be invoked either from command line (as `stockfish.exe gensfen ...`, but this is not recommended because it's not possible to specify UCI options before `gensfen` executes) or in the interactive prompt. As all commands in stockfish `generate_training_data` can be invoked either from command line (as `stockfish.exe generate_training_data ...`, but this is not recommended because it's not possible to specify UCI options before `generate_training_data` executes) or in the interactive prompt.
It is recommended to set the `PruneAtShallowDepth` UCI option to `false` as it will increase the quality of fixed depth searches. It is recommended to set the `PruneAtShallowDepth` UCI option to `false` as it will increase the quality of fixed depth searches.
It is recommended to keep the `EnableTranspositionTable` UCI option at the default `true` value as it will make the generation process faster without noticably harming the uniformity of the data. It is recommended to keep the `EnableTranspositionTable` UCI option at the default `true` value as it will make the generation process faster without noticably harming the uniformity of the data.
`gensfen` takes named parameters in the form of `gensfen param_1_name param_1_value param_2_name param_2_value ...`. `generate_training_data` takes named parameters in the form of `generate_training_data param_1_name param_1_value param_2_name param_2_value ...`.
Currently the following options are available: Currently the following options are available:
`set_recommended_uci_options` - this is a modifier not a parameter, no value follows it. If specified then some UCI options are set to recommended values. `set_recommended_uci_options` - this is a modifier not a parameter, no value follows it. If specified then some UCI options are set to recommended values.
`depth` - minimum depth of evaluation of each position. Default: 3. `depth` - sets minimum and maximum depth of evaluation of each position. Default: 3.
`depth2` - maximum depth of evaluation of each position. If not specified then the same as `depth`. `mindepth` - minimum depth of evaluation of each position. If not specified then the same as `depth`.
`maxdepth` - minimum depth of evaluation of each position. If not specified then the same as `depth`.
`nodes` - the number of nodes to use for evaluation of each position. This number is multiplied by the number of PVs of the current search. This does NOT override the `depth` and `depth2` options. If specified then whichever of depth or nodes limit is reached first applies. `nodes` - the number of nodes to use for evaluation of each position. This number is multiplied by the number of PVs of the current search. This does NOT override the `depth` and `depth2` options. If specified then whichever of depth or nodes limit is reached first applies.
`loop` - the number of training data entries to generate. 1 entry == 1 position. Default: 8000000000 (8B). `count` - the number of training data entries to generate. 1 entry == 1 position. Default: 8000000000 (8B).
`output_file_name` - the name of the file to output to. If the extension is not present or doesn't match the selected training data format the right extension will be appened. Default: generated_kifu `output_file_name` - the name of the file to output to. If the extension is not present or doesn't match the selected training data format the right extension will be appened. Default: generated_kifu
`eval_limit` - evaluations with higher absolute value than this will not be written and will terminate a self-play game. Should not exceed 10000 which is VALUE_KNOWN_WIN, but is only hardcapped at mate in 2 (\~30000). Default: 3000 `eval_limit` - evaluations with higher absolute value than this will not be written and will terminate a self-play game. Should not exceed 10000 which is VALUE_KNOWN_WIN, but is only hardcapped at mate in 2 (\~30000). Default: 3000
`random_move_minply` - the minimal ply at which a random move may be executed instead of a move chosen by search. Default: 1. `random_move_min_ply` - the minimal ply at which a random move may be executed instead of a move chosen by search. Default: 1.
`random_move_maxply` - the maximal ply at which a random move may be executed instead of a move chosen by search. Default: 24. `random_move_max_ply` - the maximal ply at which a random move may be executed instead of a move chosen by search. Default: 24.
`random_move_count` - maximum number of random moves in a single self-play game. Default: 5. `random_move_count` - maximum number of random moves in a single self-play game. Default: 5.
@@ -40,9 +42,9 @@ Currently the following options are available:
`random_multi_pv_depth` - the depth to use for multiPV search for random move. Default: `depth2`. `random_multi_pv_depth` - the depth to use for multiPV search for random move. Default: `depth2`.
`write_minply` - minimum ply for which the training data entry will be emitted. Default: 16. `write_min_ply` - minimum ply for which the training data entry will be emitted. Default: 16.
`write_maxply` - maximum ply for which the training data entry will be emitted. Default: 400. `write_max_ply` - maximum ply for which the training data entry will be emitted. Default: 400.
`book` - a path to an opening book to use for the starting positions. Currently only .epd format is supported. If not specified then the starting position is always the standard chess starting position. `book` - a path to an opening book to use for the starting positions. Currently only .epd format is supported. If not specified then the starting position is always the standard chess starting position.
@@ -50,17 +52,13 @@ Currently the following options are available:
`random_file_name` - if specified then the output filename will be chosen randomly. Overrides `output_file_name`. `random_file_name` - if specified then the output filename will be chosen randomly. Overrides `output_file_name`.
`write_out_draw_game_in_training_data_generation` - either 0 or 1. If 1 then training data from drawn games will be emitted too. Default: 1. `keep_draws` - either 0 or 1. If 1 then training data from drawn games will be emitted too. Default: 1.
`use_draw_in_training_data_generation` - deprecated, alias for `write_out_draw_game_in_training_data_generation` `adjudicate_draws_by_score` - either 0 or 1. If 1 then drawn games will be adjudicated when the score remains 0 for at least 8 plies after ply 80. Default: 1.
`detect_draw_by_consecutive_low_score` - either 0 or 1. If 1 then drawn games will be adjudicated when the score remains 0 for at least 8 plies after ply 80. Default: 1. `adjudicate_draws_by_insufficient_mating_material` - either 0 or 1. If 1 then position with insufficient material will be adjudicated as draws. Default: 1.
`use_game_draw_adjudication` - deprecated, alias for `detect_draw_by_consecutive_low_score` `data_format` - format of the training data to use. Either `bin` or `binpack`. Default: `binpack`.
`detect_draw_by_insufficient_mating_material` - either 0 or 1. If 1 then position with insufficient material will be adjudicated as draws. Default: 1.
`sfen_format` - format of the training data to use. Either `bin` or `binpack`. Default: `binpack`.
`ensure_quiet` - this is a flag option. When specified the positions will be from the qsearch leaf. `ensure_quiet` - this is a flag option. When specified the positions will be from the qsearch leaf.

10
docs/gensfen_nonpv.md → docs/generate_training_data_nonpv.md
View File

@@ -1,14 +1,14 @@
# Gensfen NonPV # generate_training_data_nonpv
`gensfen_nonpv` command allows generation of training data from self-play in a manner that suits training better than traditional games. It plays fixed nodes self play games for exploration and records [some of] the evaluated positions. Then rescores them with fixed depth search. `generate_training_data_nonpv` command allows generation of training data from self-play in a manner that suits training better than traditional games. It plays fixed nodes self play games for exploration and records [some of] the evaluated positions. Then rescores them with fixed depth search.
As all commands in stockfish `gensfen_nonpv` can be invoked either from command line (as `stockfish.exe gensfen_nonpv ...`, but this is not recommended because it's not possible to specify UCI options before `gensfen_nonpv` executes) or in the interactive prompt. As all commands in stockfish `generate_training_data_nonpv` can be invoked either from command line (as `stockfish.exe generate_training_data_nonpv ...`, but this is not recommended because it's not possible to specify UCI options before `generate_training_data_nonpv` executes) or in the interactive prompt.
It is recommended to set the `PruneAtShallowDepth` UCI option to `false` as it will increase the quality of fixed depth searches. It is recommended to set the `PruneAtShallowDepth` UCI option to `false` as it will increase the quality of fixed depth searches.
It is recommended to keep the `EnableTranspositionTable` UCI option at the default `true` value as it will make the generation process faster without noticably harming the uniformity of the data. It is recommended to keep the `EnableTranspositionTable` UCI option at the default `true` value as it will make the generation process faster without noticably harming the uniformity of the data.
`gensfen_nonpv` takes named parameters in the form of `gensfen_nonpv param_1_name param_1_value param_2_name param_2_value ...`. `generate_training_data_nonpv` takes named parameters in the form of `generate_training_data_nonpv param_1_name param_1_value param_2_name param_2_value ...`.
Currently the following options are available: Currently the following options are available:
@@ -36,6 +36,6 @@ Currently the following options are available:
`book` - a path to an opening book to use for the starting positions. Currently only .epd format is supported. If not specified then the starting position is always the standard chess starting position. `book` - a path to an opening book to use for the starting positions. Currently only .epd format is supported. If not specified then the starting position is always the standard chess starting position.
`sfen_format` - format of the training data to use. Either `bin` or `binpack`. Default: `binpack`. `data_format` - format of the training data to use. Either `bin` or `binpack`. Default: `binpack`.
`seed` - seed for the PRNG. Can be either a number or a string. If it's a string then its hash will be used. If not specified then the current time will be used. `seed` - seed for the PRNG. Can be either a number or a string. If it's a string then its hash will be used. If not specified then the current time will be used.

45
src/tools/training_data_generator.cpp
View File

@@ -83,7 +83,7 @@ namespace Stockfish::Tools
uint64_t save_every = std::numeric_limits<uint64_t>::max(); uint64_t save_every = std::numeric_limits<uint64_t>::max();
std::string output_file_name = "generated_kifu"; std::string output_file_name = "training_data";
SfenOutputType sfen_format = SfenOutputType::Binpack; SfenOutputType sfen_format = SfenOutputType::Binpack;
@@ -831,20 +831,25 @@ namespace Stockfish::Tools
break; break;
if (token == "depth") if (token == "depth")
{
is >> params.search_depth_min;
params.search_depth_max = params.search_depth_min;
}
else if (token == "min_depth")
is >> params.search_depth_min;
else if (token == "max_depth")
is >> params.search_depth_min; is >> params.search_depth_min;
else if (token == "depth2")
is >> params.search_depth_max;
else if (token == "nodes") else if (token == "nodes")
is >> params.nodes; is >> params.nodes;
else if (token == "loop") else if (token == "count")
is >> loop_max; is >> loop_max;
else if (token == "output_file_name") else if (token == "output_file_name")
is >> params.output_file_name; is >> params.output_file_name;
else if (token == "eval_limit") else if (token == "eval_limit")
is >> params.eval_limit; is >> params.eval_limit;
else if (token == "random_move_minply") else if (token == "random_move_min_ply")
is >> params.random_move_minply; is >> params.random_move_minply;
else if (token == "random_move_maxply") else if (token == "random_move_max_ply")
is >> params.random_move_maxply; is >> params.random_move_maxply;
else if (token == "random_move_count") else if (token == "random_move_count")
is >> params.random_move_count; is >> params.random_move_count;
@@ -856,9 +861,9 @@ namespace Stockfish::Tools
is >> params.random_multi_pv_diff; is >> params.random_multi_pv_diff;
else if (token == "random_multi_pv_depth") else if (token == "random_multi_pv_depth")
is >> params.random_multi_pv_depth; is >> params.random_multi_pv_depth;
else if (token == "write_minply") else if (token == "write_min_ply")
is >> params.write_minply; is >> params.write_minply;
else if (token == "write_maxply") else if (token == "write_max_ply")
is >> params.write_maxply; is >> params.write_maxply;
else if (token == "save_every") else if (token == "save_every")
is >> params.save_every; is >> params.save_every;
@@ -866,15 +871,13 @@ namespace Stockfish::Tools
is >> params.book; is >> params.book;
else if (token == "random_file_name") else if (token == "random_file_name")
is >> random_file_name; is >> random_file_name;
// Accept also the old option name. else if (token == "keep_draws")
else if (token == "use_draw_in_training_data_generation" || token == "write_out_draw_game_in_training_data_generation")
is >> params.write_out_draw_game_in_training_data_generation; is >> params.write_out_draw_game_in_training_data_generation;
// Accept also the old option name. else if (token == "adjudicate_draws_by_score")
else if (token == "use_game_draw_adjudication" || token == "detect_draw_by_consecutive_low_score")
is >> params.detect_draw_by_consecutive_low_score; is >> params.detect_draw_by_consecutive_low_score;
else if (token == "detect_draw_by_insufficient_mating_material") else if (token == "adjudicate_draws_by_insufficient_material")
is >> params.detect_draw_by_insufficient_mating_material; is >> params.detect_draw_by_insufficient_mating_material;
else if (token == "sfen_format") else if (token == "data_format")
is >> sfen_format; is >> sfen_format;
else if (token == "seed") else if (token == "seed")
is >> params.seed; is >> params.seed;
@@ -934,25 +937,25 @@ namespace Stockfish::Tools
params.enforce_constraints(); params.enforce_constraints();
std::cout << "INFO: Executing gensfen command\n"; std::cout << "INFO: Executing generate_training_data command\n";
std::cout << "INFO: Parameters:\n"; std::cout << "INFO: Parameters:\n";
std::cout std::cout
<< " - search_depth_min = " << params.search_depth_min << endl << " - search_depth_min = " << params.search_depth_min << endl
<< " - search_depth_max = " << params.search_depth_max << endl << " - search_depth_max = " << params.search_depth_max << endl
<< " - nodes = " << params.nodes << endl << " - nodes = " << params.nodes << endl
<< " - num sfens to generate = " << loop_max << endl << " - count = " << loop_max << endl
<< " - eval_limit = " << params.eval_limit << endl << " - eval_limit = " << params.eval_limit << endl
<< " - num threads (UCI) = " << params.num_threads << endl << " - num threads (UCI) = " << params.num_threads << endl
<< " - random_move_minply = " << params.random_move_minply << endl << " - random_move_min_ply = " << params.random_move_minply << endl
<< " - random_move_maxply = " << params.random_move_maxply << endl << " - random_move_max_ply = " << params.random_move_maxply << endl
<< " - random_move_count = " << params.random_move_count << endl << " - random_move_count = " << params.random_move_count << endl
<< " - random_move_like_apery = " << params.random_move_like_apery << endl << " - random_move_like_apery = " << params.random_move_like_apery << endl
<< " - random_multi_pv = " << params.random_multi_pv << endl << " - random_multi_pv = " << params.random_multi_pv << endl
<< " - random_multi_pv_diff = " << params.random_multi_pv_diff << endl << " - random_multi_pv_diff = " << params.random_multi_pv_diff << endl
<< " - random_multi_pv_depth = " << params.random_multi_pv_depth << endl << " - random_multi_pv_depth = " << params.random_multi_pv_depth << endl
<< " - write_minply = " << params.write_minply << endl << " - write_min_ply = " << params.write_minply << endl
<< " - write_maxply = " << params.write_maxply << endl << " - write_max_ply = " << params.write_maxply << endl
<< " - book = " << params.book << endl << " - book = " << params.book << endl
<< " - output_file_name = " << params.output_file_name << endl << " - output_file_name = " << params.output_file_name << endl
<< " - save_every = " << params.save_every << endl << " - save_every = " << params.save_every << endl
@@ -969,6 +972,6 @@ namespace Stockfish::Tools
TrainingDataGenerator gensfen(params); TrainingDataGenerator gensfen(params);
gensfen.generate(loop_max); gensfen.generate(loop_max);
std::cout << "INFO: TrainingDataGenerator finished." << endl; std::cout << "INFO: generate_training_data finished." << endl;
} }
} }

10
src/tools/training_data_generator_nonpv.cpp
View File

@@ -61,7 +61,7 @@ namespace Stockfish::Tools
int exploration_min_pieces = 8; int exploration_min_pieces = 8;
std::string output_file_name = "generated_gensfen_nonpv"; std::string output_file_name = "training_data_nonpv";
SfenOutputType sfen_format = SfenOutputType::Binpack; SfenOutputType sfen_format = SfenOutputType::Binpack;
@@ -426,7 +426,7 @@ namespace Stockfish::Tools
is >> params.exploration_save_rate; is >> params.exploration_save_rate;
else if (token == "book") else if (token == "book")
is >> params.book; is >> params.book;
else if (token == "sfen_format") else if (token == "data_format")
is >> sfen_format; is >> sfen_format;
else if (token == "seed") else if (token == "seed")
is >> params.seed; is >> params.seed;
@@ -458,7 +458,7 @@ namespace Stockfish::Tools
params.enforce_constraints(); params.enforce_constraints();
std::cout << "INFO: Executing gensfen_nonpv command\n"; std::cout << "INFO: Executing generate_training_data_nonpv command\n";
std::cout << "INFO: Parameters:\n"; std::cout << "INFO: Parameters:\n";
std::cout std::cout
@@ -471,7 +471,7 @@ namespace Stockfish::Tools
<< " - exploration_min_pieces = " << params.exploration_min_pieces << endl << " - exploration_min_pieces = " << params.exploration_min_pieces << endl
<< " - exploration_save_rate = " << params.exploration_save_rate << endl << " - exploration_save_rate = " << params.exploration_save_rate << endl
<< " - book = " << params.book << endl << " - book = " << params.book << endl
<< " - sfen_format = " << sfen_format << endl << " - data_format = " << sfen_format << endl
<< " - seed = " << params.seed << endl << " - seed = " << params.seed << endl
<< " - count = " << count << endl; << " - count = " << count << endl;
@@ -483,6 +483,6 @@ namespace Stockfish::Tools
TrainingDataGeneratorNonPv gensfen(params); TrainingDataGeneratorNonPv gensfen(params);
gensfen.generate(count); gensfen.generate(count);
std::cout << "INFO: gensfen_nonpv finished." << endl; std::cout << "INFO: generate_training_data_nonpv finished." << endl;
} }
} }