diff --git a/README.md b/README.md index 7a237480..8cacd1b3 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ Stockfish NNUE is a port of a shogi neural network named NNUE (efficiently updat ## Training Guide ### Generating Training Data -To generate training data from the classic eval, use gensfen command with setting "Use NNUE" to "false". The given example is generation in its simplest form. There are more commands. +To generate training data from the classic eval, use the gensfen command with the setting "Use NNUE" set to "false". The given example is generation in its simplest form. There are more commands. ``` uci setoption name Use NNUE value false @@ -44,16 +44,16 @@ Nets get saved in the "evalsave" folder. - lambda is the amount of weight it puts to eval of learning data vs win/draw/loss results. 1 puts all weight on eval, lambda 0 puts all weight on WDL results. ### Reinforcement Learning -If you would like to do some reinforcement learning on your original network, you must first generate training data using the learn binaries with setting `Use NNUE` to true. Make sure that your previously trained network is in the eval folder. Use the commands specified above. Make sure `SkipLoadingEval` is set to false so that the data generated is using the neural net's eval by typing the command `uci setoption name SkipLoadingEval value false` before typing the `isready` command. You should aim to generate less positions than the first run, around 1/10 of the number of positions generated in the first run. The depth should be higher as well. You should also do the same for validation data, with the depth being higher than the last run. +If you would like to do some reinforcement learning on your original network, you must first generate training data using the learn binaries with the setting `Use NNUE` set to true. Make sure that your previously trained network is in the eval folder. Use the commands specified above. Make sure `SkipLoadingEval` is set to false so that the data generated is using the neural net's eval by typing the command `setoption name SkipLoadingEval value false` before typing the `isready` command. You should aim to generate less positions than the first run, around 1/10 of the number of positions generated in the first run. The depth should be higher as well. You should also do the same for validation data, with the depth being higher than the last run. After you have generated the training data, you must move it into your training data folder and delete the older data so that the binary does not accidentally train on the same data again. Do the same for the validation data and name it to val-1.bin to make it less confusing. Make sure the evalsave folder is empty. Then, using the same binary, type in the training commands shown above. Do __NOT__ set `SkipLoadingEval` to true, it must be false or you will get a completely new network, instead of a network trained with reinforcement learning. You should also set eval_save_interval to a number that is lower than the amount of positions in your training data, perhaps also 1/10 of the original value. The validation file should be set to the new validation data, not the old data. After training is finished, your new net should be located in the "final" folder under the "evalsave" directory. You should test this new network against the older network to see if there are any improvements. ## Using Your Trained Net -If you want to use your generated net, copy the net located in the "final" folder under the "evalsave" directory and move it into a new folder named "eval" under the directory with the binaries. You can then use the halfkp_256x2 binaries pertaining to your CPU with a standard chess GUI, such as Cutechess. Refer to the [releases page](https://github.com/nodchip/Stockfish/releases) to find out which binary is best for your CPU. +If you want to use your generated net, copy the net located in the "final" folder under the "evalsave" directory and move it into a new folder named "eval" under the directory with the binaries. You can then use the halfkp_256x2 binaries pertaining to your CPU with a standard chess GUI, such as Cutechess. Refer to the [releases page](https://abrok.eu/stockfish) to find out which binary is best for your CPU. -If the engine does not load any net file, or shows "Error! *** not found or wrong format", please try to sepcify the net with the full file path with the "EvalFile" option by typing the command `setoption name EvalFile value path` where path is the full file path. +If the engine does not load any net file, or shows "Error! *** not found or wrong format", please try to specify the net with the full file path with the "EvalFile" option by typing the command `setoption name EvalFile value path` where path is the full file path. The "Use NNUE" option must be set to true with the command `setoption name Use NNUE value true`. ## Resources - [Stockfish NNUE Wiki](https://www.qhapaq.org/shogi/shogiwiki/stockfish-nnue/) diff --git a/Readme.md b/Readme.md deleted file mode 100644 index 7a237480..00000000 --- a/Readme.md +++ /dev/null @@ -1,65 +0,0 @@ -

- -

- -

Stockfish NNUE

- -## Overview -Stockfish NNUE is a port of a shogi neural network named NNUE (efficiently updateable neural network backwards) to Stockfish 11. To learn more about the Stockfish chess engine, look [here](stockfish.md) for an overview and [here](https://github.com/official-stockfish/Stockfish) for the official repository. - -## Training Guide -### Generating Training Data -To generate training data from the classic eval, use gensfen command with setting "Use NNUE" to "false". The given example is generation in its simplest form. There are more commands. -``` -uci -setoption name Use NNUE value false -setoption name Threads value x -setoption name Hash value y -setoption name SyzygyPath value path -isready -gensfen depth a loop b use_draw_in_training_data_generation 1 eval_limit 32000 -``` -Specify how many threads and how much memory you would like to use with the x and y values. The option SyzygyPath is not necessary, but if you would like to use it, you must first have Syzygy endgame tablebases on your computer, which you can find [here](http://oics.olympuschess.com/tracker/index.php). You will need to have a torrent client to download these tablebases, as that is probably the fastest way to obtain them. The path is the path to the folder containing those tablebases. It does not have to be surrounded in quotes. - -This will save a file named "generated_kifu.bin" in the same folder as the binary. Once generation is done, rename the file to something like "1billiondepth12.bin" to remember the depth and quantity of the positions and move it to a folder named "trainingdata" in the same directory as the binaries. -#### Generation Parameters -- Depth is the searched depth per move, or how far the engine looks forward. This value is an integer. -- Loop is the amount of positions generated. This value is also an integer -### Generating Validation Data -The process is the same as the generation of training data, except for the fact that you need to set loop to 1 million, because you don't need a lot of validation data. The depth should be the same as before or slightly higher than the depth of the training data. After generation rename the validation data file to val.bin and drop it in a folder named "validationdata" in the same directory to make it easier. -### Training a Completely New Network -Use the "learn" binary. Create an empty folder named "evalsave" in the same directory as the binaries. -``` -uci -setoption name SkipLoadingEval value true -setoption name Use NNUE value true -setoption name Threads value x -isready -learn targetdir trainingdata loop 100 batchsize 1000000 use_draw_in_training 1 use_draw_in_validation 1 eta 1 lambda 1 eval_limit 32000 nn_batch_size 1000 newbob_decay 0.5 eval_save_interval 250000000 loss_output_interval 1000000 mirror_percentage 50 validation_set_file_name validationdata\val.bin -``` -Nets get saved in the "evalsave" folder. - -#### Training Parameters -- eta is the learning rate -- lambda is the amount of weight it puts to eval of learning data vs win/draw/loss results. 1 puts all weight on eval, lambda 0 puts all weight on WDL results. - -### Reinforcement Learning -If you would like to do some reinforcement learning on your original network, you must first generate training data using the learn binaries with setting `Use NNUE` to true. Make sure that your previously trained network is in the eval folder. Use the commands specified above. Make sure `SkipLoadingEval` is set to false so that the data generated is using the neural net's eval by typing the command `uci setoption name SkipLoadingEval value false` before typing the `isready` command. You should aim to generate less positions than the first run, around 1/10 of the number of positions generated in the first run. The depth should be higher as well. You should also do the same for validation data, with the depth being higher than the last run. - -After you have generated the training data, you must move it into your training data folder and delete the older data so that the binary does not accidentally train on the same data again. Do the same for the validation data and name it to val-1.bin to make it less confusing. Make sure the evalsave folder is empty. Then, using the same binary, type in the training commands shown above. Do __NOT__ set `SkipLoadingEval` to true, it must be false or you will get a completely new network, instead of a network trained with reinforcement learning. You should also set eval_save_interval to a number that is lower than the amount of positions in your training data, perhaps also 1/10 of the original value. The validation file should be set to the new validation data, not the old data. - -After training is finished, your new net should be located in the "final" folder under the "evalsave" directory. You should test this new network against the older network to see if there are any improvements. - -## Using Your Trained Net -If you want to use your generated net, copy the net located in the "final" folder under the "evalsave" directory and move it into a new folder named "eval" under the directory with the binaries. You can then use the halfkp_256x2 binaries pertaining to your CPU with a standard chess GUI, such as Cutechess. Refer to the [releases page](https://github.com/nodchip/Stockfish/releases) to find out which binary is best for your CPU. - -If the engine does not load any net file, or shows "Error! *** not found or wrong format", please try to sepcify the net with the full file path with the "EvalFile" option by typing the command `setoption name EvalFile value path` where path is the full file path. - -## Resources -- [Stockfish NNUE Wiki](https://www.qhapaq.org/shogi/shogiwiki/stockfish-nnue/) -- [Training instructions](https://twitter.com/mktakizawa/status/1273042640280252416) from the creator of the Elmo shogi engine -- [Original Talkchess thread](http://talkchess.com/forum3/viewtopic.php?t=74059) discussing Stockfish NNUE -- [Guide to Stockfish NNUE](http://yaneuraou.yaneu.com/2020/06/19/stockfish-nnue-the-complete-guide/) -- [Unofficial Stockfish Discord](https://discord.gg/nv8gDtt) - -A more updated list can be found in the #sf-nnue-resources channel in the Discord. diff --git a/src/learn/learner.cpp b/src/learn/learner.cpp index fc178c92..acc028f5 100644 --- a/src/learn/learner.cpp +++ b/src/learn/learner.cpp @@ -2539,8 +2539,26 @@ void shuffle_files_on_memory(const vector& filenames,const string output std::cout << "..shuffle_on_memory done." << std::endl; } -void convert_bin(const vector& filenames, const string& output_file_name, const int ply_minimum, const int ply_maximum, const int interpolate_eval, const bool check_illegal_move) +bool fen_is_ok(Position& pos, std::string input_fen) { + std::string pos_fen = pos.fen(); + std::istringstream ss_input(input_fen); + std::istringstream ss_pos(pos_fen); + + // example : "2r4r/4kpp1/nb1np3/p2p3p/B2P1BP1/PP6/4NPKP/2R1R3 w - h6 0 24" + // --> "2r4r/4kpp1/nb1np3/p2p3p/B2P1BP1/PP6/4NPKP/2R1R3" + std::string str_input, str_pos; + ss_input >> str_input; + ss_pos >> str_pos; + + // Only compare "Piece placement field" between input_fen and pos.fen(). + return str_input == str_pos; +} + +void convert_bin(const vector& filenames, const string& output_file_name, const int ply_minimum, const int ply_maximum, const int interpolate_eval, const bool check_invalid_fen, const bool check_illegal_move) { + std::cout << "check_invalid_fen=" << check_invalid_fen << std::endl; + std::cout << "check_illegal_move=" << check_illegal_move << std::endl; + std::fstream fs; uint64_t data_size=0; uint64_t filtered_size = 0; @@ -2573,16 +2591,16 @@ void convert_bin(const vector& filenames, const string& output_file_name std::string value; ss >> token; if (token == "fen") { - states = StateListPtr(new std::deque(1)); // Drop old and create a new one + states = StateListPtr(new std::deque(1)); // Drop old and create a new one std::string input_fen = line.substr(4); tpos.set(input_fen, false, &states->back(), Threads.main()); - if (!tpos.pos_is_ok() || tpos.fen() != input_fen) { + if (check_invalid_fen && !fen_is_ok(tpos, input_fen)) { ignore_flag_fen = true; filtered_size_fen++; } else { - tpos.sfen_pack(p.sfen); - } + tpos.sfen_pack(p.sfen); + } } else if (token == "move") { ss >> value; @@ -2609,7 +2627,7 @@ void convert_bin(const vector& filenames, const string& output_file_name } p.gamePly = uint16_t(temp); // No cast here? if (interpolate_eval != 0){ - p.score = min(3000, interpolate_eval * temp); + p.score = min(3000, interpolate_eval * temp); } } else if (token == "result") { @@ -2617,17 +2635,17 @@ void convert_bin(const vector& filenames, const string& output_file_name ss >> temp; p.game_result = int8_t(temp); // Do you need a cast here? if (interpolate_eval){ - p.score = p.score * p.game_result; + p.score = p.score * p.game_result; } } else if (token == "e") { - if(!(ignore_flag_fen || ignore_flag_move || ignore_flag_ply)){ - fs.write((char*)&p, sizeof(PackedSfenValue)); - data_size+=1; - // debug - // std::cout<& filenames, const string& output_file_name } } std::cout << "done " << data_size << " parsed " << filtered_size << " is filtered" - << " (illegal fen:" << filtered_size_fen << ", illegal move:" << filtered_size_move << ", illegal ply:" << filtered_size_ply << ")" << std::endl; + << " (invalid fen:" << filtered_size_fen << ", illegal move:" << filtered_size_move << ", invalid ply:" << filtered_size_ply << ")" << std::endl; ifs.close(); } std::cout << "all done" << std::endl; @@ -2985,6 +3003,7 @@ void learn(Position&, istringstream& is) int ply_minimum = 0; int ply_maximum = 114514; bool interpolate_eval = 0; + bool check_invalid_fen = false; bool check_illegal_move = false; // convert teacher in pgn-extract format to Yaneura King's bin bool use_convert_bin_from_pgn_extract = false; @@ -3126,6 +3145,7 @@ void learn(Position&, istringstream& is) else if (option == "convert_plain") use_convert_plain = true; else if (option == "convert_bin") use_convert_bin = true; else if (option == "interpolate_eval") is >> interpolate_eval; + else if (option == "check_invalid_fen") is >> check_invalid_fen; else if (option == "check_illegal_move") is >> check_illegal_move; else if (option == "convert_bin_from_pgn-extract") use_convert_bin_from_pgn_extract = true; else if (option == "pgn_eval_side_to_move") is >> pgn_eval_side_to_move; @@ -3238,7 +3258,7 @@ void learn(Position&, istringstream& is) { Eval::init_NNUE(); cout << "convert_bin.." << endl; - convert_bin(filenames,output_file_name, ply_minimum, ply_maximum, interpolate_eval, check_illegal_move); + convert_bin(filenames,output_file_name, ply_minimum, ply_maximum, interpolate_eval, check_invalid_fen, check_illegal_move); return; } diff --git a/src/search.cpp b/src/search.cpp index fe1771a3..b7561a96 100644 --- a/src/search.cpp +++ b/src/search.cpp @@ -68,6 +68,8 @@ namespace { return Value(227 * (d - improving)); } + bool training; + // Reductions lookup table, initialized at startup int Reductions[MAX_MOVES]; // [depth or moveNumber] @@ -193,6 +195,8 @@ void Search::init() { for (int i = 1; i < MAX_MOVES; ++i) Reductions[i] = int((24.8 + std::log(Threads.size())) * std::log(i)); + + training = Options["Training"]; } @@ -1013,6 +1017,7 @@ moves_loop: // When in check, search starts from here // Step 13. Pruning at shallow depth (~200 Elo) if ( !rootNode + && !(training && PvNode) && pos.non_pawn_material(us) && bestValue > VALUE_TB_LOSS_IN_MAX_PLY) { @@ -2070,10 +2075,10 @@ namespace Learner // Increase the generation of the substitution table for this thread because it is a new search. //TT.new_search(th->thread_id()); - // If you call new_search here, it may be a loss because you can't use the previous search result. + // ª If you call new_search here, it may be a loss because you can't use the previous search result. // Do not do this here, but caller should do TT.new_search(th->thread_id()) for each station ... - // Because we want to avoid reaching the same final diagram, use the substitution table commonly for all threads when generating teachers. + // ¨Because we want to avoid reaching the same final diagram, use the substitution table commonly for all threads when generating teachers. //#endif } } @@ -2263,7 +2268,7 @@ namespace Learner } // Pass PV_is(ok) to eliminate this PV, there may be NULL_MOVE in the middle. - // PV should not be NULL_MOVE because it is PV + // ¨ PV should not be NULL_MOVE because it is PV // MOVE_WIN has never been thrust. (For now) for (Move move : rootMoves[0].pv) { diff --git a/src/ucioption.cpp b/src/ucioption.cpp index 168e73a9..ef40fe82 100644 --- a/src/ucioption.cpp +++ b/src/ucioption.cpp @@ -69,6 +69,7 @@ void init(OptionsMap& o) { o["Move Overhead"] << Option(10, 0, 5000); o["Slow Mover"] << Option(100, 10, 1000); o["nodestime"] << Option(0, 0, 10000); + o["Training"] << Option(false); o["UCI_Chess960"] << Option(false); o["UCI_AnalyseMode"] << Option(false); o["UCI_LimitStrength"] << Option(false);