HCha/Stockfish
1
0
Fork 0
mirror of https://github.com/HChaZZY/Stockfish.git synced 2025-12-27 04:26:24 +08:00
Code Issues Packages Projects Releases Wiki Activity

New NNUE architecture and net

Browse Source 
Introduces a new NNUE network architecture and associated network parameters,
as obtained by a new pytorch trainer.

The network is already very strong at short TC, without regression at longer TC,
and has potential for further improvements.

https://tests.stockfishchess.org/tests/view/60a159c65085663412d0921d
TC: 10s+0.1s, 1 thread
ELO: 21.74 +-3.4 (95%) LOS: 100.0%
Total: 10000 W: 1559 L: 934 D: 7507
Ptnml(0-2): 38, 701, 2972, 1176, 113

https://tests.stockfishchess.org/tests/view/60a187005085663412d0925b
TC: 60s+0.6s, 1 thread
ELO: 5.85 +-1.7 (95%) LOS: 100.0%
Total: 20000 W: 1381 L: 1044 D: 17575
Ptnml(0-2): 27, 885, 7864, 1172, 52

https://tests.stockfishchess.org/tests/view/60a2beede229097940a03806
TC: 20s+0.2s, 8 threads
LLR: 2.93 (-2.94,2.94) <0.50,3.50>
Total: 34272 W: 1610 L: 1452 D: 31210
Ptnml(0-2): 30, 1285, 14350, 1439, 32

https://tests.stockfishchess.org/tests/view/60a2d687e229097940a03c72
TC: 60s+0.6s, 8 threads
LLR: 2.94 (-2.94,2.94) <-2.50,0.50>
Total: 45544 W: 1262 L: 1214 D: 43068
Ptnml(0-2): 12, 1129, 20442, 1177, 12

The network has been trained (by vondele) using the https://github.com/glinscott/nnue-pytorch/ trainer (started by glinscott),
specifically the branch https://github.com/Sopel97/nnue-pytorch/tree/experiment_56.
The data used are in 64 billion positions (193GB total) generated and scored with the current master net
d8: https://drive.google.com/file/d/1hOOYSDKgOOp38ZmD0N4DV82TOLHzjUiF/view?usp=sharing
d9: https://drive.google.com/file/d/1VlhnHL8f-20AXhGkILujnNXHwy9T-MQw/view?usp=sharing
d10: https://drive.google.com/file/d/1ZC5upzBYMmMj1gMYCkt6rCxQG0GnO3Kk/view?usp=sharing
fishtest_d9: https://drive.google.com/file/d/1GQHt0oNgKaHazwJFTRbXhlCN3FbUedFq/view?usp=sharing

This network also contains a few architectural changes with respect to the current master:

    Size changed from 256x2-32-32-1 to 512x2-16-32-1
        ~15-20% slower
        ~2x larger
        adds a special path for 16 valued ClippedReLU
        fixes affine transform code for 16 inputs/outputs, buy using InputDimensions instead of PaddedInputDimensions
            this is safe now because the inputs are processed in groups of 4 in the current affine transform code
    The feature set changed from HalfKP to HalfKAv2
        Includes information about the kings like HalfKA
        Packs king features better, resulting in 8% size reduction compared to HalfKA
    The board is flipped for the black's perspective, instead of rotated like in the current master
    PSQT values for each feature
        the feature transformer now outputs a part that is fowarded directly to the output and allows learning piece values more directly than the previous network architecture. The effect is visible for high imbalance positions, where the current master network outputs evaluations skewed towards zero.
        8 PSQT values per feature, chosen based on (popcount(pos.pieces()) - 1) / 4
        initialized to classical material values on the start of the training
    8 subnetworks (512x2->16->32->1), chosen based on (popcount(pos.pieces()) - 1) / 4
        only one subnetwork is evaluated for any position, no or marginal speed loss

A diagram of the network is available: https://user-images.githubusercontent.com/8037982/118656988-553a1700-b7eb-11eb-82ef-56a11cbebbf2.png
A more complete description: https://github.com/glinscott/nnue-pytorch/blob/master/docs/nnue.md

closes https://github.com/official-stockfish/Stockfish/pull/3474

Bench: 3806488
This commit is contained in:
Tomasz Sobczyk
2021-05-18 17:36:26 +02:00
committed by Joost VandeVondele
parent f90274d8ce
commit e8d64af123
13 changed files with 265 additions and 173 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
src/nnue/features/half_kp.cpp → src/nnue/features/half_ka_v2.cpp
View File

@@ -16,32 +16,32 @@
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
//Definition of input features HalfKP of NNUE evaluation function
//Definition of input features HalfKAv2 of NNUE evaluation function
#include "half_kp.h"
#include "half_ka_v2.h"
#include "../../position.h"
namespace Stockfish::Eval::NNUE::Features {
// Orient a square according to perspective (rotates by 180 for black)
inline Square HalfKP::orient(Color perspective, Square s) {
return Square(int(s) ^ (bool(perspective) * 63));
inline Square HalfKAv2::orient(Color perspective, Square s) {
return Square(int(s) ^ (bool(perspective) * 56));
}
// Index of a feature for a given king position and another piece on some square
inline IndexType HalfKP::make_index(Color perspective, Square s, Piece pc, Square ksq) {
inline IndexType HalfKAv2::make_index(Color perspective, Square s, Piece pc, Square ksq) {
return IndexType(orient(perspective, s) + PieceSquareIndex[perspective][pc] + PS_NB * ksq);
}
// Get a list of indices for active features
void HalfKP::append_active_indices(
void HalfKAv2::append_active_indices(
const Position& pos,
Color perspective,
ValueListInserter<IndexType> active
) {
Square ksq = orient(perspective, pos.square<KING>(perspective));
Bitboard bb = pos.pieces() & ~pos.pieces(KING);
Bitboard bb = pos.pieces();
while (bb)
{
Square s = pop_lsb(bb);
@@ -52,7 +52,7 @@ namespace Stockfish::Eval::NNUE::Features {
// append_changed_indices() : get a list of indices for recently changed features
void HalfKP::append_changed_indices(
void HalfKAv2::append_changed_indices(
Square ksq,
StateInfo* st,
Color perspective,
@@ -63,7 +63,6 @@ namespace Stockfish::Eval::NNUE::Features {
Square oriented_ksq = orient(perspective, ksq);
for (int i = 0; i < dp.dirty_num; ++i) {
Piece pc = dp.piece[i];
if (type_of(pc) == KING) continue;
if (dp.from[i] != SQ_NONE)
removed.push_back(make_index(perspective, dp.from[i], pc, oriented_ksq));
if (dp.to[i] != SQ_NONE)
@@ -71,15 +70,15 @@ namespace Stockfish::Eval::NNUE::Features {
}
}
int HalfKP::update_cost(StateInfo* st) {
int HalfKAv2::update_cost(StateInfo* st) {
return st->dirtyPiece.dirty_num;
}
int HalfKP::refresh_cost(const Position& pos) {
return pos.count<ALL_PIECES>() - 2;
int HalfKAv2::refresh_cost(const Position& pos) {
return pos.count<ALL_PIECES>();
}
bool HalfKP::requires_refresh(StateInfo* st, Color perspective) {
bool HalfKAv2::requires_refresh(StateInfo* st, Color perspective) {
return st->dirtyPiece.piece[0] == make_piece(perspective, KING);
}

51
src/nnue/features/half_kp.h → src/nnue/features/half_ka_v2.h
View File

@@ -18,8 +18,8 @@
//Definition of input features HalfKP of NNUE evaluation function
#ifndef NNUE_FEATURES_HALF_KP_H_INCLUDED
#define NNUE_FEATURES_HALF_KP_H_INCLUDED
#ifndef NNUE_FEATURES_HALF_KA_V2_H_INCLUDED
#define NNUE_FEATURES_HALF_KA_V2_H_INCLUDED
#include "../nnue_common.h"
@@ -32,33 +32,34 @@ namespace Stockfish {
namespace Stockfish::Eval::NNUE::Features {
// Feature HalfKP: Combination of the position of own king
// and the position of pieces other than kings
class HalfKP {
// Feature HalfKAv2: Combination of the position of own king
// and the position of pieces
class HalfKAv2 {
// unique number for each piece type on each square
enum {
PS_NONE = 0,
PS_W_PAWN = 1,
PS_B_PAWN = 1 * SQUARE_NB + 1,
PS_W_KNIGHT = 2 * SQUARE_NB + 1,
PS_B_KNIGHT = 3 * SQUARE_NB + 1,
PS_W_BISHOP = 4 * SQUARE_NB + 1,
PS_B_BISHOP = 5 * SQUARE_NB + 1,
PS_W_ROOK = 6 * SQUARE_NB + 1,
PS_B_ROOK = 7 * SQUARE_NB + 1,
PS_W_QUEEN = 8 * SQUARE_NB + 1,
PS_B_QUEEN = 9 * SQUARE_NB + 1,
PS_NB = 10 * SQUARE_NB + 1
PS_W_PAWN = 0,
PS_B_PAWN = 1 * SQUARE_NB,
PS_W_KNIGHT = 2 * SQUARE_NB,
PS_B_KNIGHT = 3 * SQUARE_NB,
PS_W_BISHOP = 4 * SQUARE_NB,
PS_B_BISHOP = 5 * SQUARE_NB,
PS_W_ROOK = 6 * SQUARE_NB,
PS_B_ROOK = 7 * SQUARE_NB,
PS_W_QUEEN = 8 * SQUARE_NB,
PS_B_QUEEN = 9 * SQUARE_NB,
PS_KING = 10 * SQUARE_NB,
PS_NB = 11 * SQUARE_NB
};
static constexpr IndexType PieceSquareIndex[COLOR_NB][PIECE_NB] = {
// convention: W - us, B - them
// viewed from other side, W and B are reversed
{ PS_NONE, PS_W_PAWN, PS_W_KNIGHT, PS_W_BISHOP, PS_W_ROOK, PS_W_QUEEN, PS_NONE, PS_NONE,
PS_NONE, PS_B_PAWN, PS_B_KNIGHT, PS_B_BISHOP, PS_B_ROOK, PS_B_QUEEN, PS_NONE, PS_NONE },
{ PS_NONE, PS_B_PAWN, PS_B_KNIGHT, PS_B_BISHOP, PS_B_ROOK, PS_B_QUEEN, PS_NONE, PS_NONE,
PS_NONE, PS_W_PAWN, PS_W_KNIGHT, PS_W_BISHOP, PS_W_ROOK, PS_W_QUEEN, PS_NONE, PS_NONE }
{ PS_NONE, PS_W_PAWN, PS_W_KNIGHT, PS_W_BISHOP, PS_W_ROOK, PS_W_QUEEN, PS_KING, PS_NONE,
PS_NONE, PS_B_PAWN, PS_B_KNIGHT, PS_B_BISHOP, PS_B_ROOK, PS_B_QUEEN, PS_KING, PS_NONE },
{ PS_NONE, PS_B_PAWN, PS_B_KNIGHT, PS_B_BISHOP, PS_B_ROOK, PS_B_QUEEN, PS_KING, PS_NONE,
PS_NONE, PS_W_PAWN, PS_W_KNIGHT, PS_W_BISHOP, PS_W_ROOK, PS_W_QUEEN, PS_KING, PS_NONE }
};
// Orient a square according to perspective (rotates by 180 for black)
@@ -69,17 +70,17 @@ namespace Stockfish::Eval::NNUE::Features {
public:
// Feature name
static constexpr const char* Name = "HalfKP(Friend)";
static constexpr const char* Name = "HalfKAv2(Friend)";
// Hash value embedded in the evaluation file
static constexpr std::uint32_t HashValue = 0x5D69D5B8u;
static constexpr std::uint32_t HashValue = 0x5f234cb8u;
// Number of feature dimensions
static constexpr IndexType Dimensions =
static_cast<IndexType>(SQUARE_NB) * static_cast<IndexType>(PS_NB);
// Maximum number of simultaneously active features. 30 because kins are not included.
static constexpr IndexType MaxActiveDimensions = 30;
// Maximum number of simultaneously active features.
static constexpr IndexType MaxActiveDimensions = 32;
// Get a list of indices for active features
static void append_active_indices(
@@ -107,4 +108,4 @@ namespace Stockfish::Eval::NNUE::Features {
} // namespace Stockfish::Eval::NNUE::Features
#endif // #ifndef NNUE_FEATURES_HALF_KP_H_INCLUDED
#endif // #ifndef NNUE_FEATURES_HALF_KA_V2_H_INCLUDED