Commit Graph

5825 Commits

Author SHA1 Message Date
Vizvezdenec
be7a03a957 Introduce static history
The idea of this patch can be described as following: we update static
history stats based on comparison of the static evaluations of the
position before and after the move. If the move increases static evaluation
it's assigned positive bonus, if it decreases static evaluation
it's assigned negative bonus. These stats are used in movepicker
to sort quiet moves.

passed STC
https://tests.stockfishchess.org/tests/view/5fca4c0842a050a89f02cd66
LLR: 3.00 (-2.94,2.94) {-0.25,1.25}
Total: 78152 W: 7409 L: 7171 D: 63572
Ptnml(0-2): 303, 5695, 26873, 5871, 334

passed LTC
https://tests.stockfishchess.org/tests/view/5fca6be442a050a89f02cd75
LLR: 2.94 (-2.94,2.94) {0.25,1.25}
Total: 40240 W: 1602 L: 1441 D: 37197
Ptnml(0-2): 19, 1306, 17305, 1475, 15

closes https://github.com/official-stockfish/Stockfish/pull/3253

bench 3845156
2020-12-05 16:48:33 +01:00
Tomasz Sobczyk
28d6d7cb03 Avoid computing gradient for validation loss. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
fafb9557a8 Get train loss from update_parameters. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
4eb0e77a2a Store references instead of copying the results of intermediate autograd computations. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
6cd0b03098 Add some comments regarding the current state of autograd loss computation. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
99cb869db3 Reintroduce use_wdl. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
cf6bc7ecaf Cleanup around get_loss 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
256c4b55ec Properly apply gradient norm clipping after it's scaled in the update_parameters. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
de675e3503 Reintroduce optional scaling of the teacher signal. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
01ae7b1e2c Simplify passing constants that may vary between calls. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
cbd973fdaa Detect constant expressions in autograd and return 0 grad early. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
e975889132 Move cross_entropy calculation to a separate function. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
891abf5511 Make the autograd loss expression chain thread_local. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
8adf00ae6e Identify a single evalation chain by ID in autograd to prevent cache reuse for subsequent evaluations of the same expression tree. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
cb812c742c Add [[nodiscard]] attributes to autograd functions. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
26f19e1429 Make automatic differentiation node types constexpr. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
aec6017195 When forming an autograd expression only copy parts that are rvalue references, store references to lvalues. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
a5c20bee5b Apply gradient clipping. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
d103867558 Add memoization to the autograd expression evaluator. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
aa55692b97 Cross entropy loss. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
539bd2d1c8 Replace the old loss/grad calculation completely. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
b71d1e8620 Pass the new loss function to update_parameters 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
5a58eb803a Loss func with autograd 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
541fb8177a More utility in autograd. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
6ce0245787 Basic autograd 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
1322a9a5fd Prevent false sharing of num_calls counter in the shared input trainer. Fix current_operation not being local to the executing thread. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk
2aa7f5290e Fix comparison of integers with different signedness. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk
a97b65eaef Fix compilation error with USE_BLAS 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk
622e0b14c2 Remove superfluous example shuffling. Shuffling now only happens on reading. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk
34510dd08a Remove used examples asyncronously. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk
0bee8fef64 Don't unnecessarily copy the batch part. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk
e954b14196 Prefetch weights for feature transformer backprop to shared cache. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk
8009973381 Special case for alpha=1 in saxpy, slight performance increase. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk
49b2dcb1f3 Preallocate memory for unique_features. Keep the training_features temporary buffer as a thread_local so we reuse the storage. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk
1c8495b54b Remove handwritten saxpy because compilers optimize the second look anyway. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk
15c528ca7b Prepare feature transformer learner. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk
a3c78691a2 Prepare input slice trainer. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk
401fc0fbab Prepare clipped relu trainer. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk
774b023641 Add chunked for each with workers. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk
cc11375f6d Skeleton for new evaluate learner 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk
0d4b803b08 Prepare trainer affine transform. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk
4ea8572b6d Add single threaded sgemm. 2020-11-30 08:54:53 +09:00
SFisGOD
7364006757 Update default net to nn-62ef826d1a6d.nnue
Include scaling change as suggested by Dietrich Kappe,
the one who trained net for Komodo.  According to him,
some nets may require different scaling in order to utilize its full strength.

STC:
LLR: 2.93 (-2.94,2.94) {-0.25,1.25}
Total: 99856 W: 9669 L: 9401 D: 80786
Ptnml(0-2): 374, 7468, 34037, 7614, 435
https://tests.stockfishchess.org/tests/view/5fc2697642a050a89f02c8ec

LTC:
LLR: 2.96 (-2.94,2.94) {0.25,1.25}
Total: 29840 W: 1220 L: 1081 D: 27539
Ptnml(0-2): 10, 969, 12827, 1100, 14
https://tests.stockfishchess.org/tests/view/5fc2ea5142a050a89f02c957

Bench: 3561701
2020-11-29 16:54:06 +01:00
Unai Corzo
2442ba2b0e Reductions simplification
Simplify increase reduction for captures/promotions if late move and at low depth.

STC https://tests.stockfishchess.org/tests/view/5fbff65067cbf42301d6b3ae
LLR: 2.97 (-2.94,2.94) {-1.25,0.25}
Total: 49088 W: 4607 L: 4555 D: 39926
Ptnml(0-2): 177, 3615, 16932, 3619, 201

LTC https://tests.stockfishchess.org/tests/view/5fc0902967cbf42301d6b3fc
LLR: 2.99 (-2.94,2.94) {-0.75,0.25}
Total: 160944 W: 6153 L: 6193 D: 148598
Ptnml(0-2): 90, 5525, 69294, 5461, 102

closes https://github.com/official-stockfish/Stockfish/pull/3248

bench: 3834568
2020-11-29 16:52:51 +01:00
syzygy1
045728a7da Remove piece lists
This patch removes the incrementally updated piece lists from the Position object.

This has been tried before but always failed. My reasons for trying again are:

* 32-bit systems (including phones) are now much less important than they were some years ago (and are absent from fishtest);
* NNUE may have made SF less finely tuned to the order in which moves were generated.

STC:
LLR: 2.94 (-2.94,2.94) {-1.25,0.25}
Total: 55272 W: 5260 L: 5216 D: 44796
Ptnml(0-2): 208, 4147, 18898, 4159, 224
https://tests.stockfishchess.org/tests/view/5fc2986a42a050a89f02c926

LTC:
LLR: 2.96 (-2.94,2.94) {-0.75,0.25}
Total: 16600 W: 673 L: 608 D: 15319
Ptnml(0-2): 14, 533, 7138, 604, 11
https://tests.stockfishchess.org/tests/view/5fc2f98342a050a89f02c95c

closes https://github.com/official-stockfish/Stockfish/pull/3247

Bench: 3940967
2020-11-29 16:51:01 +01:00
Unai Corzo
2bc4ae172a Update README.md
fix a few typos

closes https://github.com/official-stockfish/Stockfish/pull/3245

No functional change
2020-11-29 16:47:00 +01:00
Unai Corzo
6c429c4d65 Search simplification
STC https://tests.stockfishchess.org/tests/view/5fc2083942a050a89f02c8bb
LLR: 2.93 (-2.94,2.94) {-1.25,0.25}
Total: 23200 W: 2251 L: 2160 D: 18789
Ptnml(0-2): 86, 1726, 7895, 1797, 96

LTC https://tests.stockfishchess.org/tests/view/5fc22d7b42a050a89f02c8d0
LLR: 2.92 (-2.94,2.94) {-0.75,0.25}
Total: 15832 W: 653 L: 590 D: 14589
Ptnml(0-2): 7, 521, 6795, 588, 5

closes https://github.com/official-stockfish/Stockfish/pull/3244/files

bench: 3827317

Simplify search.
2020-11-29 16:42:40 +01:00
lonfom169
66da1e802c Remove bonus for killers.
Passed non-regression STC:
LLR: 2.94 (-2.94,2.94) {-1.25,0.25}
Total: 14712 W: 1416 L: 1315 D: 11981
Ptnml(0-2): 59, 1029, 5082, 1124, 62
https://tests.stockfishchess.org/tests/view/5fbfa31f67cbf42301d6b36e

Passed non-regression LTC:
LLR: 2.95 (-2.94,2.94) {-0.75,0.25}
Total: 27536 W: 1099 L: 1044 D: 25393
Ptnml(0-2): 11, 929, 11838, 974, 16
https://tests.stockfishchess.org/tests/view/5fbfac9167cbf42301d6b371

closes https://github.com/official-stockfish/Stockfish/pull/3241

Bench: 3887644
2020-11-29 16:40:06 +01:00
Lolligerhans
d6d6972a66 Refine rook penalty on closed files
+-----------------+
| . . . . . . . . | All files are closed. Some files are
| . . . . . o o . | more valuable for rooks, because
| . . . . o . . o | they might open in the future.
| . . . o x . . x |
| o . o x . x x . |
| x o x . . . . . | x  our pawns
| . x . . . . . . | o  their pawns
| . . . . . . . . | ^  rooks are scored higher on these files
+-----------------+
            ^ ^

Files containing none of our own pawns are open or half-open (otherwise
they are closed). Rooks on (half-)open files recieve a bonus for the
future potential to act along all ranks.

This commit refines the (relative) penalty of rooks on closed files.
Files that contain one of our blocked pawns are considered less likely
to open in the future; rooks on these files are now penalized stronger.

This bonus does not generally correlate with mobility. If the condition
is sufficiently refined in the future, it may be beneficial to adjust or
override mobility scores in some cases.

LTC
LLR: 2.94 (-2.94,2.94) {0.25,1.25}
Total: 494384 W: 71565 L: 70231 D: 352588
Ptnml(0-2): 3907, 48050, 142118, 49036, 4081
https://tests.stockfishchess.org/tests/view/5fb9312e67cbf42301d6afb9

LTC (non-regression w/ book noob_3moves.epd)
LLR: 2.95 (-2.94,2.94) {-0.75,0.25}
Total: 208520 W: 27044 L: 26937 D: 154539
Ptnml(0-2): 1557, 19850, 61391, 19853, 1609
https://tests.stockfishchess.org/tests/view/5fc01ced67cbf42301d6b3df

STC
LLR: 2.94 (-2.94,2.94) {-0.25,1.25}
Total: 98392 W: 20269 L: 19868 D: 58255
Ptnml(0-2): 1804, 11297, 22589, 11706, 1800
https://tests.stockfishchess.org/tests/view/5fb7f88a67cbf42301d6af10

closes https://github.com/official-stockfish/Stockfish/pull/3242

Bench: 3682630
2020-11-29 16:38:03 +01:00
mstembera
9b7983a452 Cleaned up MakeIndex()
The index order in kpp_board_index[][] is reversed to be more optimal for the access pattern

STC https://tests.stockfishchess.org/tests/view/5fbd74f967cbf42301d6b24f
LLR: 2.93 (-2.94,2.94) {-1.25,0.25}
Total: 27504 W: 2686 L: 2607 D: 22211
Ptnml(0-2): 84, 2001, 9526, 2034, 107

closes https://github.com/official-stockfish/Stockfish/pull/3233

No functional change
2020-11-29 16:36:49 +01:00