nodchip
|
ae045e2cd8
|
Merge pull request #258 from kennyfrc/stockfish-nnue-2020-08-30-macos
mac-compatible makefile with instructions for stockfish-nnue-2020-08-30
|
2020-12-09 08:39:36 +09:00 |
|
Kenn Costales
|
055f907315
|
Merge branch 'master' into stockfish-nnue-2020-08-30-macos
|
2020-12-08 22:49:11 +08:00 |
|
kennyfrc
|
bb26ce5aa1
|
mac specific makefile with compilation instructions
|
2020-12-08 22:14:18 +08:00 |
|
Tomasz Sobczyk
|
3a1bd1185f
|
Add binpack coarse shuffle tool.
|
2020-12-06 19:08:52 +09:00 |
|
Tomasz Sobczyk
|
28d6d7cb03
|
Avoid computing gradient for validation loss.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
fafb9557a8
|
Get train loss from update_parameters.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
4eb0e77a2a
|
Store references instead of copying the results of intermediate autograd computations.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
6cd0b03098
|
Add some comments regarding the current state of autograd loss computation.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
99cb869db3
|
Reintroduce use_wdl.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
cf6bc7ecaf
|
Cleanup around get_loss
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
256c4b55ec
|
Properly apply gradient norm clipping after it's scaled in the update_parameters.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
de675e3503
|
Reintroduce optional scaling of the teacher signal.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
01ae7b1e2c
|
Simplify passing constants that may vary between calls.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
cbd973fdaa
|
Detect constant expressions in autograd and return 0 grad early.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
e975889132
|
Move cross_entropy calculation to a separate function.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
891abf5511
|
Make the autograd loss expression chain thread_local.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
8adf00ae6e
|
Identify a single evalation chain by ID in autograd to prevent cache reuse for subsequent evaluations of the same expression tree.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
cb812c742c
|
Add [[nodiscard]] attributes to autograd functions.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
26f19e1429
|
Make automatic differentiation node types constexpr.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
aec6017195
|
When forming an autograd expression only copy parts that are rvalue references, store references to lvalues.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
a5c20bee5b
|
Apply gradient clipping.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
d103867558
|
Add memoization to the autograd expression evaluator.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
aa55692b97
|
Cross entropy loss.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
539bd2d1c8
|
Replace the old loss/grad calculation completely.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
b71d1e8620
|
Pass the new loss function to update_parameters
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
5a58eb803a
|
Loss func with autograd
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
541fb8177a
|
More utility in autograd.
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
6ce0245787
|
Basic autograd
|
2020-12-02 08:56:20 +09:00 |
|
Tomasz Sobczyk
|
1322a9a5fd
|
Prevent false sharing of num_calls counter in the shared input trainer. Fix current_operation not being local to the executing thread.
|
2020-11-30 08:54:53 +09:00 |
|
Tomasz Sobczyk
|
2aa7f5290e
|
Fix comparison of integers with different signedness.
|
2020-11-30 08:54:53 +09:00 |
|
Tomasz Sobczyk
|
a97b65eaef
|
Fix compilation error with USE_BLAS
|
2020-11-30 08:54:53 +09:00 |
|
Tomasz Sobczyk
|
622e0b14c2
|
Remove superfluous example shuffling. Shuffling now only happens on reading.
|
2020-11-30 08:54:53 +09:00 |
|
Tomasz Sobczyk
|
34510dd08a
|
Remove used examples asyncronously.
|
2020-11-30 08:54:53 +09:00 |
|
Tomasz Sobczyk
|
0bee8fef64
|
Don't unnecessarily copy the batch part.
|
2020-11-30 08:54:53 +09:00 |
|
Tomasz Sobczyk
|
e954b14196
|
Prefetch weights for feature transformer backprop to shared cache.
|
2020-11-30 08:54:53 +09:00 |
|
Tomasz Sobczyk
|
8009973381
|
Special case for alpha=1 in saxpy, slight performance increase.
|
2020-11-30 08:54:53 +09:00 |
|
Tomasz Sobczyk
|
49b2dcb1f3
|
Preallocate memory for unique_features. Keep the training_features temporary buffer as a thread_local so we reuse the storage.
|
2020-11-30 08:54:53 +09:00 |
|
Tomasz Sobczyk
|
1c8495b54b
|
Remove handwritten saxpy because compilers optimize the second look anyway.
|
2020-11-30 08:54:53 +09:00 |
|
Tomasz Sobczyk
|
15c528ca7b
|
Prepare feature transformer learner.
|
2020-11-30 08:54:53 +09:00 |
|
Tomasz Sobczyk
|
a3c78691a2
|
Prepare input slice trainer.
|
2020-11-30 08:54:53 +09:00 |
|
Tomasz Sobczyk
|
401fc0fbab
|
Prepare clipped relu trainer.
|
2020-11-30 08:54:53 +09:00 |
|
Tomasz Sobczyk
|
774b023641
|
Add chunked for each with workers.
|
2020-11-30 08:54:53 +09:00 |
|
Tomasz Sobczyk
|
cc11375f6d
|
Skeleton for new evaluate learner
|
2020-11-30 08:54:53 +09:00 |
|
Tomasz Sobczyk
|
0d4b803b08
|
Prepare trainer affine transform.
|
2020-11-30 08:54:53 +09:00 |
|
Tomasz Sobczyk
|
4ea8572b6d
|
Add single threaded sgemm.
|
2020-11-30 08:54:53 +09:00 |
|
nodchip
|
ef4fdb40f9
|
Merge pull request #254 from noobpwnftw/merge
Merge
|
2020-11-28 09:03:38 +09:00 |
|
noobpwnftw
|
0b2ae6cb64
|
Merge remote-tracking branch 'remotes/official/master' into merge
|
2020-11-28 06:47:04 +08:00 |
|
Tomasz Sobczyk
|
92b14a5ba2
|
Add docs for transform.
|
2020-11-27 09:16:22 +09:00 |
|
Tomasz Sobczyk
|
89294e2e4f
|
Add transform command. Add transform nudged_static subcommand.
|
2020-11-27 09:16:22 +09:00 |
|
Vizvezdenec
|
190dd26b9f
|
use classical for certain endgames.
STC https://tests.stockfishchess.org/tests/view/5fbc64c067cbf42301d6b1d6
LLR: 2.97 (-2.94,2.94) {-0.25,1.25}
Total: 53360 W: 5223 L: 5024 D: 43113
Ptnml(0-2): 184, 3877, 18390, 4014, 215
LTC https://tests.stockfishchess.org/tests/view/5fbc97f267cbf42301d6b1ee
LLR: 2.96 (-2.94,2.94) {0.25,1.25}
Total: 126472 W: 5111 L: 4766 D: 116595
Ptnml(0-2): 50, 4032, 54749, 4333, 72
closes https://github.com/official-stockfish/Stockfish/pull/3240
bench: 3820648
|
2020-11-26 08:20:06 +01:00 |
|