Tomasz Sobczyk
a7378f3249
Make next_fen in opening_book a critical section
2020-12-14 09:03:04 +09:00
Joost VandeVondele
b49fd3ab30
Add -lstdc++fs to the link line of gcc
...
older versions of gcc (<8.1) need this, even if they accept -std=c++17
with this patch, the code can be run on fishtest again,
at least by the majority of workers (fishtest doesn't require c++17 to be available)
See e.g.
https://tests.stockfishchess.org/tests/view/5fcfbf801ac1691201888235
Bench: 3820648
2020-12-09 08:40:34 +09:00
Tomasz Sobczyk
28d6d7cb03
Avoid computing gradient for validation loss.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
fafb9557a8
Get train loss from update_parameters.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
4eb0e77a2a
Store references instead of copying the results of intermediate autograd computations.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
6cd0b03098
Add some comments regarding the current state of autograd loss computation.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
99cb869db3
Reintroduce use_wdl.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
cf6bc7ecaf
Cleanup around get_loss
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
256c4b55ec
Properly apply gradient norm clipping after it's scaled in the update_parameters.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
de675e3503
Reintroduce optional scaling of the teacher signal.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
01ae7b1e2c
Simplify passing constants that may vary between calls.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
cbd973fdaa
Detect constant expressions in autograd and return 0 grad early.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
e975889132
Move cross_entropy calculation to a separate function.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
891abf5511
Make the autograd loss expression chain thread_local.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
8adf00ae6e
Identify a single evalation chain by ID in autograd to prevent cache reuse for subsequent evaluations of the same expression tree.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
cb812c742c
Add [[nodiscard]] attributes to autograd functions.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
26f19e1429
Make automatic differentiation node types constexpr.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
aec6017195
When forming an autograd expression only copy parts that are rvalue references, store references to lvalues.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
a5c20bee5b
Apply gradient clipping.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
d103867558
Add memoization to the autograd expression evaluator.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
aa55692b97
Cross entropy loss.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
539bd2d1c8
Replace the old loss/grad calculation completely.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
5a58eb803a
Loss func with autograd
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
541fb8177a
More utility in autograd.
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
6ce0245787
Basic autograd
2020-12-02 08:56:20 +09:00
Tomasz Sobczyk
89294e2e4f
Add transform command. Add transform nudged_static subcommand.
2020-11-27 09:16:22 +09:00
Tomasz Sobczyk
9030020a85
Add smart_fen_skipping option to learn.
2020-11-23 19:22:11 +09:00
Tomasz Sobczyk
ee13cfce67
Fix result assigned for a psvector when the positions are not continuous.
2020-11-23 08:32:08 +09:00
Tomasz Sobczyk
3cee6881ee
Move the terminal position check to after qsearch, otherwise qsearch may end up in a terminal position.
2020-11-23 08:29:38 +09:00
Tomasz Sobczyk
e1dbad47ce
Add support for opening book to gensfen.
2020-11-17 09:43:23 +09:00
Tomasz Sobczyk
d4350a16f3
Add representation of an opening book.
2020-11-17 09:43:23 +09:00
Tomasz Sobczyk
3dbc45bdfc
Add gradient clipping.
2020-11-16 10:08:56 +09:00
Tomasz Sobczyk
00bc80c3c4
Add assume_quiet option to the learner.
2020-11-15 22:18:13 +09:00
Tomasz Sobczyk
00797a3d86
add option ensure_quiet for gensfen that makes the generated position quiet
2020-11-15 22:18:13 +09:00
Tomasz Sobczyk
69bc3ef9be
Output loss more often.
2020-11-14 12:33:25 +09:00
Tomasz Sobczyk
5d88e7bce8
Add optional move validation to training data conversion. No longer rely on static initialization order for magics initialization.
2020-11-10 10:21:09 +09:00
Tomasz Sobczyk
ee0917a345
Pass ThreadPool to update_parameters, propagate, and backpropagate.
2020-10-29 09:21:19 +09:00
Tomasz Sobczyk
317fda2516
Cleanup eval saving and lr scheduling.
2020-10-28 23:08:05 +09:00
Tomasz Sobczyk
f81fa3d712
Replace global_learning_rate with learning_rate local to the learner and passed to update_parameters as a parameter.
2020-10-28 09:36:07 +09:00
Tomasz Sobczyk
cde6ec2bf2
Make all grad related functions in learn static. Pass calc_grad as a parameter.
2020-10-27 14:47:50 +09:00
Tomasz Sobczyk
e4868cb59e
Move setting learn search limits to learner.
2020-10-27 14:47:07 +09:00
Tomasz Sobczyk
c229929d26
Remove the position parameter from learn.
2020-10-27 00:35:43 +09:00
Tomasz Sobczyk
a8066cd4a9
Rename elmo lambdas
2020-10-27 00:33:58 +09:00
Tomasz Sobczyk
f7de49eb66
Create a collective parameter struct for learner.
2020-10-27 00:33:58 +09:00
Tomasz Sobczyk
e01397c674
Remove multi_think
2020-10-26 19:40:40 +09:00
Tomasz Sobczyk
e515f1f61f
Move SfenWriter to a separate file
2020-10-26 19:39:58 +09:00
Tomasz Sobczyk
03abfae41f
Reorder members, renaming.
2020-10-26 09:37:59 +09:00
Tomasz Sobczyk
6d4d20c4be
Cleaner printing and some renaming.
2020-10-26 09:37:59 +09:00
Tomasz Sobczyk
d77b3d176e
Always flush sfen writer at the end of gensfen and when it is destroyed.
2020-10-26 09:37:59 +09:00
Tomasz Sobczyk
21fac7c53c
A collective struct for gensfen parameters.
2020-10-26 09:37:59 +09:00
