219 Commits

Author	SHA1	Message	Date
Tomasz Sobczyk	99cb869db3	Reintroduce use_wdl.	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	cf6bc7ecaf	Cleanup around get_loss	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	256c4b55ec	Properly apply gradient norm clipping after it's scaled in the update_parameters.	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	de675e3503	Reintroduce optional scaling of the teacher signal.	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	01ae7b1e2c	Simplify passing constants that may vary between calls.	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	cbd973fdaa	Detect constant expressions in autograd and return 0 grad early.	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	e975889132	Move cross_entropy calculation to a separate function.	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	891abf5511	Make the autograd loss expression chain thread_local.	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	8adf00ae6e	Identify a single evalation chain by ID in autograd to prevent cache reuse for subsequent evaluations of the same expression tree.	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	cb812c742c	Add [[nodiscard]] attributes to autograd functions.	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	26f19e1429	Make automatic differentiation node types constexpr.	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	aec6017195	When forming an autograd expression only copy parts that are rvalue references, store references to lvalues.	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	a5c20bee5b	Apply gradient clipping.	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	d103867558	Add memoization to the autograd expression evaluator.	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	aa55692b97	Cross entropy loss.	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	539bd2d1c8	Replace the old loss/grad calculation completely.	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	5a58eb803a	Loss func with autograd	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	541fb8177a	More utility in autograd.	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	6ce0245787	Basic autograd	2020-12-02 08:56:20 +09:00
Tomasz Sobczyk	89294e2e4f	Add transform command. Add transform nudged_static subcommand.	2020-11-27 09:16:22 +09:00
Tomasz Sobczyk	9030020a85	Add smart_fen_skipping option to learn.	2020-11-23 19:22:11 +09:00
Tomasz Sobczyk	ee13cfce67	Fix result assigned for a psvector when the positions are not continuous.	2020-11-23 08:32:08 +09:00
Tomasz Sobczyk	3cee6881ee	Move the terminal position check to after qsearch, otherwise qsearch may end up in a terminal position.	2020-11-23 08:29:38 +09:00
Tomasz Sobczyk	e1dbad47ce	Add support for opening book to gensfen.	2020-11-17 09:43:23 +09:00
Tomasz Sobczyk	d4350a16f3	Add representation of an opening book.	2020-11-17 09:43:23 +09:00
Tomasz Sobczyk	3dbc45bdfc	Add gradient clipping.	2020-11-16 10:08:56 +09:00
Tomasz Sobczyk	00bc80c3c4	Add assume_quiet option to the learner.	2020-11-15 22:18:13 +09:00
Tomasz Sobczyk	00797a3d86	add option ensure_quiet for gensfen that makes the generated position quiet	2020-11-15 22:18:13 +09:00
Tomasz Sobczyk	69bc3ef9be	Output loss more often.	2020-11-14 12:33:25 +09:00
Tomasz Sobczyk	5d88e7bce8	Add optional move validation to training data conversion. No longer rely on static initialization order for magics initialization.	2020-11-10 10:21:09 +09:00
Tomasz Sobczyk	ee0917a345	Pass ThreadPool to update_parameters, propagate, and backpropagate.	2020-10-29 09:21:19 +09:00
Tomasz Sobczyk	317fda2516	Cleanup eval saving and lr scheduling.	2020-10-28 23:08:05 +09:00
Tomasz Sobczyk	f81fa3d712	Replace global_learning_rate with learning_rate local to the learner and passed to update_parameters as a parameter.	2020-10-28 09:36:07 +09:00
Tomasz Sobczyk	cde6ec2bf2	Make all grad related functions in learn static. Pass calc_grad as a parameter.	2020-10-27 14:47:50 +09:00
Tomasz Sobczyk	e4868cb59e	Move setting learn search limits to learner.	2020-10-27 14:47:07 +09:00
Tomasz Sobczyk	c229929d26	Remove the position parameter from learn.	2020-10-27 00:35:43 +09:00
Tomasz Sobczyk	a8066cd4a9	Rename elmo lambdas	2020-10-27 00:33:58 +09:00
Tomasz Sobczyk	f7de49eb66	Create a collective parameter struct for learner.	2020-10-27 00:33:58 +09:00
Tomasz Sobczyk	e01397c674	Remove multi_think	2020-10-26 19:40:40 +09:00
Tomasz Sobczyk	e515f1f61f	Move SfenWriter to a separate file	2020-10-26 19:39:58 +09:00
Tomasz Sobczyk	03abfae41f	Reorder members, renaming.	2020-10-26 09:37:59 +09:00
Tomasz Sobczyk	6d4d20c4be	Cleaner printing and some renaming.	2020-10-26 09:37:59 +09:00
Tomasz Sobczyk	d77b3d176e	Always flush sfen writer at the end of gensfen and when it is destroyed.	2020-10-26 09:37:59 +09:00
Tomasz Sobczyk	21fac7c53c	A collective struct for gensfen parameters.	2020-10-26 09:37:59 +09:00
Tomasz Sobczyk	cb61dc9c9b	Make sfen writer a part of gensfen.	2020-10-26 09:37:59 +09:00
Tomasz Sobczyk	3f289546da	Make some gensfen members private.	2020-10-26 09:37:59 +09:00
Tomasz Sobczyk	821b655bc6	Move gensfen progress reporting from sfen writer to gensfen	2020-10-26 09:37:59 +09:00
Tomasz Sobczyk	af238fe132	Rewrite gensfen to use stockfish's thread pool.	2020-10-26 09:37:59 +09:00
Tomasz Sobczyk	2c477d76ec	Cleaner and more outputs during training initialization.	2020-10-25 22:18:28 +09:00
Tomasz Sobczyk	4b72658409	Synchronize printed info regions in the learner and sfen reader.	2020-10-25 22:18:28 +09:00