Tolstoi Char RNN¶
-
class
deepobs.tensorflow.testproblems.tolstoi_char_rnn.
tolstoi_char_rnn
(batch_size, weight_decay=None)[source]¶ DeepOBS test problem class for a two-layer LSTM for character-level language modelling (Char RNN) on Tolstoi's War and Peace.
Some network characteristics:
128
hidden units per LSTM cell- sequence length
50
- cell state is automatically stored in variables between subsequent steps
- when the phase placeholder swithches its value from one step to the next, the cell state is set to its zero value (meaning that we set to zero state after each round of evaluation, it is therefore important to set the evaluation interval such that we evaluate after a full epoch.)
Working training parameters are:
- batch size
50
200
epochs- SGD with a learning rate of \(\approx 0.1\) works
Parameters: - batch_size (int) -- Batch size to use.
- weight_decay (float) -- No weight decay (L2-regularization) is used in this
test problem. Defaults to
None
and any input here is ignored.
-
dataset
¶ The DeepOBS data set class for Tolstoi.
-
train_init_op
¶ A tensorflow operation initializing the test problem for the training phase.
-
train_eval_init_op
¶ A tensorflow operation initializing the test problem for evaluating on training data.
-
test_init_op
¶ A tensorflow operation initializing the test problem for evaluating on test data.
-
losses
¶ A tf.Tensor of shape (batch_size, ) containing the per-example loss values.
-
regularizer
¶ A scalar tf.Tensor containing a regularization term.
-
accuracy
¶ A scalar tf.Tensor containing the mini-batch mean accuracy.