Identifying the style by a qualified reader on a short fragment of
  generated poetry
Boris Orekhov
Style is an important concept in today's challenges in natural language
generating. After the success in the field of image style transfer, the task of
text style transfer became actual and attractive. Researchers are also
interested in the tasks of style reproducing in generation of the poetic text.
Evaluation of style reproducing in natural poetry generation remains a problem.
I used 3 character-based LSTM-models to work with style reproducing assessment.
All three models were trained on the corpus of texts by famous Russian-speaking
poets. Samples were shown to the assessors and 4 answer options were offered,
the style of which poet this sample reproduces. In addition, the assessors were
asked how well they were familiar with the work of the poet they had named.
Students studying history of literature were the assessors, 94 answers were
received. It has appeared that accuracy of definition of style increases if the
assessor can quote the poet by heart. Each model showed at least 0.7
macro-average accuracy. The experiment showed that it is better to involve a
professional rather than a naive reader in the evaluation of style in the tasks
of poetry generation, while lstm models are good at reproducing the style of
Russian poets even on a limited training corpus.