Exploration of Sentiments and Genre in Spanish American Novels

Background, aims, and hypotheses
In 19th century Spanish American novels, the expression of emotionality is an essential characteristic of the texts belonging to different subgenres.
This contribution is concerned with the linguistic manifestation of emotions in literary texts on the textual surface. See Winko, 2003 for a discussion of how emotional meaning and literary texts are related. Especially during the Romantic period in the first half of the century, many sentimental novels have been written (Zó, 2015). But emotions also play an important role in other types of novels: a love story is often a basic plot element for example in historical or costumbrista novels. Also, there are novels characterized more by negative emotions, like Cuban anti-slavery novels (Rivas, 1990), Argentine anti-Rosas novels (Molina, 2011: 285-312, García Ardeo, 2006), or sociopolitical novels in general.

In text mining, a common method to analyze emotions is Sentiment Analysis (Pang and Lee, 2008). Sentiment Analysis is the computational treatment of sentiment, opinion, or emotion in text. Sentiments are usually modelled in terms of polarity values (positive, negative, neutral) or emotion values (such as trust, fear, joy, etc.).
The aim of this proposal is to test several hypotheses about sentiments in subgenres with an explorative analysis of a corpus of Spanish American novels. To this end, sentiment values are used as features in a text classification task. A secondary objective of this contribution is to compare the results of two different sentiment lexica for Spanish to see how well they perform.
The first hypothesis of this proposal is that the degree and kind of emotionality in the novels differs for different subgenres. The second hypothesis here is that not just emotions in general matter, but also whether they are expressed in the direct speech of the characters of the novels or in narrated text.
The anti-slavery novel, for example, has been defined in terms of its atmosphere of fear, but also by vigorous interferences of the narrator. Cf. Rivas, 1990.

State of the Art
Two recent examples for the usage of Sentiment Analysis with literary texts are Zehe et al., 2016 for the prediction of happy endings in German novels and Kim et al., 2017 for the analysis of prototypical emotion developments in literary genres with English texts. Sentiment Analysis has been used with Spanish texts, as well, mainly for the analysis of reviews and tweets (see Henríquez Miranda and Guzmán, 2017 for an overview). To the best of my knowledge, there are no applications of Sentiment Analysis on Spanish novels yet, and the distinction of direct speech and narrated text has not previously been used in combination with the analysis of sentiments in literary texts.

Data
For this analysis, a corpus of 30 Spanish American novels has been selected. The collection has the following characteristics: The novels have been published between 1840 and 1910 (13 before 1880 and 17 after 1880), are from three countries (Argentina: 16, Cuba: 9, Mexico: 5), and have been written by 16 different authors.
This is a subcollection of a larger corpus of Spanish American novels being prepared in the context of the junior research group
Computation Literary Genre Stylistics (CLiGS), see
.
Fig. 1 shows the distribution of novels per decade and subgenre:

Distribution of novels per decade and subgenre
As the texts at hand are not easily distinguishable genre fiction but more general literary fiction, the assignment of subgenre labels is a non-trivial task. For the assignment of subgenre labels to the novels, the subgenres as given in titles and subtitles of the novels were collected and subgenre assignments made in secondary literature were considered. Both types of information were used to derive four kinds of interpretive
Because the many variations found had to be normalized for this computational analysis, an interpretive step was unavoidable. subgenre labels corresponding to four broad types of novels: costumbrista (6 novels)
Novels of manners in the context of the Costumbrismo movement., historical (8), sentimental (9), and sociopolitical (7) novels.
The distribution of novels shows that there is a tendency for sentimental novels to belong to the first half and for non-sentimental novels to the second half of the century. This observation may be relevant for future tasks with a bigger corpus and interested in the development of genres over time. More detailed metadata for the novels can be found at
.

Methods
In general, Sentiment Analysis can be done with a machine learning approach and a lexicon-based approach. Here, two sentiment lexica were used: (1) SentiWordNet 3.0, an adaptation of WordNet 3.0 for sentiment analysis (Miller, 1995, Baccianella et al., 2010) and (2) the NRC Emotion Lexicon (Saif and Turney, 2013). The two lexica differ in how sentiments are modelled and also in their volume. SentiWordNet has polarity values (positivity, negativity, neutrality) for WordNet synsets which range between 0 and 1 and sum up to 1. The NRC lexicon, in contrast, has only binary values (0 or 1), but those are provided for positivity and negativity as well as eight basic emotions (Trust, Fear, Joy, Sadness, Anger, Disgust, Anticipation, Surprise). SentiWordNet contains 117,653 entries, the NRC lexicon just 14,182.
SentiWordNet can be used for Spanish because the synset IDs can be mapped to the Spanish version of WordNet. The NRC lexicon has been translated into Spanish automatically. See Baccianella et al., 2010 for evaluation reports for SentiWordNet. The authors of the NRC lexicon state that the translated versions may contain errors. An orthographic check on the NRC lexicon returned 409 entries that were not recognized as Spanish words. A further evaluation and improvement of the translated lexica is desirable.

In order to use the sentiment lexica, the texts had to be lemmatized (for NRC) and annotated with WordNet synsets (for SentiWordNet) which was done with the NLP library FreeLing (Padró and Stanislovsky, 2012). To be able to use the distinction between direct speech and narrated text as a feature, the texts were annotated semi-automatically in their TEI master files (see Fig. 2):

Example of a paragraph with annotated direct speech, from "Camila o la virtud triunfante" (1856) by Estanislao del Campo
Each paragraph was split into sentences. Each sentence was annotated with FreeLing and the words with sentiment values were determined using the lexicons. The sentiment values for the words were summed up for each sentence.
The Sentiment Analysis could be refined further by considering the sentence structure (and negation), which is a future task. For the eight basic emotions of the NRC (Trust, Fear, etc.), a sentence is assigned the emotion with a highest value in the sentence. Besides the sentiment features that come directly from the lexicons, the following features were determined for each sentence:

Additional features for the Sentiment Analysis

A Decision Tree classifier was used for the classification of the novels by subgenre, using the above-mentioned features (see Manning and Schütze, 1999: 578-589 on this method). The advantage of Decision Trees is that they can be interpreted. This is desirable in an explorative analysis interested in the kind of sentiment-based features that are relevant to differentiate novels of different subgenres. When compared to other types of classifiers, Decision Trees do not necessarily yield the best results in terms of accuracy, but their interpretability is valued higher here in order to gain insight into how sentiments, the opposition of direct speech vs. narrated text, and subgenres are related.

To generate data forthe machine learning task, the values of the single sentences were aggregated into five sections and divided by the section length (number of sentences contained in the section), resulting in150 data points for the 30 novels. 60 different experiments were run, varying the sentiment features and lexicon used, and the depth of the decision tree. A 5-fold cross-validation was applied.

Results and Discussion
The results of the experiments are summarized in Fig. 4 below. The depth of the Decision Tree was varied between 1 and 15.
Restricting the tree depth helps to prevent overfitting and usually leads to a better performance of the classifier on the test set. The accuracy is given as the mean F1 score obtained from the cross-validation. Four different sets of sentiment features were used: Features from the SentiWordNet lexicon (WN) and from the NRC lexicon (NRC), both without differentiating between direct speech and narrated text, as well as WN- and NRC-features with separate sentiment values for direct speech and narrated text (WN speech and NRC speech). The results of all experiments are compared to the “most frequent”-baseline and to a baseline obtained with an SVM classifier, using the 5,000 most frequent words.

Results for subgenre classification with sentiment features

Although the F1 scores are not very high (the highest mean value being at 0.52), almost all of them outperform the “most frequent”-baseline (0.3) which confirms that sentiment features are relevant for subgenre classification. Still, the results do not reach the best mean score of the MFW classification (0.57).
See Hettinger et al., 2016 for a discussion of various types of features (MFW, topics, networks) for subgenre classification, stating that genre classification in general works best with most frequent words, all words, and the like. In terms of classification accuracy, a next step will be to combine both sentiment features and MFW to see if the sentiment features can contribute to improve the overall results.

When comparing the results for the two different sentiment lexica, the NRC lexicon performs better than SentiWordNet, although the latter covers almost ten times as many words as the first one. A look into the feature importance shows that the eight basic emotions, which are only present in the NRC lexicon, are crucial (see Fig. 5 and 6).

Feature importance for a tree with depth 3, using NRC and speech vs. narrated text

Feature importance for a tree with depth 6, using NRC and speech vs. narrated text

Regarding the difference between speech and narration, Fig. 4 above also shows that the highest values for both WN and NRC are reached when the sentiment values are calculated separately for direct speech and narrated text. The best scores are obtained for the feature set “NRC speech”. The most important feature in both example trees is positive speech, followed by narrated fear. Fig. 7 shows the Decision Tree corresponding to the feature importance in Fig. 5 above.

A Decision Tree for the classification of subgenres, based on the best parameters

The tree shows that novels with higher values of positive speech are more likely to be sentimental novels. Other features that contribute to the distinction of sentimental novels are lower values of trust and higher values of anticipation in narrated text. The path for historical novels includes less positive speech and more fear and anger in narrated text. Costumbrista novels are characterized by less sadness in narrated text than sociopolitical novels and by more trust in narrated text than sentimental novels. Sociopolitical novels differ from historical novels in that they have a lower value of fear and anger in narrated text.
The results of all experiments can be found at
.

Conclusion and Future Work
This exploration of sentiments in Spanish American Novels showed that Sentiment Analysis can be used as a basis for subgenre classification tasks. It has been shown that the distinction between emotions in direct speech and emotions in narrated improves the classification results considerably. Regarding the two sentiment lexica that were tested, the NRC Emotion Lexicon performs better than SentiWordNet.
The Decision Trees resulting from the classification give much insight into how sentiments in general, in direct speech and in narrated text are related to different types of novels. That the features can be interpreted easily contributes to a better understanding of what textual features are connected to the subgenres, but the classification results themselves can still be improved. Other classifiers, for example Random Forest trees or an SVM, might yield better results but will also be less interpretable. Another important next step is to increase the corpus size to make the results more stable.

Bibliography

Baccianella, S., Esuli, A. and Sebastiani, F. (2010). SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining.
Proceedings of LREC 2010.Valletta, Malta:
ELRA: 2200-2204.
(accessed April 27 2018).

García Ardeo, J. M. (2006). Eduardo Gutiérrez y sus dramas del terror.
Letras
54: 77-94.

Henríquez Miranda, C. and Guzmán, J. (2017). A Review of Sentiment Analysis in Spanish. Una Revisión Sobre el Análisis de Sentimientos en Español.
TECCIENCIA
12 (22): 35-48. doi: 10.18180/tecciencia.2017.22.5.

Hettinger, L., Jannidis, F., Reger, I. and Hotho, A. (2016). Classification of Literary Subgenres.
DHd2016. Leipzig: Universität Leipzig: 154-158.
(accessed April 27 2018).

Kim, E., Padó, S. and Klinger, R. (2017). Prototypical Emotion Developments in Literary Genres.
Digital Humanities 2017. Conference Abstracts. Montréal: McGill University.
(accessed April 27 2018).

Manning, C. D. and Schütze, H. (1999).
Foundations of Statistical Natural Language Processing. Cambridge, Mass.: The MIT Press.

Miller, G. A. (1995). WordNet: A Lexical Database for English.
Communications of the ACM
38 (11), 39-41.

Molina, H. B. (2011).
Como crecen los hongos. La novela argentina entre 1838 y 1872. Buenos Aires: Teseo.

Padró, L. and Stanislovsky, E. (2012). FreeLing 3.0: Towards Wider Multilinguality.
Proceedings of the Language Resources and Evaluation Conference (LREC 2012). Istanbul, Turkey: ELRA: 2473-2479.
(accessed April 27 2018).

Pang, B. and Lee, L. (2008). Opinion Mining and Sentiment Analysis.
Foundations and Trends in Information Retrieval
2 (1-2): 1-135.

Rivas, M. (1990).
Literatura y esclavitud en la novela cubana del siglo IXI. Sevilla: Escuela de Estudios Hispano-Americanos.

Saif, M. and Turney, P. (2013). Crowdsourcing a Word-Emotion Association Lexicon.
Computational Intelligence
29 (3), 436-465.

Zehe, A., Becker, M., Hettinger, L., Hotho, A., Reger, I., and Jannidis, F. (2016): Prediction of Happy Endings in German Novels based on Sentiment Information.
Proceedings of DMNLP, Workshop at ECML/PKDD. Riva del Garda, Italy.
(accessed April 27 2018).

Zó, R. E. (2015).
Emociones escriturales. La novela sentimental latinoamericana. Saarbrücken: Editorial Académica Española.

Winko, S. (2003). Über Regeln emotionaler Bedeutung in und von literarischen Texten. In Jannidis, F., Lauer, G., Martínez, M., Winko, S. (eds.),
Regeln der Bedeutung. Berlin: de Gruyter, pp. 329-348.