The dominant paradigm in social listening tools is to rate all content with a positive, neutral, or negative value. We call this the Generic Sentiment Score. A major source of dissatisfaction among marketers and PR professionals is the overall accuracy generated by this score and the machines that try to approximate this value. This experiment proves that there are many versions of truth among humans when asked to score the same content. The authors conclude that the Generic Sentiment Score is not enough and that a more comprehensive approach is required.
Hypothesis 1: There is no unanimous agreement among all respondents on the sentiment score of a single
response to any question within the survey.
Hypothesis 2: No two completed surveys will contain identical responses. Every completed survey will be unique.
Hypothesis 3: The reported mood of the scorer and the Generic Sentiment Score are correlated such that negative moods generate negative scores.
Hypothesis 4: If the responder starts off rating the first five statements as negative, the subsequent responses will tend to be more negative than those who started off positive.
Hypothesis 5: The tendency to anchor and adjust causes a responder to score the same question stated later on in the questionnaire in the same session, differently.
The study may be downloaded here.
