I am using a Likert style questionnaire that measures a subject on 3 scales, producing a profile for each subject. However, I am having problems analysing the results due to the subjective nature of the responses.
The instrument analyses the subjects' cognitive style with a MCT that ask them to score themselves on a number of questions out of 5. Each question contributes to one of 3 profiles, which I will call A, B and C. They get a score for each of these profiles based on the average of the questions that belong to each category.
However, my problem is that because it is self reported and subjective, people are not consistent on how they report preferences. Some people avoid the extremes, so would rarely if ever use the 5 or the 1. Others see things as black and white and will avoid the 4 and 2 and use the 3 as a 'don't know'. So two people with a similar cognitive profile, both of whom are high A, average B and low C may score themselves differently as either 5, 3, 1 or 4, 3, 2. When we see the results, the first subject may end up with a profile of A=4.8, B=3.0, C=1.2 and the second A=3.8, B=3.0, C=2.2.
Looking at the scores, the first subject is apparently more A than the second subject, but this may not be the case at all. Is there any way of managing this to come up with adjusted scores so that their profiles become more similar, i.e. manage the pattern rather than the actual score, or am I just stuck with this as a limitation of Likert style surveys?
Please excuse me if I am not using the correct terminolgy; it is 30 odd years since I last studied maths formally and this is a new area for me.
The instrument analyses the subjects' cognitive style with a MCT that ask them to score themselves on a number of questions out of 5. Each question contributes to one of 3 profiles, which I will call A, B and C. They get a score for each of these profiles based on the average of the questions that belong to each category.
However, my problem is that because it is self reported and subjective, people are not consistent on how they report preferences. Some people avoid the extremes, so would rarely if ever use the 5 or the 1. Others see things as black and white and will avoid the 4 and 2 and use the 3 as a 'don't know'. So two people with a similar cognitive profile, both of whom are high A, average B and low C may score themselves differently as either 5, 3, 1 or 4, 3, 2. When we see the results, the first subject may end up with a profile of A=4.8, B=3.0, C=1.2 and the second A=3.8, B=3.0, C=2.2.
Looking at the scores, the first subject is apparently more A than the second subject, but this may not be the case at all. Is there any way of managing this to come up with adjusted scores so that their profiles become more similar, i.e. manage the pattern rather than the actual score, or am I just stuck with this as a limitation of Likert style surveys?
Please excuse me if I am not using the correct terminolgy; it is 30 odd years since I last studied maths formally and this is a new area for me.