Tourism Management Journal Ratings
A Report of an Examination of the Perceptions of Journal Quality by Tourism Management Academics
Conducted by The Centre for Leisure Management Research, Deakin University
December 2006
1 Approach Identification of Journals Twenty-eight journals were evaluated in this study including journals directly servicing tourism management and those journals that published research relevant to tourism management. This list was determined through the input of academics at the institution undertaking the research. All journals included in the study published manuscripts according to the accepted peer review process. Population A sample of academics was framed by including all tourism management academics at the rank of professor and associate professor in Australia and New Zealand plus, using a judgement sample, determining a cohort of senior lecturers within Australia and New Zealand. In Europe, North America and Asia, a judgement sampling technique was also used to construct a list of academics at the rank of professor and associate professor. To be included in this sample, academics had to be dedicated to tourism management research, teaching and/or a managerial role responsible for tourism management programs, and have current or past editorial board experience in one or more of the journals in the field. Consequently, twenty academics were deemed to meet the study criteria. Instrument Design This study has been based on the weighting criteria developed by Polonsky and Whitelaw (2005). The four criteria used by Polonsky and Whitelaw include: • Prestige of Journal; • Contribution to Theory; • Contribution to Practice; and • Contribution to Teaching. Based on a seven point scale, which was represented by A+=1, A=2, B+=3, B=4, C+=5, C=6, D=7, respondents were asked to rate each of the four criterion for each journal. Prior to undertaking the ratings, respondents were asked to indicate their perceived importance of the four criteria, as a percentage. In other words, when assessing peer-reviewed journals respondents were asked which of the four criteria are deemed to be the most important. Respondents were asked to allocate a total of 100% amongst the four criterion. For example, respondent A might allocate Prestige 40%, Contribution to Theory, 30%, Contribution to Practice 15% and Contribution to Teaching 15%. The scores for each criterion would be used to determine an overall composite weighted score, by multiplying the weightings respondents had allocated to each criterion by their rating of each criterion for each journal. These scores were then averaged for all respondents to provide a composite weighted score for each journal. For example, a respondent might suggest Prestige is worth 40% in the evaluation of journals and they subsequently rate a journal as 2 (A) for Prestige. The weighted score for a journal in the area of Prestige would be 2 × 0.4 = .8, which would then be summed with the weighted score for the other three criteria to produce a weighted composite score for that journal.
2 Procedures As previously described, unweighted and weighted composite scores were calculated for each journal allowing for a comparison of the impact of the weightings. Individual unweighted scores for each of the four criteria were calculated for each journal. A cluster analysis was performed using the four individual criteria weighted mean scores for each journal. Cluster analysis was employed to ascertain groupings among the journals to assist with the allocation of a rating for each journal. A K-means cluster analysis was used to identify homogenous groups within the journals. This was also the type of cluster analysis used by Polonsky and Whitelaw (2007, in Press). The procedure required the number of clusters to be specified at the outset, and a threecluster solution was adopted based on the composite weighted scores, the initial indications of possible groupings, and the match between cluster and grade boundaries.
Results Respondents and Familiarity Twenty respondents listed their primary expertise to be tourism management. Respondents consisted of 14 males (70%) and 6 females (30%) and were from three geographic regions, Australia/New Zealand 13 (65%), USA/Canada 3 (15%) and United Kingdom/Europe 4 (20%). Thirteen respondents (65%) described their academic position as Dean, Department Chair or Professor, 4 (20%) as associate professor, 1 (5%) as senior lecturer, 1 (5%) as lecturer, and 1 (5%) in administration. The familiarity index shown in Table 1 denotes the number of respondents to rate each journal and was determined by dividing the number of respondents to rate each journal by the total number of respondents. The second column of Table 1 lists the familiarity score, expressed as a decimal, and the rank (i.e., 1 being the most familiar to 28 being the least familiar) for each journal. The most familiar journal was the Annals of Tourism Research (0.85) followed by Tourism Management (0.75) and the least familiar were the Journal of Hospitality and Tourism Management and Tourism and Hospitality Research (0.40) Individual Criteria Results The next phase of the analysis was to examine the mean unweighted and weighted scores for the four evaluative criteria. Similar to the techniques used by Polonsky and Whitelaw (2007, In Press) the maximum score on any criterion was 1 (A+) and the minimum score was 7 (D), although Polonsky and Whitelaw used an increasing 7point scale (a score of 1 was considered a D, and 7 an A+). The mean unweighted score for the 28 journals for Prestige was 3.65, with a range of 2.24 to 4.78; Contribution to Theory was 3.75, with a range of 1.65 to 5.11; Contribution to Practice, was 3.96, with a range of 2.45 to 5.30, and Contribution to Teaching, was 3.94, with a range of 2.73 to 5.10. Extensive variation is noted across the four areas and the 28 journals. Composite Criteria Results The focus of this study, and the method used to determine journal ratings was based on a weighting of the four criteria as developed by Polonsky and Whitelaw (2005). Consequently, respondents were asked to indicate their perceived importance of the
3 four criteria, as a percentage, which was used to determine the weighted score, by multiplying the weightings respondents allocated to each criterion by their rating of each criterion for each journal. These scores were then averaged for all respondents for each journal to provide a composite weighted score. The average weight allocated to each of the four individual criteria by the respondents was: Prestige, 41.00% (SD = 10.46); Contribution to Theory, 21.25% (SD = 8.56); Contribution to Practice, 20.50% (SD = 7.05); and Contribution to Teaching, 17.25% (SD = 10.32). The overall mean weighted score for the fourteen journals was 3.78 (B+), with scores ranging from 2.39 (A) to 4.95 (B). Composite scores for the five A rated journals ranged from 2.39 to 3.20, for the twelve B+ rated journals 3.16 to 4.22, and for the eleven B rated journals 4.00 – 4.95. Cluster Analysis The final stage of the analysis was to undertake a cluster analysis of the 28 journals using the four individual criteria mean scores for each journal. A three-cluster solution was chosen. The three-cluster solution supported final ratings categories of A, B+, and B. As seen in Table 1, clustering according to the weighted scores produced clusters whose membership could be accurately predicted by the composite weighted journal score. However, in the B+ group there were two journals that scored within the A and B group, but were considered to be B+ journals by the cluster analysis. References Polonsky, M. J. & Whitelaw, P. (2005). What are we measuring when we evaluate journals? Journal of Marketing Education, 27, 189-201. Polonsky, M.J. & Whitelaw, P. (2007 in press). A Multi-dimensional examination of marketing journal rankings by North American academics. Marketing Education Review,
4 Table 1: Criteria Scores, Ratings and Cluster Categories Familiarit y Weight (Rank) 0.7 (4) 0.85 (1) 0.55 (14) 0.55 (14) 0.6 (8) 0.55 (14) 0.5 (21) 0.55 (14) 0.45 (25) 0.6 (8) 0.4 (27) 0.45 (25) 0.75 (2) 0.5 (21) 0.7 (4) 0.65 (6) 0.6 (8) 0.6 (8) 0.6 (8) 0.55 (14) 0.6 (8) 0.55 (14) 0.4 (27) 0.55 (14) 0.75 (2) 0.65 (6) 0.5 (21) 0.5 (21) Weighte d Score (Rank) 3.55 (10) 2.80 (2) 4.05 (18) 3.36 (8) 3.27 (7) 4.00 (17) 3.63 (14) 3.55 (10) 4.22 (21) 4.20 (19) 4.43 (24) 4.82 (27) 3.20 (6) 4.53 (26) 3.16 (5) 3.63 (13) 3.04 (3) 4.47 (25) 3.56 (12) 3.69 (15) 3.50 (9) 4.20 (19) 4.39 (23) 3.11 (4) 2.39 (1) 3.85 (16) 4.25 (22) 4.95 (28) Weighted Criteria – 3 Clusters B+ A B B+ B+ B B+ B+ B+ B B B A B B+ B+ A B B+ B+ B+ B B A A B+ B B
Journal Annals of Leisure Research Annals of Tourism Research Asia Pacific Journal of Tourism Research Cornell Hotel and Restaurant Administration Quarterly Current Issues in Tourism International Journal of Event Management Research International Journal of Hospitality Management International Journal of Tourism Research Journal of Canadian Association of Leisure Studies Journal of Hospitality and Leisure Marketing Journal of Hospitality and Tourism Management Journal of Hospitality, Leisure, Sport and Tourism Education Journal of Leisure Research Journal of Sport Tourism Journal of Sustainable Tourism Journal of Travel & Tourism Marketing Journal of Travel and Tourism Research Journal of Vacation Marketing Leisure Sciences Leisure Studies The Journal of the Leisure Studies Association Tourism Analysis Tourism and Hospitality Research Tourism Economics: The Business and Finance of Tourism and Recreation Tourism Management Tourism Recreation Research Tourism Review International Tourism, Culture & Communication