Document Sample

Movie Advisor Tel Aviv University Faculty of Engineering M.Sc. Project Doron Harlev Supervisor: Dr. Dana Ron March 7, 2010 Movie Advisor Project 2/51 March 7, 2010 Table of Contents Introduction ................................................................................................................................ 3 Database Characteristics ............................................................................................................ 4 Data Sets .................................................................................................................................... 7 Evaluation Criteria ..................................................................................................................... 8 Mean Average Error (MAE) .................................................................................................. 8 Coverage ................................................................................................................................ 8 Base Algorithms......................................................................................................................... 9 All Average ............................................................................................................................ 9 Movie Average....................................................................................................................... 9 User Average ......................................................................................................................... 9 User Movie Average .............................................................................................................. 9 Summary .............................................................................................................................. 10 Prediction Methods .................................................................................................................. 11 Pearson R Algorithm............................................................................................................ 11 Base Algorithm ................................................................................................................ 11 Additions To The Base Pearson R Algorithm ................................................................. 13 Mean Square Difference (MSD) Algorithm ........................................................................ 17 Genre Based Algorithms ...................................................................................................... 19 Genre Statistics ................................................................................................................ 20 Base Algorithm ................................................................................................................ 20 Genre Algorithm .............................................................................................................. 21 Hybrid Genre Algorithm .................................................................................................. 23 Algorithm Comparison ........................................................................................................ 25 MAE Vs. Coverage For All Algorithms .......................................................................... 25 Database Alteration .......................................................................................................... 26 Conclusions .............................................................................................................................. 31 Reference ................................................................................................................................. 32 Appendix I: Implementation .................................................................................................... 33 Introduction .......................................................................................................................... 33 Matlab Code ......................................................................................................................... 34 General ............................................................................................................................. 34 Database Statistics ........................................................................................................... 36 Base Algorithms............................................................................................................... 37 Pearson r Algorithm ......................................................................................................... 38 MSD ................................................................................................................................. 41 Genre ................................................................................................................................ 43 Movie Advisor Project 3/51 March 7, 2010 Introduction This paper will analyze existing and proposed algorithms designed to provide predictions of a users‟ rating of a particular movie. The predictions will rely on the users‟ own rating of other movies as well as the scores provided by neighboring users. The ability to predict a users‟ rating may prove useful in many contexts. One such context may be enhancing a users‟ experience in an online store by recommending some items while helping avoid others. The problem set described in this paper is studied in the field of Recommender Systems or Collaborative Filtering. This paper will start by examining the statistical properties of the chosen database. Once the base and test data sets have been characterized, evaluation criteria such as Mean Average Error (MAE) and coverage will be explained. These criteria will then be used to measure the performance of trivial prediction methods applied to the base and test data sets. The performance of trivial methods will provide a reference for more sophisticated algorithms. The first of these algorithms makes use of the Pearson R coefficient. The Pearson R coefficient provides a correlation measure between two vectors and can be used to provide a distance metric between two users. Use of the Pearson R correlation coefficient is quite common in the field of collaborative filtering, and results obtained with this method will be used to gauge the performance of other algorithms. The Pearson R algorithm will be further enhanced in accordance with improvements proposed by other papers as well this one. Another baseline algorithm to be examined is the Mean Square Difference (MSD) algorithm. Although generally considered inferior to the Pearson R algorithm, elements in the MSD algorithm will prove valuable in newly proposed algorithms. The database used in this paper also provides genre information for all movies. The statistics of genre information in the database will be analyzed. Novel prediction algorithms relying on genre information will then be proposed. As a reference, a new trivial prediction algorithm will be presented and its performance compared to previous trivial methods. Several genre-based prediction algorithms will then be proposed and analyzed with respect to the test and base data sets. Finally, all algorithms will be compared. This comparison will include a different instantiation of the initial data set, as well as altered versions of the entire database. Movie Advisor Project 4/51 March 7, 2010 Database Characteristics The database used for the project is the GroupLens database available on the internet at http://www.cs.umn.edu/Research/GroupLens/. The database contains 100,000 ratings on a scale of 1-5. The ratings are made for 1682 movies by 943 users. If the database were to be viewed as a matrix with users designating rows and movies designating columns, the matrix would be extremely sparse with values in merely 6% of its entries. Despite its apparent sparseness, the database is sufficient in allowing the analysis of the prediction algorithms discussed in this paper. The mean score provided for all entries in the database is 3.53, the median is 41 and the standard deviation is 1.13. Figure 1 depicts a histogram of all user ratings: 4 x 10 3.5 3 2.5 2 1.5 1 0.5 0 1 2 3 4 5 Score Figure 1: Historgram of all movie scores 1 The median is actually 4-, that is to say, a score of 4 is included in the upper half of the median. Movie Advisor Project 5/51 March 7, 2010 From the histogram and average score we learn that users tend to rate movies they liked more than movies they disliked. The average number of scores given to each movie is around 60 and the median number of scores is 57. Figure 2 depicts a histogram of the number of scores given to each movie: 700 600 500 400 300 200 100 0 0 100 200 300 400 500 600 Number of Scores for Movie Figure 2: Histogram of number of scores given to each movie The histogram shows that while some movies have a significant number of ratings of more than 500, many others have 10 ratings or less. The average number of scores given by each user is 106 and the median is 65. The minimum number of scores given by each user is 20. Figure 3 depicts a histogram of the number of scores given by users: 300 250 200 150 100 50 0 0 100 200 300 400 500 600 700 800 Number of Scores Per User Figure 3: Histogram of number of scores given by each user Movie Advisor Project 6/51 March 7, 2010 This histogram shows a distribution similar to the ratings of movies. While some users rated as many as 700 hundred movies, many others rated the minimum allowed. Figure 4 depicts a graphical representation of the database: Figure 4: Graphical representation of rated movies Each rated item in the database is designated by a blue dot. The parabolic decline of rated movies as a function of user ID implies that when the database was established, each new user was provided with a more elaborate list of movies. It is also evident that movies with higher ID‟s average less ratings than those with lower ID‟s. Movie Advisor Project 7/51 March 7, 2010 Data Sets To assess the prediction methods, the data set is divided into two parts: base and test. The test data set contains 5 entries for 10% of the users yielding a total of 470 entries. The base data set contains all remaining entries (99,530). The test data set is used as a reference for the accuracy of the predictions. Since the division into base and test data sets is made randomly, the performance of the tested algorithms will vary depending on the specific division that was made. To stabilize the performance throughout the paper, all results are calculated for a specific base/test division. This issue will be further addressed in the final analysis. Movie Advisor Project 8/51 March 7, 2010 Evaluation Criteria Two types of evaluation criteria will be used in this paper. Mean Average Error (MAE) Once predictions for the test data set are obtained, Ei is defined as the difference between 1 n prediction, Si, and the actual score given by the user, Ri. The MAE is MAE Si Ri , n i 1 where n is the length of the predicted data set. Other error coefficients such as standard deviation and ROC may be used. However, related papers show that MAE is consistent with other measures and proves to be the most widely used. Coverage As will later be shown, predictions cannot be made for all movies in the test data set. Coverage is a measure of the percentage of movies in the test data set that can be predicted. In general, the smaller the MAE the smaller the coverage becomes. The importance of coverage depends on the application using the recommender system. Some applications may require predictions for most items in the database, while others may choose to compromise coverage for improved accuracy. Movie Advisor Project 9/51 March 7, 2010 Base Algorithms Four base algorithms are analyzed in this section. These algorithms are “basic” in that they rely on trivial aspects of the user‟s entries to provide predictions. All Average In this method the average rating of all entries in the base data set is calculated. This calculated value is then used as the predicted score for all values in the test data set. This method yields an MAE of 1.046 and coverage of 100%. Movie Average In this method the average score given to each movie in the base data set is calculated. This average score is then used to predict values for all occurrences of the movie in the test data set. This method yields a vast improvement over the previous one. The MAE drops to 0.887. Since not all movies in the base data set contain scores, some values in the test dataset cannot be predicted. The coverage in this method is 99.8%. User Average In this method the average score given by each user in the base dataset is calculated. This average score is then used to predict values for the user in the test data set. This method yields an improvement over the previous one. The MAE drops to 0.849. The coverage in this method is 100% since all users made at least 20 predictions. Figure 5 depicts a histogram of the error using the user average base algorithm. 60 50 40 30 20 10 0 -3 -2 -1 0 1 2 3 Error Figure 5: Histogram of error using User Movie Average prediction User Movie Average This method is the most sophisticated of the basic methods. The method combines user average and movie average to produce a prediction. In order to produce a prediction for user a and movie i we perform the following calculation: S a ,i ra ru ,i ru 1 n u Movie Advisor Project 10/51 March 7, 2010 where ra is user a‟s average rating, u is an index running through all users who rated and the movie, and n is the number of users who rated the movie. This method yields yet another improvement over the previous one. The MAE drops to 0.83 while the coverage is 99.8%, identical to method 2. Summary It is clear that any desirable prediction method must improve upon the results obtained in this section. Table 1 summarizes the results: Method MAE Coverage [%] All Average 1.046 100 Movie Average 0.887 99.8 User Average 0.849 100 User Movie Average 0.830 99.8 Table 1: Summary of base algorithms results Movie Advisor Project 11/51 March 7, 2010 Prediction Methods The following prediction methods attempt to provide further improvement over the base algorithms. The improvement is possible through the use of additional information. One form of additional information is to intelligently weigh other users‟ ratings. The Pearson R and MSD methods use this approach to improve prediction. Another form of additional information is to make use of genre information provided with the database. Methods using this approach will be presented in the ensuing discussion. Pearson R Algorithm The Pearson R algorithm relies on the Pearson R coefficient to produce a correlation metric between users. This correlation is then used to weigh the score of each relevant user. The Pearson R algorithm is used widely in the study of recommender systems, and is used as a reference in this paper. Base Algorithm The Pearson R correlation between users a and u is defined as: r ra ru ,i ru m a ,i Pa ,u i 1 a u where m is the number of movies that both users rated, ra ,i is the score user a gave movie i, and ra is the average score user a gave all movies. Since the base data set is sparse, when calculating the Pearson R coefficient, most users have only a few overlapping scored movies. In calculating the coefficient, only movies ranked by both users are taken into account. This affects the sum in the numerator, and the variance in the denominator. Figure 6 depicts a histogram of the average number of mutually rated movies (MRM) for all users in the test data set: 40 35 30 25 20 15 10 5 0 0 10 20 30 40 50 60 70 Average Number of Mutually Rated Movies Figure 6: Histogram of average number of MRM for all users in the test data set Movie Advisor Project 12/51 March 7, 2010 The mean MRM is 17.6 and the median is 12. We note that many users have an average MRM below 10, while some have a relatively high average MRM of 60. Users who provide more entries will tend to have a higher average MRM. Figure 7 depicts a histogram of the Pearson R coefficient between a particular user and all other users: 150 100 50 0 -1 -0.5 0 0.5 1 Pearson R Coefficient for user 104 Figure 7: Historgram of Pearson R coefficient It is evident that most users are slightly positively correlated. While there is such a thing as two people with similar tastes, it is highly unlikely to find two users with opposite tastes. Opposite tastes would require users to consistently dislike what the other user likes and vice versa. This observation is further discussed at Shardanand [1]. Once the Pearson R correlation between a user and all other users is obtained, the predicted movie score is calculated as: r ru Pa ,u n u ,i S a ,i ra u 1 n P u 1 a ,u This approach is very similar to that of the user movie average discussed earlier. The difference is that the Pearson R coefficient is used to weigh the score given by each movie, whereas in the user movie average algorithm all users are weighted equally. Applying the Pearson R base algorithm to the test data set yields a MAE of 0.79 with coverage of 99.8%. The coverage is slightly limited, because not all movies in the test data set have a score in the base data set. Since no limitations have been made on the predictions, the coverage is identical to the movie average and user movie average base algorithms. Movie Advisor Project 13/51 March 7, 2010 Figure 8 depicts a histogram of the prediction error using the Pearson R base algorithm. 120 100 80 60 40 20 0 -3 -2 -1 0 1 2 3 Error Figure 8: Histogram of prediction error for Pearson R base algorithm The MAE obtained is clearly an improvement over all base methods. Note that in order to normalize the weights, all user scores are divided by a sum of the absolute value of the correlations. Since the Pearson R correlation may be either positive or negative, adding all correlations may produce low values in the denominator. These low values may cause the prediction to be either too low or too high, at times exceeding a value of 5. Using the absolute value has a drawback in that it produces an unbalanced normalization factor. Results obtained without the absolute value were extensively tested and shown to produce substantially poorer results. In further discussion, only the absolute value approach will be applied. Additions To The Base Pearson R Algorithm Several modifications can be made to the base Pearson R algorithm to improve its performance. These modifications have to do with setting thresholds in order to reduce the effects of „noisy‟ data. Pearson R Thresholding One modification, initially suggested by Shardanand [1], is to avoid making use of users who are not highly correlated. To implement this approach we modify Pa ,u as follows: Pa ,u L 1 L if Pa ,u L P L Pa ,u a ,u | if Pa ,u L 1 L 0 otherwise where L is the Pearson R threshold. After calculating Pa| ,u we simply replace it with Pa ,u and produce the expected score. Movie Advisor Project 14/51 March 7, 2010 Figure 9 shows the effect of increasing the Pearson R threshold on the MAE and coverage: Users TH=3, Herlock TH=1 0.95 Pearson R Algorithm 0.9 User Average MAE 0.85 0.8 0.75 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 Pearson R Threshold 1 Coverage 0.9 0.8 0.7 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 Pearson R Threshold Figure 9: MAE and Coverage as a function of Pearson R threshold The results were obtained with two additional thresholds, users and Herlock. The users threshold is the number of correlated users required to make a prediction. When the threshold is not met, the algorithm will not make a prediction for the item in the test data set. For consistency, all methods in this paper will be examined with a threshold of 3 users. A Herlock threshold of 1 is identical to no threshold at all. This threshold will be explained later. The Pearson R threshold initially reduces the MAE (MAE=0.7867 at P TH=0.1) slightly without compromising coverage significantly. Later on, the Pearson R threshold has the effect of increasing the MAE and decreasing coverage – both effects undesirable. The decreasing coverage is caused by an increased amount of movies that cannot be predicted due to an insufficient number of correlated users. The user average plot, present in the upper chart, is the MAE of the user average algorithm calculated on the predicted test data set. The test data set experiences changes as the coverage changes. These changes cause slight variations in the results user average method. The plot of user average is adds information about the type of movies being omitted with decreasing coverage. This measure will prove more significant in later discussion. Significance Weighting Herlocker et. al. [2] suggested that it may be advisable to make use of the number of mutually rated movies (MRM). Recall Figure 6, which shows a histogram of the average MRM for all users in the test data set. Herlocker suggested that weighing should be applied to decrease the correlation coefficient for neighbors with a small MRM. The weighting scheme suggested alters the Pearson R coefficient as follows: H Pa ,u if MRM H Pa ,u | MRM Pa ,u if MRM H Movie Advisor Project 15/51 March 7, 2010 where H is a given threshold (designated Herlock threshold in the paper). Figure 10 depicts the effect of increasing the Herlocker threshold with a Pearson R threshold of 0.1. Users TH=3, Pearson TH=0.1 0.95 0.9 Pearson R Algorithm User Average MAE 0.85 0.8 0.75 0 20 40 60 80 100 120 140 Herlocker Threshold 1 Coverage 0.8 0.6 0.4 0 20 40 60 80 100 120 140 Herlocker Threshold Figure 10: MAE and coverage as a function of Herlocker threshold, Pearson threshold=0.1 As we can see, an increase in the threshold reduces coverage, while also slightly improving the MAE. In an attempt to improve Herlocker‟s method, a slightgly altered weighing scheme was applied: H 2 P if MRM H Pa| ,u a ,u MRM P if MRM H a ,u This slight alteration has the effect of truncating users with low MRM‟s faster, while still making use of „borderline‟ users. Movie Advisor Project 16/51 March 7, 2010 Figure 11 depicts the effect of increasing the Herlocker threshold with a Pearson R threshold of 0.1. Both squared and none squared approaches are shown. Users TH=3, Pearson TH=0.10 0.9 H/MRM 0.88 2 (H/MRM) 0.86 User Average 0.84 0.82 MAE 0.8 0.78 0.76 0.74 0.72 0.7 0.5 0.6 0.7 0.8 0.9 1 Coverage Figure 11: MAE Vs. Coverage with two types of Herlocker thresholding As we can see, the performance of both square and none-square approaches is relatively similar. In future discussion, only the none-square approach will be used. Another alteration to Herlocker‟s threshold was attempted. Instead of a rigid threshold, a threshold based on each users‟ average MRM was calculated for each user. The threshold is a constant multiplied by the average MRM for each user. The logic behind this approach is that some users tend to rate more movies than others, and would therefore tend to have higher MRM‟s. Using a constant threshold for all users‟ MRM does not take into account this variance. Movie Advisor Project 17/51 March 7, 2010 Figure 12 depicts the results obtained with this method. Users TH=3, Pearson TH=0.100 0.9 Pearson R Algorithm User Average MAE 0.85 0.8 0.75 0 2 4 6 8 10 12 Mult. Threshold 0.9 Coverage 0.8 0.7 0.6 0.5 0 2 4 6 8 10 12 Mult. Threshold Figure 12: MAE and coverage as a function of multiplication threshold As we can see, this method performs worse than Herlocker‟s initial proposal and will be discarded. Mean Square Difference (MSD) Algorithm The MSD algorithm is very similar to the Pearson R algorithm, except that it relies on mean square distance as a distance metric, instead of Pearson R correlation. The distance between two users is calculated as: r ra ru ,i ru m 2 a ,i Da ,u i 1 m where m denotes the number of MRM between users a and u. Once the distance vector Da is obtained we calculate Pa as: L Da ,u if Da ,u L Pa ,u L 0 otherwise where L is a threshold for the distance measure. The larger the threshold, the more distant the users we take into account to produce a prediction. Once Pa is obtained, the score is calculated identically to the Pearson R algorithm. Note that all distances are positive, so the absolute value on the distance measure is not necessary. Movie Advisor Project 18/51 March 7, 2010 Figure 13 depicts the effect of decreasing the L threshold on the MAE and coverage: Users TH=3 0.85 MSD Method 0.84 User Average 0.83 0.82 MAE 0.81 0.8 0.79 0.78 0.77 0.5 0.6 0.7 0.8 0.9 1 Coverage Figure 13: MAE Vs. Coverage for MSD algorithm We note that this algorithm does not perform as well as Pearson R at all coverages. We also note a phenomenon of improved user average MAE with decreased coverage. This effect was not present with the Pearson R algorithm, where user average performance was relatively constant with decreasing coverage. This decrease implies that the algorithm tends to make predictions for items that are closer to the users‟ average. Figure 14 may provide help in understanding this effect: 1.1 Discarded User Average 1.05 1 0.95 MAE 0.9 0.85 0.8 0.75 0.5 0.6 0.7 0.8 0.9 1 Coverage Figure 14: MAE Vs. Coverage of discarded items for user average algorithm The figure depicts the MAE of the user average algorithm for items that were omitted with the MSD algorithm for a given coverage. To understand this curve we need to examine it in Movie Advisor Project 19/51 March 7, 2010 three parts. At high high coverage (>0.95) very few items are present in the calculation yielding very noisy results. At mid to high coverage (0.8-0.95) we note that the MSD algorithm actually omits items with user average MAE higher than items that are not omitted. At lower coverages the MSD omits items with better user average MAE, so the MAE begins to decrease. It is possible that movies that are difficult to predict with MSD are esoteric movies that few people rated. Since few people rated these movies, the MSD algorithm eventually omits them. Since the movies are esoteric, a users score for these movies will tend to vary greatly from their mean score. Genre Based Algorithms The GroupLens database provides additional information about each movie in the database. The following is a description of the information provided2: u.item -- Information about the items (movies); this is a tab separated list of movie id | movie title | release date | video release date | IMDb URL | unknown | Action | Adventure | Animation | Children's | Comedy | Crime | Documentary | Drama | Fantasy | Film-Noir | Horror | Musical | Mystery | Romance | Sci-Fi | Thriller | War | Western | The last 19 fields are the genres, a 1 indicates the movie is of that genre, a 0 indicates it is not; movies can be in several genres at once. Since the genres will be referenced as numbers in the ensuing discussion the following table provides a summary of movie names and numbers. # Name # Name # Name # Name 1 Unknown 6 Comedy 11 Film-Noir 16 Sci-Fi 2 Action 7 Crime 12 Horror 17 Thriller 3 Adventure 8 Documentary 13 Musical 18 War 4 Animation 9 Drama 14 Mystery 19 Western 5 Children‟s 10 Fantasy 15 Romance Table 2: Summary of genre numbers The genre algorithm presented in this paper attempts to make use of this additional information to improve the prediction accuracy. Only genre information will be used – dates are omitted. 2 The description is copied verbatim from the README file downloaded with the database. Movie Advisor Project 20/51 March 7, 2010 Genre Statistics Figure 15 depicts average scores and the number of user ratings for each genre: Average Number of User Ratings 40 30 20 10 0 2 4 6 8 10 12 14 16 18 20 Genre Number 0.4 Average Score 0.2 0 -0.2 -0.4 2 4 6 8 10 12 14 16 18 20 Genre Number Figure 15: Average scores and number of user rating for all genres In order to produce the average number of user ratings for each genre all entries in the database are taken into account. For each entry, counters for all genres present in the movie are increased. Once all entries have been searched, the counters are divided by the number of users. The average score is calculated in a similar manner. Initially, a new matrix of user ratings is produced. This new matrix subtracts the users‟ average score from all their ratings. Once the new matrix is obtained, all entries are searched again. For each item a counter for each genre present is increased, and the value from the calculated matrix is added to an accumulator. Note that since user averages were subtracted from the matrix, the accumulators may contain negative values. Once all entries have been searched, the accumulator for each genre is divided by counter for that genre. Not surprisingly, the most popular genres are action, comedy and drama. The negative average score received by both comedy and action implies that although they are popular they are generally disliked. Drama is the only genre that is both widely rated and liked. Base Algorithm Once we obtain genre information, we take aim at another base algorithm. We start by calculating how much a user likes or dislikes a particular genre. To do so we calculate a vector R ra ra for all movies the user entered. For each movie we add the score in vector R to all genres associated with the movie. After going through all movies, we divide the sum obtained for each genre with the number of movies that the genre appeared in. This calculation is similar to the one performed to obtain average genre scores for the entire database, except that it is performed for each user. We thus obtain a vector with an average score for each genre for every user. For future reference this this vector is designated Ga , and the matrix for all users G. A positive score indicates that a user likes a particular genre. Movie Advisor Project 21/51 March 7, 2010 To produce a prediction for a particular item, the base algorithm averages all of the users‟ scores of genres present in the movie and adds the result to the users‟ average. The following equation shows this calculation: n G a ,u S a ,i ra u 1 n where n is the number of genres present in the movie. This approach yields the following results for the base and test data sets: MAE Coverage [%] 0.836 98.72 Table 3: Result for the genre base algorithm Recalling Table 1, it should be noted that this approach provides an improvement over the user average base algorithm. This implies that genre information is useful in making predictions. Genre Algorithm In this section a novel prediction algorithm based on genre information is proposed. This approach makes use of other users‟ scores for the movie as well as genre information. At first the matrix G is calculated for all users. This matrix is then used to calculate the mean square distance (MSD) between users, as shown previously. Once distances for a particular user are obtained, they are used to weigh movie scores made for the movie by other users. Figure 16 depicts the performance of the genre algorithm. In order to reduce the MAE while lowering our coverage, the distance threshold, L, is decreased. Users TH=3 0.85 Genre Method User Average 0.8 MAE 0.75 0.7 0.5 0.6 0.7 0.8 0.9 1 Coverage Figure 16: MAE Vs. Coverage for genre algorithm using MSD The performance of this algorithm is better than that of Pearson R at low coverages, and comparable at high coverages. Nonetheless, the calculations necessary to produce predictions are much simpler. In order to produce each prediction with the Pearson R algorithm, it is necessary to perform calculations on the entire data base containing Movie Advisor Project 22/51 March 7, 2010 6 16829431.610 items. If the algorithm used is efficient, the calculation complexity may be reduced to the number of entries, 100,000. To produce a single result with the genre algorithm, only 9431918,000 need to be taken into account. In practice, predicting with the genre algorithm proved to be much faster. As with the MSD algorithm, a phenomenon of decreased user average MAE with decreased coverage is encountered. The three area in the MSD curve are present in the genre algorithm as well. The explanation for this phenomenon is identical to that of MSD. An attempt to provide further explanation by analyzing average genre scores of omitted items proved inconclusive. Figure 17 depicts the user average MAE for omitted items as a function of coverage. 1.15 Discarded User Average 1.1 MAE 1.05 1 0.95 0.5 0.6 0.7 0.8 0.9 1 Coverage Figure 17: MAE Vs. Coverage of discarded items for user average algorithm Movie Advisor Project 23/51 March 7, 2010 Another implementation of the genre algorithm may rely on Pearson R as the distance measure. The result obtained with this approach is depicted in Figure 18: Users TH=3, Pears TH=0.1 0.86 Genre Pears Method User Average 0.85 0.84 MAE 0.83 0.82 0.81 0.8 0.5 0.6 0.7 0.8 0.9 1 Coverage Figure 18: MAE Vs. Coverage for genre algorithm using Pearson correlation As wee can see, the performance of this approach is generally worse than the MSD approach and does not improve with reduced coverage. The approach will be discarded from ensuing discussion. Hybrid Genre Algorithm While the genre algorithm makes use of user information to calculate the distance between the user and other users, it does not make use of the genres the users themselves like. In this section a hybrid algorithm that takes into account the weighted score by other users as well as the weighted score of the genres in the movie itself is proposed. The following equation shows how the prediction is calculated: r r P n n u ,i u a ,u G a ,u S a ,i ra u 1 n 1 u 1 P n a ,u u 1 where designates the weight ratio between other users‟ rating of the movie, and the users‟ own average rating of the genre. When =0 the algorithm resorts to the base genre algorithm, and when =1 the proposed genre algorithm is used solely. Movie Advisor Project 24/51 March 7, 2010 This approach produces further improvement. Figure 19 shows the MAE as a function of for two thresholds. Users TH=3 0.83 L=0.9, coverage=98.1 0.82 L=0.5, coverage=71.3 0.81 0.8 0.79 MAE 0.78 0.77 0.76 0.75 0.74 0 0.2 0.4 0.6 0.8 1 Ratio Figure 19: MAE Vs. for hybrid genre algorithm An examination of both curves shows that they have an absolute minimum at approximately =0.65. At this mostly other users‟ opinions are taken into account, but a strong weight is still given to the genres present in the movie. To test the robustness of the value of the optimal , the same plot was obtained for a different instance of the base and test data sets. Figure 20 depicts the result obtained: Users TH=3 0.84 0.83 L=0.9, coverage=95.5 0.82 L=0.5, coverage=78.1 0.81 MAE 0.8 0.79 0.78 0.77 0 0.2 0.4 0.6 0.8 1 Ratio Figure 20: MAE Vs. for hybrid genre algorithm, different data-set Movie Advisor Project 25/51 March 7, 2010 As can be seen, the optimal is still obtained around 0.65. Figure 21 compares the genre algorithm to the hybrid genre algorithm with =0.65. Users TH=3, rat=0.65 0.82 Hybrid Method 0.8 Genre Method 0.78 0.76 MAE 0.74 0.72 0.7 0.68 0.5 0.6 0.7 0.8 0.9 1 Coverage Figure 21: MAE Vs. Coverage of genre and hybrid genre (=0.65) algorithms The figure shows that the hybrid genre algorithm performs better than the genre algorithms at all coverages. The additional complexity of the hybrid algorithm is negligible. Algorithm Comparison MAE Vs. Coverage For All Algorithms Figure 22 shows a comparison of the main algorithms discussed in this paper. 0.82 0.8 0.78 MAE 0.76 0.74 Hybrid Genre 0.72 MSD Pearson R 0.7 0.5 0.6 0.7 0.8 0.9 1 Coverage Figure 22: MAE Vs. Coverage for hybrid genre, MSD, and Pearson R algorithms Movie Advisor Project 26/51 March 7, 2010 It is clear that the hybrid genre algorithm performs better at low coverages, while Pearson R performs better at high coverages. It should be noted that the hybrid genre algorithm is the only algorithm able to significantly improve its performance with decreased coverage. Database Alteration In this section the robustness of the results will be tested against several alterations to the data set. Instantiation As noted in earlier discussion, results may differ if a different instance of division into base and test data sets is used. Table 4 shows results obtained with base algorithms for new division of base/test data sets: Method MAE Coverage [%] All Average 0.948 100 Movie Average 0.832 99.8 User Average 0.85 100 User Movie Average 0.785 99.8 Genre Base 0.85 98.7 Table 4: Base algorithm results for new instance As Table 4 shows, the user average method produces results similar to the genre base method. This suggests, that it is more difficult for genre based algorithms to produce accurate results for this division of test and base data sets. Figure 23 depicts a comparison of the main algorithms for the new instance of test and base data sets: 0.84 Hybrid Genre MSD 0.82 Pearson R 0.8 MAE 0.78 0.76 0.74 0.72 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 Coverage Figure 23: MAE Vs. Coverage for Hybrid Genre, MSD, and Pearson R algorithms In this instance, the Pearson R algorithm seems superior at most coverages. The hybrid genre algorithms still manages to improve accuracy with reduced coverage, yielding the lowest MAE at coverages below 0.65. The general behavior of the three algorithms is still maintained. Movie Advisor Project 27/51 March 7, 2010 User Truncation In this section 4/5 of the users are randomly removed from the database. The test data set is comprised of 5 scores for 20% of the remaining users, yielding 190 entries. Since the data sets have been altered, the results for base algorithms are recalculated. As Table 5 shows, it is generally simpler to predict for the new data sets: Method MAE Coverage [%] All Average 0.989 100 Movie Average 0.818 99.5 User Average 0.807 100 User Movie Average 0.750 99.5 Genre Base 0.807 99.5 Table 5: Base algorithm results for data base with 1/5 of users As Table 5 indicates, genre information is once more at a disadvantage since it fails to improve the accuracy of the user average method. Figure 24 shows a comparison of the main algorithms discussed in this paper with respect to the new data sets. 0.78 0.76 MAE 0.74 0.72 Hybrid Genre MSD 0.7 Pearson R 0.5 0.6 0.7 0.8 0.9 1 Coverage Figure 24: MAE Vs. Coverage for tested algorithms with 1/5 of users in data sets The results of this section are somewhat of an anomaly with respect to all other results. The MSD algorithms seems to perform best at almost all coverages, while Pearson R and hybrid genre switch rolls with regards to performance at low and high coverages. Movie Truncation In this section 2/3 of the movies are removed from the database leaving 336 movies. The test data set is comprised of 5 scores for 10% of the remaining users, producing 470 entries. Since the data sets have been altered, the results for base algorithms are recalculated. As Table 6 shows, it is generally simpler to predict for the new data sets: Method MAE Coverage [%] All Average 0.931 100 Movie Average 0.786 100 Movie Advisor Project 28/51 March 7, 2010 User Average 0.855 100 User Movie Average 0.771 100 Genre Base 0.884 98.9 Table 6: Base algorithm results for data base with 1/3 of movies Once more, the genre algorithm is at a disadvantage doing more harm than good with respect to the user average base algorithm. Figure 25 shows a comparison of the main algorithms discussed in this paper with respect to the new data sets. 0.84 Hybrid Genre 0.82 MSD Pearson R 0.8 0.78 MAE 0.76 0.74 0.72 0.7 0.5 0.6 0.7 0.8 0.9 1 Coverage Figure 25: MAE Vs. Coverage for tested algorithms with 1/3 of movies in data sets Again, the results are generally in line with previous data sets. The Pearson R algorithm seems to perform very well consistently. One of the reasons behind this improved performance may be the truncation of movies with generally less predictions. Recall Figure 4 which indicated that movies with higher ID‟s tended to have more predictions since they were introduced to the data base later. Movie Advisor Project 29/51 March 7, 2010 Figure 26 shows a histogram of the number of scores given to each movie: 50 45 40 35 30 25 20 15 10 5 0 0 100 200 300 400 500 600 Number of scores for movie Figure 26: Histogram of number of scores given to each movie Comparing this figure to Figure 2, we learn that most movies in the new data set generally tend to have more predictions. The median of the number of scores for each movie has risen from 57 to 129. This reduced sparseness of the database may improve the performance of the Pearson R algorithm. Database reduction The purpose of this section is to test to performance of the algorithms with respect to a more sparse database. At first, 2/3 of the entries for each user are randomly removed. From the remaining entries, one score from 20% of users is used as the test data set, yielding 189 entries. Since the data sets have been altered, the results for base algorithms are recalculated. Table 7 summarizes the results obtained with the base algorithms: Method MAE Coverage [%] All Average 0.998 100 Movie Average 0.868 98.9 User Average 0.807 100 User Movie Average 0.809 98.9 Genre Base 0.778 98.4 Table 7: Base algorithm results for data base with 1/3 of entries Movie Advisor Project 30/51 March 7, 2010 Figure 27 shows a comparison of the main algorithms discussed in this paper with respect to the new data sets. Hybrid Genre 0.85 MSD Pearson R 0.8 MAE 0.75 0.7 0.65 0.5 0.6 0.7 0.8 0.9 1 Coverage Figure 27: MAE Vs. Coverage for tested algorithms with 1/3 of entries Results obtained with the diluted database show the hybrid genre algorithms to be superior at most coverages. This superiority may arise from the fact that the hybrid genre algorithm uses as little as 19 features to represent each user. These 19 features may quickly stabilize, even with a very sparse data set. Movie Advisor Project 31/51 March 7, 2010 Conclusions Several prediction algorithms were analyzed using the GroupdLens database. Existing methods such as Pearson R and MSD were presented and compared to proposed algorithms based on genre information. The Pearson R algorithm performs quite well under variations of the database. It is always able to yield improvement over trivial methods. The best of the proposed algorithms, hybrid genre, was shown to generally excel at low coverages, and increased sparseness. Besides performing better at specific problem sets, the hybrid genre algorithm is substantially less complex. Coupled with its ability to perform well in sparse environments, the hybrid genre algorithm may prove to be very practical in real world applications. One practical approach may be to intentionally dilute the database in order to reduce computational complexity. Further research may be conducted to improve the performance of the hybrid genre algorithm. One direction may be to collapse several genres into one, or to omit unpopular genres altogether. This direction was shown to improve results while working on this paper, but was not presented. The logic behind this approach is that unpopular genres are given less of an opportunity to affect results for the popular genres, and therefore reducing „noise‟. Another direction may be the complete opposite of this one. Several genres may be designated as new genres when appearing together. For example, people may have a specific affection for action-comedies. Movie Advisor Project 32/51 March 7, 2010 Reference 1. Upendra Shardanand, “Social Information Filtering for Music Recommendation”, MIT EECS M. Eng. Thesis, also TR-94-04, Learning and Common Sense Group, MIT Media Laboratory, 1994. 2. Herlocker, J.L., Konstan, J.A., Borchers, A., Riedl, J., 1999. An algorithmic framework for performing collaborative filtering. Proceedings of the 1999 Conference on Research and Development in Information Retrieval. Movie Advisor Project 33/51 March 7, 2010 Appendix I: Implementation Introduction All results in this paper were calculated and plotted using Matlab 5.3. To improve efficiency, databases were defined as sparse matrixes and for loops were avoided as much as possible by using look up tables and matrix multiplication. Pearson R algorithms were rather slow to produce results. An attempt to compile m files into mex files showed no significant improvement and was discarded. The following lists the matlab code used to generate results in this paper. Matlab Code General List2Mat.m This m-file is used to convert a list of entries into a sparse matrix. function mat=list2mat(list) % USAGE: mat=list2mat(list) % This function converts a list (UID, MID, R) into a sparse matrix U x M = R mat=0; mat=sparse(mat); l=length(list); h=waitbar(0,'Processing List2Mat'); for i=1:l, if mod(i,2000)==0 waitbar(i/l,h); end mat(list(i,1),list(i,2))=list(i,3); end close(h) Mkdata.m This m-file is used to generate base and test data sets. function [base_mat, test_list]=mkdata(mat_base,varargin) % This function resizes an input sparse matrix and divides % the data into a base matrix (UxM<-r) and a test list (uid mid r). % Base and test data are divided according to parameters r and tnum. % % USAGE:[base_mat, test_list]=mkdata(mat_base,r,tnum,mnum,unum) % % base_mat- base matrix % test_list- test list % mat_base- input sparse matrix containg all data % r- ratio of users tested (default= 0.1) % tnum- number of movies tested per user (default = 5) Movie Advisor Project 35/51 March 7, 2010 % mnum- number of movies in output data (default= base_mat) % unum- number of users in output data (default = base_mat) r=0.1; % percent of users tested [unum, mnum]=size(mat_base); % default matrix size, untruncated tnum=5; if nargin>=2 r=varargin{1}; end if nargin>=3 tnum=varargin{2}; end if nargin>=4 mnum=varargin{3}; end if nargin==5 unum=varargin{4}; end if (nargin<1 | nargin>5) error('Improper number of input arguements'); end base_mat=mat_base(1:unum,1:mnum); % truncate matrix if necessary n_u_tst=round(r*unum); % number of users tested p=randperm(unum); % randomize user index for i=1:n_u_tst, idx=find(base_mat(p(i),:)~=0); % indices of rated movies of user ns=length(idx); % number of scored items rs=randperm(ns); % randomize scored movie index for j=1:tnum, % select 5 random movies for each user test_list((i-1)*tnum+j,:)=[p(i) idx(rs(j)) mat_base(p(i),idx(rs(j)))]; base_mat(p(i),idx(rs(j)))=0; end end [tmp,index]=sort(test_list,1); % sort _test_list_ according to user number Movie Advisor Project 36/51 March 7, 2010 test_list=test_list(index(:,1),:); Database Statistics This m-file is used to generate general statistics about the GroupLens database. Stats.m % This code is used to generate statistics for the entire MovieLens database [uid, mid, r]=textread('u.data','%u %u %u %*u'); list=[uid mid r]; clear uid mid r rnum=length(list); mat_base= list2mat(list); % load data_load <- used to load file and avoid processing of above... % calcualte mean and median fprintf('\nMean Value is %1.2f, median is %1.2f\n',mean(list(:,3)),median(list(:,3))); % plot score historgram hist(list(:,3),[1 2 3 4 5]); xlabel('Score') % number of movie ratings, histogram and mean mr=full(sum(mat_base~=0)); fprintf('The average number of ratings per movie is %2.1f\n',mean(mr)); hist(mr,50) xlabel('Number of Scores for Movie') % number of movie ratings mu=full(sum(mat_base~=0,2)); fprintf('The average number of ratings per user is %2.1f\n',mean(mu)); hist(mu,50) xlabel('Number of Scores Per User') Movie Advisor Project 37/51 March 7, 2010 % view score matrix spy(mat_base) xlabel('Movie ID') ylabel('User ID') Base Algorithms Basepr.m This m-file is used to calculate results for the base algorithms. function [ave_pred, u_pred, m_pred, um_pred] =basepr(mat_base,list_test); % This function takes a rating matrix (UxM,R) and a list of observations (uid, mid, r) % and returns a list of predictions using several methods % USAGE: [ave_pred, u_pred, m_pred, um_pred]=basepr(mat_base,list_test) % ave_pred- average score in rating matrix % u_pred- average user score predicted for each movie % m_pred- average movie score predicted for each movie (0 is not available) % um_pred- average user and movie score predicted for each movie (0 is not available) [unum,mnum]=size(mat_base); user_mean=sum(mat_base,2)./sum(mat_base~=0,2); % vector of mean scores of each user rated=sum(mat_base~=0)~=0; % is the movie rated at all? mov_ave= rated.*sum(mat_base)./(sum(mat_base~=0)+~rated); % mean score of each movie, 0 if not rated mrated=rated(list_test(:,2)); % vecotor designating rated movies for list_test mrated_i=find(mrated~=0); % indices of movies predicted with mean score of movie ave_pred=full(sum(mov_ave)./sum(mov_ave~=0)); % average rating of all movies u_pred=full(user_mean(list_test(:,1))); % predict movie score based on user's average scores m_pred=full(mov_ave(list_test(:,2))); % predict movie score based on movie's average scores Movie Advisor Project 38/51 March 7, 2010 mat_ave=(mat_base-repmat(user_mean,1,mnum)).*(mat_base~=0); % rating matrix minus mean of each user, 0 for unrated movies mat_ave=rated.*sum(mat_ave)./(sum(mat_ave~=0)+~rated); um_pred=full(user_mean(list_test(:,1))+mat_ave(list_test(:,2))'); r=list_test(:,3); % output mean and STD for each method... fprintf('\nAll Average\tMAE=%1.3f\tstd=%1.3f\n',mean(abs(r-ave_pred)),std(r-ave_pred)) fprintf('User average\tMAE=%1.3f\tstd=%1.3f\n',mean(abs(u_pred-r)),std(u_pred-r)) fprintf('Movie average\tMAE=%1.3f\tstd=%1.3f\tPredicted %2.1f\n',... full(sum(mrated.*abs(m_pred-r'))./sum(mrated)),full(std(m_pred(mrated_i)-r(mrated_i)')),... (length(r)-length(find(m_pred==0)))/length(r)*100); fprintf('UM average\tMAE=%1.3f\tstd=%1.3f\tPredicted %2.1f\n',... full(sum(mrated'.*abs(um_pred-r))./sum(mrated)),full(std(um_pred(mrated_i)-r(mrated_i))),... (length(r)-length(find(m_pred==0)))/length(r)*100); Pearson r Algorithm Pearsnn.m This m-file is used to produce predictions using the Pearson R algorithms. The m-file also outputs several statistics to the standard output. function list_pred=pearsnn(mat_base,list_test, varargin); % This function takes a rating matrix (UxM,R) and a list of observations (uid, mid, r) % and returns a list of predictions using the pearson r coefficient. % % USAGE: list_pred=pearsnn(mat_base,list_test,pears_th, users_th, herlck_th) % list_pred- a list of [uid mid r pred u_ave] % % mat_base- base dataset in the form of a sparse matrix % list_test- test list in the form of [uid mid r], not presumed to be sorted % Optional Parameters: % pears_th- pearson threshhold (default=0.1) % users_th- minimal number of correlated users to make a prediction (default=3) % herlck_th- number of matching rated movies between users (default=25) herlck_th=25; Movie Advisor Project 39/51 March 7, 2010 pears_th=0.1; % default pearson r threshhold users_th=3; % default minimum number of 'affecting users' if nargin>=3 pears_th=varargin{1}; end if nargin>=4 users_th=varargin{2}; end if nargin==5 herlck_th=round(varargin{3}); end if (nargin<2 | nargin>5) error('Improper number of input arguements'); end h=waitbar(0,'Processing Pears...'); [tmp,index]=sort(list_test,1); % sort _list_test_ according to user number list_test=list_test(index(:,1),:); [unum,mnum]=size(mat_base); num_test=size(list_test,1); prev_user=0; count=1; user_mean=sum(mat_base,2)./sum(mat_base~=0,2); % vector of mean score of each user mat_ave=(mat_base-repmat(user_mean,1,mnum)).*(mat_base~=0); % rating matrix minus mean of each user, 0 for unrated movies for i=1:num_test, % calculate expected score for each test item user=list_test(i,1); mid=list_test(i,2); if prev_user~=user; % the distance is recaulculated whenever a new user is encounterd prev_user=user; ref=repmat(mat_ave(user,:),unum,1); % repeated matrix of tested user minus mean, 0 for unrated movies rated=(mat_ave~=0).*(ref~=0); % mutually rated movies... Movie Advisor Project 40/51 March 7, 2010 srated=sum(rated,2); % number of mutually rated movies cv=sqrt(sum((mat_ave.^2).*rated,2).*sum((ref.^2).*rated,2))+(sum(rated,2)==0); % cov between the _user_ and others, 1 if no match (to avoid division by 0) pr=(sum(rated,2)~=0).*sum(ref.*mat_ave.*rated,2)./cv; % pearson r coef. between _user_ and others, if there is no overlap pr=0 % should be [-1,1], higher absolute values signify stronger correlation % herlck_th=3*full(round(sum(srated)./sum(srated~=0))); <- unused 'adaptive herlck' pr=srated/herlck_th.*pr.*(srated<=herlck_th)+pr.*(srated>herlck_th); %pr=(srated/herlck_th).^2.*pr.*(srated<=herlck_th)+pr.*(srated>herlck_th); % add herlock, used to lessen the influence of users with small srated... w=(abs(pr)>=pears_th).*sign(pr).*(abs(pr)-pears_th)/(1-pears_th); % weight used instead of pearson r coef, used to incorporate threshholding end mscr=full(mat_ave(:,mid)); % vector of movie scores minus user average sumw=abs(w)'*(mscr~=0); % sum of w for correlated users who scored movie rel_users=full(sum((w~=0).*(mscr~=0))); % if sumw~=0 % predict only if other matching users rated the movie... if rel_users>users_th score= user_mean(user)+full(w'*mscr./sumw); % expected score score=(score<=1)+score*((score<5)&(score>1))+5*(score>=5); % truncate exceptional values list_pred(count,:)=[user mid list_test(i,3) score user_mean(user)]; count=count+1; end if mod(i,25)==0 waitbar(i/num_test,h); end end close(h) if exist('list_pred') fprintf('\nPearson TH=%.3f, Herlock TH=%i, Users TH=%i\n',pears_th, herlck_th, users_th); fprintf('Coverage=%.4f, MAE Average=%.4f, MAE Pearson=%.4f\n', ... (count-1)/num_test, ... mean(abs(list_pred(:,3)-list_pred(:,5))), ... mean(abs(list_pred(:,3)-list_pred(:,4)))); else fprintf('No predictions made'); end Movie Advisor Project 41/51 March 7, 2010 MSD Msd.m This m-file is used to produce predictions using the MSD algorithm. The m-file also outputs several statistics to the standard output. function [list_pred, list_npred]=msd(mat_base,list_test, varargin); % This function takes a rating matrix (UxM,R) a list of observations (uid, mid, r) % and returns a list of predictions using the mean square difference. % % USAGE: [list_pred, list_npred]=msd(mat_base,list_test, g, L, users_th) % list_pred- a list of predicted values [uid mid r pred u_ave] % list_npred- a list of values that were not predicted [uid mid r u_ave] % % mat_base- base dataset in the form of a sparse matrix % list_test- test list in the form of [uid mid r], not presumed to be sorted % % Optional Parameter: % L- msd threshold (default = 0.2) % users_th- minimal number of related users to make a prediction (default=3) users_th=3; L=.8; if nargin>=3 L=varargin{1}; end if nargin>=4 users_th=varargin{2}; end if (nargin<2 | nargin>4) error('Improper number of input arguements'); end h=waitbar(0,'Processing msd...'); [tmp,index]=sort(list_test,1); % sort _list_test_ according to user number list_test=list_test(index(:,1),:); Movie Advisor Project 42/51 March 7, 2010 [unum,mnum]=size(mat_base); num_test=size(list_test,1); prev_user=0; count=1; countn=1; user_mean=sum(mat_base,2)./sum(mat_base~=0,2); % vector of mean score of each user mat_ave=(mat_base-repmat(user_mean,1,mnum)).*(mat_base~=0); % rating matrix minus mean of each user, 0 for unrated movies for i=1:num_test, % calculate expected score for each test item user=list_test(i,1); mid=list_test(i,2); if prev_user~=user; % the distance is recaulculated whenever a new user is encounterd prev_user=user; ref=repmat(mat_ave(user,:),unum,1); % repeated matrix of tested user minus mean, 0 for unrated movies rated=(mat_ave~=0).*(ref~=0); % mutually rated movies... srated=sum(rated,2); % number of mutually rated movies pr=sqrt(sum((mat_ave-ref).^2.*rated,2)./(srated+(srated==0))); w=(srated~=0).*(pr<L).*(L-pr)/L; % weight used instead of pearson r coef, used to incorporate threshholding end mscr=full(mat_ave(:,mid)); % vector of movie scores minus user average sumw=w'*(mscr~=0); % sum of w for correlated users who scored movie rel_users=full(sum((w~=0).*(mscr~=0))); % if sumw~=0 % predict only if other matching users rated the movie... if rel_users>users_th score= user_mean(user)+full(w'*mscr./sumw); % expected score score=(score<=1)+score*((score<5)&(score>1))+5*(score>=5); % truncate exceptional values list_pred(count,:)=[user mid list_test(i,3) score user_mean(user)]; count=count+1; else list_npred(countn,:)=[user mid list_test(i,3) user_mean(user)]; countn=countn+1; end Movie Advisor Project 43/51 March 7, 2010 if mod(i,25)==0 waitbar(i/num_test,h); end end close(h) if exist('list_pred') fprintf('\nL TH=%.3f, Users TH=%i\n',L, users_th); fprintf('Coverage=%.4f, MAE Average=%.4f, MAE msd=%.4f\n', ... (count-1)/num_test, ... mean(abs(list_pred(:,3)-list_pred(:,5))), ... mean(abs(list_pred(:,3)-list_pred(:,4)))); else fprintf('No predictions made'); end if exist('list_npred') fprintf('ommited values, MAE Average=%1.4f\n',mean(abs(list_npred(:,3)-list_npred(:,4)))) end Genre Basegen.m This m-file is used to generate results for the baseline genre algorithm. function g_pred=basegen(mat_base,list_test, g); % base genre prediction algorithm [tmp,index]=sort(list_test,1); % sort _list_test_ according to user number list_test=list_test(index(:,1),:); [unum,mnum]=size(mat_base); num_test=size(list_test,1); user_mean=sum(mat_base,2)./sum(mat_base~=0,2); % vector of mean score of each user mat_ave=(mat_base-repmat(user_mean,1,mnum)).*(mat_base~=0); % rating matrix minus mean of each user, 0 for unrated movies ugen=mat_ave*g; % create a matrix of average user scores for each genre ugenum=(mat_ave~=0)*g; % count the number of rated movies in genre for normalization ugenum1=ugenum+(ugenum==0); % avoid divide by 0; ugen=ugen./ugenum1; % normalize Movie Advisor Project 44/51 March 7, 2010 % modify user average by adding an average of the users average rating for % each genre in the movie gen_ave=ugen(list_test(:,1),:).*g(list_test(:,2),:); % multiply users' average genre scores with genres in the movie sgen=sum(gen_ave~=0,2)~=0; % avoid divide by 0 for movies that can't be predected idx=find(sgen); g_pred=full(user_mean(list_test(:,1))+sum(gen_ave,2)./(sum(gen_ave~=0,2)+~sgen)); % predict user's rating based solely on his average ratings and the movie's genres r=list_test(:,3); fprintf('Genre average\tMAE=%1.3f\tCoverage=%2.2f\n',... mean(abs(g_pred(idx)-r(idx))),length(idx)./num_test*100); Peargen.m This m-file is used to produce predictions using the pearson genre algorithm. The m-file also outputs several statistics to the standard output.. function list_pred=peargen(mat_base,list_test, g, varargin); % This function takes a rating matrix (UxM,R) a list of observations (uid, mid, r) % and a list of genres and returns a list of predictions using the pearson r coefficient. % % USAGE: list_pred=peargen(mat_base,list_test, g, pears_th, users_th, herlck_th) % list_pred- a list of [uid mid r pred u_ave] % % mat_base- base dataset in the form of a sparse matrix % list_test- test list in the form of [uid mid r], not presumed to be sorted % g- list of genres % Optional Parameters: % pears_th- pearson threshhold (default=0.1) % users_th- minimal number of correlated users to make a prediction (default=3) % herlck_th- number of matching rated movies between users (default=30) herlck_th=30; pears_th=0.1; % default pearson r threshhold users_th=3; % default minimum number of 'affecting users' Movie Advisor Project 45/51 March 7, 2010 if nargin>=4 pears_th=varargin{1}; end if nargin>=5 users_th=varargin{2}; end if nargin==6 herlck_th=round(varargin{3}); end if (nargin<3 | nargin>6) error('Improper number of input arguements'); end h=waitbar(0,'Processing Peargen...'); [tmp,index]=sort(list_test,1); % sort _list_test_ according to user number list_test=list_test(index(:,1),:); [unum,mnum]=size(mat_base); num_test=size(list_test,1); prev_user=0; count=1; user_mean=sum(mat_base,2)./sum(mat_base~=0,2); % vector of mean score of each user mat_ave=(mat_base-repmat(user_mean,1,mnum)).*(mat_base~=0); % rating matrix minus mean of each user, 0 for unrated movies ugen=mat_ave*g; % create a matrix of average user scores for each genre ugenum=(mat_ave~=0)*g; % count the number of rated movies in genre for normalization ugenum1=ugenum+(ugenum==0); % avoid divide by 0; ugen=ugen./ugenum1; % normalize for i=1:num_test, % calculate expected score for each test item user=list_test(i,1); mid=list_test(i,2); if prev_user~=user; % the distance is recaulculated whenever a new user is encounterd prev_user=user; ref=repmat(ugen(user,:),unum,1); % repeated matrix of tested users genre prefs Movie Advisor Project 46/51 March 7, 2010 rated=(ugen~=0).*(ref~=0); % mutually rated genres... srated=sum(rated,2); % number of mutually rated genres cv=sqrt(sum((ugen.^2).*rated,2).*sum((ref.^2).*rated,2))+(sum(rated,2)==0); % cov between the _user_ and others, 1 if no match (to avoid division by 0) pr=(sum(rated,2)~=0).*sum(ref.*ugen.*rated,2)./cv; % pearson r coef. between _user_ and others, if there is no overlap pr=0 % should be [-1,1], higher absolute values signify stronger correlation pr=(srated/herlck_th).*pr.*(srated<=herlck_th)+pr.*(srated>herlck_th); % add herlock, used to lessen the influence of users with small srated... w=(abs(pr)>=pears_th).*sign(pr).*(abs(pr)-pears_th)/(1-pears_th); % weight used instead of pearson r coef, used to incorporate threshholding end mscr=full(mat_ave(:,mid)); % vector of movie scores minus user average sumw=abs(w)'*(mscr~=0); % sum of w for correlated users who scored movie rel_users=full(sum((w~=0).*(mscr~=0))); % if sumw~=0 % predict only if other matching users rated the movie... if rel_users>users_th score= user_mean(user)+full(w'*mscr./sumw); % expected score score=(score<=1)+score*((score<5)&(score>1))+5*(score>=5); % truncate exceptional values list_pred(count,:)=[user mid list_test(i,3) score user_mean(user)]; count=count+1; end if mod(i,25)==0 waitbar(i/num_test,h); end end close(h) fprintf('\nPearson TH=%.3f, Herlock TH=%i, Users TH=%i\n',pears_th, herlck_th, users_th); fprintf('Coverage=%.4f, MAE Average=%.4f, MAE Pearson=%.4f\n', ... (count-1)/num_test, ... mean(abs(list_pred(:,3)-list_pred(:,5))), ... mean(abs(list_pred(:,3)-list_pred(:,4)))); Genmsd.m This m-file is used to produce predictions using the genre MSD algorithm. The m-file also outputs several statistics to the standard output.. function [list_pred, list_npred]=genmsd(mat_base,list_test, g, varargin); % This function takes a rating matrix (UxM,R) a list of observations (uid, mid, r) Movie Advisor Project 47/51 March 7, 2010 % and a list of genres and returns a list of predictions using mean square distance. % % USAGE: [list_pred, list_npred]=genmsd(mat_base,list_test, g, L, users_th) % list_pred- a list of predicted values [uid mid r pred u_ave] % list_npred- a list of values that were not predicted [uid mid r u_ave] % % mat_base- base dataset in the form of a sparse matrix % list_test- test list in the form of [uid mid r], not presumed to be sorted % g- list of genres % Optional Parameter: % users_th- minimal number of correlated users to make a prediction (default=10) % L- rms threshold users_th=10; L=.4; if nargin>=4 L=varargin{1}; end if nargin>=5 users_th=varargin{2}; end if (nargin<3 | nargin>5) error('Improper number of input arguements'); end h=waitbar(0,'Processing Peargen...'); [tmp,index]=sort(list_test,1); % sort _list_test_ according to user number list_test=list_test(index(:,1),:); [unum,mnum]=size(mat_base); num_test=size(list_test,1); prev_user=0; count=1; countn=1; user_mean=sum(mat_base,2)./sum(mat_base~=0,2); % vector of mean score of each user mat_ave=(mat_base-repmat(user_mean,1,mnum)).*(mat_base~=0); Movie Advisor Project 48/51 March 7, 2010 % rating matrix minus mean of each user, 0 for unrated movies ugen=mat_ave*g; % create a matrix of average user scores for each genre ugenum=(mat_ave~=0)*g; % count the number of rated movies in genre for normalization ugenum1=ugenum+(ugenum==0); % avoid divide by 0; ugen=ugen./ugenum1; % normalize for i=1:num_test, % calculate expected score for each test item user=list_test(i,1); mid=list_test(i,2); if prev_user~=user; % the distance is recaulculated whenever a new user is encounterd prev_user=user; ref=repmat(ugen(user,:),unum,1); % repeated matrix of tested users genre prefs rated=(ugen~=0).*(ref~=0); % mutually rated genres... srated=sum(rated,2); % number of mutually rated genres pr=sqrt(sum((ugen-ref).^2.*rated,2)./srated); w=(pr<L).*(L-pr)/L; % weight used instead of pearson r coef, used to incorporate threshholding end mscr=full(mat_ave(:,mid)); % vector of movie scores minus user average sumw=w'*(mscr~=0); % sum of w for correlated users who scored movie rel_users=full(sum((w~=0).*(mscr~=0))); % if sumw~=0 % predict only if other matching users rated the movie... if rel_users>users_th score= user_mean(user)+full(w'*mscr./sumw); % expected score score=(score<=1)+score*((score<5)&(score>1))+5*(score>=5); % truncate exceptional values list_pred(count,:)=[user mid list_test(i,3) score user_mean(user)]; count=count+1; else list_npred(countn,:)=[user mid list_test(i,3) user_mean(user)]; countn=countn+1; end if mod(i,25)==0 waitbar(i/num_test,h); end end close(h) if exist('list_pred') Movie Advisor Project 49/51 March 7, 2010 fprintf('\nL TH=%.3f, Users TH=%i\n',L, users_th); fprintf('Coverage=%.4f, MAE Average=%.4f, MAE Genre=%.4f\n', ... (count-1)/num_test, ... mean(abs(list_pred(:,3)-list_pred(:,5))), ... mean(abs(list_pred(:,3)-list_pred(:,4)))); fprintf('ommited values, MAE Average=%1.4f\n',mean(abs(list_npred(:,3)-list_npred(:,4)))) else fprintf('No predictions made'); end Hgenmsd.m This m-file is used to produce predictions using the hybrid genre algorithm. The m-file also outputs several statistics to the standard output.. function [list_pred, list_npred]=hgenmsd(mat_base,list_test, g, varargin); % This function takes a rating matrix (UxM,R) a list of observations (uid, mid, r) % and a list of genres and returns a list of predictions using the pearson r coefficient. % The algorithm used is a hybrid of Genre algorithm, using with weighting a users' % own prefrences. % % USAGE: [list_pred, list_npred]=hgenmsd(mat_base,list_test, g, L, rat, users_th) % list_pred- a list of predicted values [uid mid r pred u_ave] % list_npred- a list of values that were not predicted [uid mid r u_ave] % % mat_base- base dataset in the form of a sparse matrix % list_test- test list in the form of [uid mid r], not presumed to be sorted % g- list of genres % % Optional Parameter: % L- msd threshold (default=.7) % rat- ratio between weight given to other users and user (default=0.65) % users_th- minimal number of correlated users to make a prediction (default=3) % obtain parameters users_th=3; L=.7; rat=.65; if nargin>=4 Movie Advisor Project 50/51 March 7, 2010 L=varargin{1}; end if nargin>=5 rat=varargin{2}; end if nargin>=6 users_th=varargin{3}; end if (nargin<3 | nargin>6) error('Improper number of input arguements'); end h=waitbar(0,'Processing hgenmsd...'); [tmp,index]=sort(list_test,1); % sort _list_test_ according to user number list_test=list_test(index(:,1),:); [unum,mnum]=size(mat_base); num_test=size(list_test,1); prev_user=0; count=1; countn=1; user_mean=sum(mat_base,2)./sum(mat_base~=0,2); % vector of mean score of each user mat_ave=(mat_base-repmat(user_mean,1,mnum)).*(mat_base~=0); % rating matrix minus mean of each user, 0 for unrated movies ugen=mat_ave*g; % create a matrix of average user scores for each genre ugenum=(mat_ave~=0)*g; % count the number of rated movies in genre for normalization ugenum1=ugenum+(ugenum==0); % avoid divide by 0; ugen=ugen./ugenum1; % normalize for i=1:num_test, % calculate expected score for each test item user=list_test(i,1); mid=list_test(i,2); if prev_user~=user; % the distance is recaulculated whenever a new user is encounterd prev_user=user; ref=repmat(ugen(user,:),unum,1); % repeated matrix of tested users genre prefs Movie Advisor Project 51/51 March 7, 2010 rated=(ugen~=0).*(ref~=0); % mutually rated genres... srated=sum(rated,2); % number of mutually rated genres pr=sqrt(sum((ugen-ref).^2.*rated,2)./(srated+(srated==0))); w=(srated~=0).*(pr<L).*(L-pr)/L; % weight used instead of pearson r coef, used to incorporate threshholding end mscr=full(mat_ave(:,mid)); % vector of movie scores minus user average sumw=w'*(mscr~=0); % sum of w for correlated users who scored movie rel_users=full(sum((w~=0).*(mscr~=0))); % if sumw~=0 % predict only if other matching users rated the movie... if rel_users>users_th idx=find(g(mid,:)~=0); u_gen_mid=sum(ugen(user,idx))./length(idx); score= user_mean(user)+rat*full(w'*mscr./sumw)+(1-rat)*u_gen_mid; % expected score score=(score<=1)+score*((score<5)&(score>1))+5*(score>=5); % truncate exceptional values list_pred(count,:)=[user mid list_test(i,3) score user_mean(user)]; count=count+1; else list_npred(countn,:)=[user mid list_test(i,3) user_mean(user)]; countn=countn+1; end if mod(i,25)==0 waitbar(i/num_test,h); end end close(h) if exist('list_pred') fprintf('\nL TH=%.3f, Users TH=%i, ratio=%.2f\n',L, users_th,rat); fprintf('Coverage=%.4f, MAE Average=%.4f, MAE Hybrid Genre=%.4f\n', ... (count-1)/num_test, ... mean(abs(list_pred(:,3)-list_pred(:,5))), ... mean(abs(list_pred(:,3)-list_pred(:,4)))); fprintf('ommited values, MAE Average=%1.4f\n',mean(abs(list_npred(:,3)-list_npred(:,4)))) else fprintf('No predictions made'); end

DOCUMENT INFO

Shared By:

Categories:

Tags:
database design, database characteristics, database systems, the user, physical characteristics, relational database, stored procedures, microsoft access, sponsored links, occupational safety and health, reference manual, relational model, design tutorial, database software, primary key

Stats:

views: | 26 |

posted: | 3/7/2010 |

language: | English |

pages: | 51 |

OTHER DOCS BY scw21404

How are you planning on using Docstoc?
BUSINESS
PERSONAL

By registering with docstoc.com you agree to our
privacy policy and
terms of service, and to receive content and offer notifications.

Docstoc is the premier online destination to start and grow small businesses. It hosts the best quality and widest selection of professional documents (over 20 million) and resources including expert videos, articles and productivity tools to make every small business better.

Search or Browse for any specific document or resource you need for your business. Or explore our curated resources for Starting a Business, Growing a Business or for Professional Development.

Feel free to Contact Us with any questions you might have.