Towards User Profiling for Web Recommendation

S. Zhang and R. Jarvis (Eds.): AI 2005, LNAI 3809, pp. 415 – 424, 2005.© Springer-Verlag Berlin Heidelberg 2005Towards User Profiling for Web RecommendationGuandong Xu 1, Yanchun Zhang 1, and Xiaofang Zhou 21School of Computer Science and Mathematics,Victoria University, PO Box 14428, VIC 8001, Australia {xu,yzhang}@.au 2 School of Information Technology & Electrical Engineering,University of Queensland, Brisbane QLD 4072, Australia zxf@.au Abstract. Collaborative recommendation is one of widely used recommenda-tion systems, which recommend items to visitor on a basis of referring other’spreference that is similar to current user. User profiling technique upon Webtransaction data is able to capture such informative knowledge of user task orinterest. With the discovered usage pattern information, it is likely to recom-mend Web users more preferred content or customize the Web presentation tovisitors via collaborative recommendation. In addition, it is helpful to identifythe underlying relationships among Web users, items as well as latent tasks dur-ing Web mining period. In this paper, we propose a Web recommendationframework based on user profiling technique. In this approach, we employProbabilistic Latent Semantic Analysis (PLSA) to model the co-occurrence ac-tivities and develop a modified k-means clustering algorithm to build user pro-files as the representatives of usage patterns. Moreover, the hidden task modelis derived by characterizing the meaningful latent factor space. With the dis-covered user profiles, we then choose the most matched profile, which pos-sesses the closely similar preference to current user and make collaborative rec-ommendation based on the corresponding page weights appeared in the selecteduser profile. The preliminary experimental results performed on real world datasets show that the proposed approach is capable of making recommendation ac-curately and efficiently.1 IntroductionIn recent years, the massive influx of information onto World Wide Web has facili-tated user, not only retrieving information, but also discovering knowledge. However, Web users usually suffer from the information overload problem due to the fact of significantly increasing and rapidly expanding growth in amount of information on the Web. One approach addressed to the information overload is the recommendation system, which aims to help users locate more needed or preferred information. Typi-cally, Web recommendation system focuses on the processes of identifying Web users or objects, collecting information with respect to users’ preference or interests as well as adapting its service to satisfy the users’ needs. In short, Web recommendation can be used to provide better quality service and application of Web to users during their browsing period.416 G. Xu, Y. Zhang, and X. ZhouTo-date, the problem of recommending appropriate items from data repository to users has been extensively studied and two paradigms named content-based filtering and collaborative filtering systems have emerged. Content-based filtering systems such as WebWatcher [8], try to recommend items that are similar to those visited by a given user in the past, whereas collaborative filtering systems intend to identify user category whose taste or preference is close enough to the given user and recommend items that are historically rated by them [6]. The former often utilizes traditional in-formation filtering and information retrieval methods, while the latter employs user correlation or nearest-neighbor algorithm. Especially, the collaborative filtering tech-nique has been gradually adopted in the context of Web recommendation applications and has achieved great success as well [5, 9] in recent years.Web usage mining technique, which exploits data mining methods, such as k-Nearest Neighbor algorithm (kNN) [5], Web user or page clustering [4, 11, 12], asso-ciation rule mining [1] and sequential pattern mining technique [2], to create model based on the analysis of usage data, has been used in building Web recommendation system recently. With the usage pattern knowledge discovered in Web usage mining process, Web recommendation system can generate usage-based user profiles as the representatives of the aggregate user behaviors for collaborative recommendation.As a result, a variety of research communities have addressed this topic and Web usage mining is becoming a potential approach for Web recommendation. To reveal the un-derlying relationships among Web objects, Latent Semantic Analysis (LSA) tech-nique has been incorporated into Web usage mining process. Some LSA-based algo-rithms are developed for Web recommendation [13, 14].In this paper, we propose a Web recommendation framework based on user profil-ing technique. The usage pattern knowledge, in the form of user profile derived from Web usage mining, is combined into Web recommendation system to improve the ef-ficiency of recommendation by predicting user-preferred content and customizing the presentation. During pattern discovery stage, probabilistic inference method based on Probabilistic Latent Semantic Analysis (PLSA) model, a variant of LSA, is exploited to model the underlying relationships among the co-occurrence activities and identify the latent task model in terms of latent semantic factor. Through Web user session clustering, we create user profiles as the representatives of usage patterns. To make Web recommendation, we match the current active user activity against such discov-ered patterns to find the most like-minded user category, in turn, determine the poten-tially interested pages as recommendation set based on the visited probabilities exhib-ited by such type of users. We demonstrate the effectiveness of the proposed technique through experiments performed on real world data sets. The evaluation re-sults show that the usage-based approach is more applicable in comparison with some traditional techniques.The rest of the paper is organized as follows. In section 2, we introduce the Web usage mining process, especially we focus on how to model Web co-occurrence ac-tivities based on PLSA. We present the algorithms for discovering usage-based user profiles and latent factors in section 3. In section 4, we propose the Web recommen-dation framework upon user profiling approach. We conduct preliminary experiments on two real world datasets, implement some comparisons against the traditional work in section 5, conclude and outline future work in section 6.Towards User Profiling for Web Recommendation 417 2 Usage-Based User Profiling with PLSAAs discussed above, Web recommendation is the ultimate goal of Web usage mining conducted on the data collected at the Web log servers of a specific Web site. This whole procedure usually consists of three steps, i.e. data collection and preprocessing, pattern mining as well as knowledge application. Figure 1 depicts the whole process.Fig. 1. The process of Web Mining and Web Recommendation2.1 Usage Data RepresentationPrior to introducing user profiling technique, we briefly discuss the issue with respect to construction of usage data. In general, the exhibited user access interests may be re-flected by the varying degrees of visits on different Web pages during one session. Thus, we can represent a user session as a weighted page vector visited by the user during a period. In this paper, we use the following notations to model the co-occurrence activities of Web users and pages:• {}12,,m S s s s = : a set of m user sessions.• {}12,,n P p p p = : a set of n Web pages.• For each user, the navigational session is represented as a sequence of visited pages with corresponding weights: {},1,2,,,i i i i n s a a a = , where ,i j a denotes the weight for page j p visited in i s user session. The corresponding weight is usually determined by the number of hit or the amount time spent on the spe-cific page. Here, we use both of them to construct usage data from two real world data sets.• {},m n i j SP a ×=: the ultimate usage data in the form of weight matrix with di-mensionality of m n ×.418 G. Xu, Y. Zhang, and X. Zhou2.2 PLSA ModelThe PLSA model is based on a statistic model called aspect model, which can be util-ized to identify the hidden semantic relationships among general co-occurrence activi-ties. Similarly, we can conceptually view the user sessions over Web pages space as co-occurrence activities in the context of Web usage mining to discover the latent us-age pattern. For the given aspect model, suppose that there is a latent factor space {}12,,k Z z z z = and each co-occurrence observation data ,i j s p <>is associated withthe factor k z Z ∈ by varying degree to k z .Based on these assumptions and Bayesian rule, we calculate the probability of an observed pair ,i j s p <> by adopting the latent factor variable z k as:(,)()(|)(|)k i j k i k j k z Z P s p P z P s z P P z ∈=••∑ (1)Following the likelihood principle, the total likelihood is determined as∑∈∈•=P p S s j i j i i j i p s P p s m L ,),(log ),( (2)where (,)i j m s p is the element of the session-page matrix corresponding to sessioni s and page j p .In order to maximize the total likelihood, we make use of Expectation Maximiza-tion (EM) algorithm to perform maximum likelihood estimation of ()k P z , (|)i k P s z , (|)j k P p z in latent variable model [3]. The executing of E-step and M-step is repeat-ing until i L is converging to a local optimal limit, which means the estimated results can represent the final probabilities of observation data. It is easily found that the computational complexity of this algorithm is ()O mnk , where m is the number of user session, n is the number of page, and k is the number of factors.3 Discovery of Latent Factors and Usage-Based User ProfilesAs we discussed in section 2, the estimated probabilities quantitatively measure the underlying relationships among Web users, pages as well as latent factors (i.e. tasks). Therefore, it is reasonable to identify the latent factors and discover the related usage-based access patterns upon probability inference process. In this section, we propose how to derive the aforementioned usage information.3.1 Characterizing Latent FactorFirst, we discuss how to capture the latent factor associated with user navigational be-havior. This aim is to be achieved by characterizing the “dominant” pages that con-tribute significantly to the factor. Note that (|)j k p p z represents the conditional oc-currence probability over the page space corresponding to a specific factor, whereas (|)k j p z p reflects the conditional probability distribution over the factor space corre-Towards User Profiling for Web Recommendation 419 sponding to a specific page. Thus, we may choose the pages whose conditional prob-abilities (|)k j p z p and (|)j k p p z are both greater than a predefined threshold to form “dominant” page set. Exploring the contents of these pages would result in character-izing the semantic meaning of each factor. In section 4, we will present various ex-amples of latent factors as well as those “dominant” pages derived from two real data sets.3.2 Building Usage-Based User ProfilesNote that the set of (|)k i P z s is conceptually representing the probability distributionover the latent factor space for a specific user session i s , we, thus, construct the ses-sion-factor matrix based on the calculated probability estimates, to reflect the rela-tionship between Web users and latent factors, which is expressed as follows:',1,2,(,,...,)i i i i k s b b b = (3)where ,i s b is the occurrence probability of session i s on factor s z . In this way, the distance between two session vectors may reflect the exhibited navigational behavior similarity. We, therefore, define their similarity by applying well-known cosine simi-larity as:()''''''22(,),()i j i ji j sim s s s s s s =• (4) where ()'',,1,i j k i m j m m s s b b ==∑, '2i s,'2js =With the page similarity measurement (4), we propose a modified k-means cluster-ing algorithm [13] to partition user sessions into corresponding clusters. As each user session is represented as a weighted page vector, it is reasonable to derive the centroid of cluster obtained as the usage pattern in the form of user profile. In this work, we compute the mean vector to represent the centroid. The algorithm for clustering user sessions and constructing user profiles is as follows:Algorithm 1. Building User ProfilesInput: the set of conditional probabilities (|)k i P z sOutput: A set of user session clusters 12 {,,, }P SCL SCL SCL SCL = and a set of user profiles 12{,,,}p PF PF PF PF =1. For all user sessions, employ the modified k-means clustering algorithm and out-put a set of usage-based session clusters {}t SCL SCL =.2. for each user session cluster, calculate the centroid of cluster as'1/i i t t t s SCL Cid SCL s ∈=•∑ (5) where t SCL is the number of sessions in the cluster.420 G. Xu, Y. Zhang, and X. Zhou3. Treat the centroid of generated cluster as the aggregate user profile, and sort the normalized weights in a descending order to reflect the relative “significance” contributed by the corresponding pages within the selected user profile, i.e.{}2112,,,,,,t t tt t t t n n PF p w p w p w =<><><> (6) where ,1/i t tj t i j s SCL w SCL a ∈=•∑, 12t t t n w w w >>> , and t j p P ∈4. Output {}t PF PF =. 4 Using PLSA for Web PersonalizationGenerally, we recommend Web items to users in customized or preferred style based on analysis of their interests exhibited by individual or groups of users. In this work, we adopt the model-based technique in our Web recommendation framework. We consider the usage-based user profiles generated in section 3.2 as the aggregated rep-resentatives of common navigational behaviors exhibited by all individuals in same particular user category. For a newly coming active user session, we utilize cosine function to measure the similarity between it and discovered user profile. We, then, choose the closest profile, which shares the highest similarity with the current user session, as the matched pattern to current user. Finally, we generate the top-N recom-mendation pages based on the historically visited probabilities of pages by other users in the selected profile. The detailed procedure is as follows:Algorithm 2. Web Recommendation Based on user profilingInput: An active user session and a set of user profilesOutput: The top-N recommendation pages1. The active session and the profiles are to be simplified as n-dimensional weight vectors a s ,p s instead of page-weight pair vector over the page space that is generated from algorithm 3 within a site, i.e. 12[,,,]p p p p n s w w w = , where pi w is the significance weight contributed by page i p in this profile, similarly12[,,]a a a a n s w w w = , where 1a i w =, if page i p is already accessed, and other-wise 0a i w =.2. Measure the similarities between the active session and all derived usage pro-files, and choose the maximum one out of the calculated similarities as the most matched pattern:22(,)max((,))max(())mat j a p a p a p a p j j sim s s sim s s s s s s ==i (7)3. Incorporate the selected profile p mat s with the active session a s , then calculate the recommendation score ()i rs p for each page i p :Towards User Profiling for Web Recommendation 421()mat i i rs p w =, mat mati p w s ∈ (8)Thus, each page in the profile will be assigned a recommendation score be-tween 0 and 1. Note that the recommendation score will be 0 if the page is al-ready visited in the current session.4. Sort the calculated recommendation scores in step 3 in a descending order, i.e.12(,,,)mat mat mat n rs w w w = , and select the N pages with the highest recom-mendation scores to construct the top-N recommendation set:1(){|()(),1,2,,}mat mat mat mat j j j j REC N p rs p rs p j N p P +=>=∈ (9)5 Experiments and EvaluationsIn order to evaluate the effectiveness of the proposed method based on PLSA model and explore the discovered latent semantic factor, we have conducted preliminary ex-periments on two real world data sets.5.1 Data SetsThe first data set we used is downloaded from KDDCUP Web site (/KDDCUP/). After data preparation, we have setup an evalua-tion data set including 9308 user sessions and 69 pages, where every session consists of 11.88 pages in average. We refer this data set to “KDDCUP data”. In this data set, the number of Web page hits by the given user determines the element in session-page matrix associated with the specific page in the given session.The second data set is from a academic Website log files[10]. The data is based on a 2-week Web log file during April of 2002. After data preprocessing stage, the fil-tered data contains 13745 sessions and 683 pages. The entries in the usage data corre-spond to the amount of time (in seconds) spent on pages during a given session. For convenience, we refer this data as “CTI data”.5.2 Latent Factors Based on PLSA ModelWe conduct experiments on the two data sets to extract the latent factors via identify-ing “dominant” page set. Here, we present the experimental results of the derived latent factors from two real data sets based on PLSA model respectively. Table 1 il-lustrates one example out of the derived factors extracted from the KDDCUP data set as well as the “dominant” page set, whose probabilities are over the predefined threshold, whereas Table 2 presents the example out of those from CTI data set. From these tables, it is easily concluded that the factor #6 in KDDCUP data set reflects the scenario involving in online shopping process, whereas the factor #13 stands for ac-tivity of searching postgraduate program information.422 G. Xu, Y. Zhang, and X. ZhouTable 1. Example of laten factor and its associated pages from KDDCUP FactorPage # Content Pgae # Content 27 main/login2 50 account/past_orders 32 main/registration 52 account/credit_info 42 account/your_account 60 checkout/thankyou 44 checkout/expresCheckout64 account/create_credit 45 checout/confirm_order 65 main/welcome# 6onlineshoppingprocess 47 account/address 66 account/edit_credit Table 2. Example of laten factor and its associated pages from CTIFactorPage # Content Pgae # Content 386 /News 588 /Prog/2002/Gradect2002 575 /Programs 590 /Prog/2002/Gradis2002 586 /Prog/2002/Gradcs2002 591 /Prog/2002/Gradmis2002# 13Postgrad-program 587 /Prog/2002/Gradds2002 592 /Prog/2002/Gradse20025.3 Evaluation Metric of User Session Clusters and Web Recommendation In order to evaluate the quality of clusters derived from PLSA-based approach, we adopt one specific metric, named the Weighted Average Visit Percentage (WAVP) [8]. This evaluation method is based on assessing each user profile individually according to the likelihood that a user session, which contains any pages in the session cluster, will include the rest pages in the cluster during the same session. Suppose T is one of session set within the evaluation set, and for s specific cluster C , let T c denote a subset of T whose elements contain at least one page from C, the WAVP is computed as:(,)c t T p PFc t C WAVP wt p pf T ∈∈⎛⎞⎛⎞•=⎜⎟⎜⎟⎝⎠⎝⎠∑∑On the other hand, we exploit a metric called hit precision [7] to measure the preci-sion in the context of top-N recommendation. Given a user session in the test set, we extract the first j pages as an active user session to generate a top-N recommendation set via the procedure described in section 4. Since the recommendation set is in de-scending order, we then obtain the rank of 1j + page in the sorted recommendation list. Furthermore, for each rank 0r >, we sum the number of test data that exactly rank the r th as ()Nb r . Let 1()()ri S r Nb i ==∑, and ()/hitp S N T =, where T repre-sents the number of testing data in the whole test set. Thus, hitp stands for the hit pre-cision of Web recommendation.In order to compare our approach with other existing methods, we implement a baseline method that is based on the clustering technique [11]. This method is toTowards User Profiling for Web Recommendation 423Fig. 2. WAVP comparison for CTI Fig. 3.Hitp comparison for CTI generate usage-based session clusters by performing k-means clustering process on us-age data explicitly. Then, the cluster centroids are treated as the aggregated access patterns.Figures 2 and 3 depict the comparison results of WAVP and hitp coefficient per-formed on CTI dataset using the two methods discussed above respectively. The re-sults demonstrate that the proposed PLSA-based technique consistently overweighs standard clustering-based algorithm in terms of WAVP and hit precision parameter. In this scenario, it can be concluded that our approach is capable of making Web rec-ommendation more accurately and effectively against the conventional method. In addition to recommendation, this approach is able to identify the hidden factors why such user sessions or Web pages are grouped together in same category.6 Conclusion and Future WorkIn this paper, we have developed a Web recommendation framework incorporating user profiling technique based on PLSA model. With the proposed probabilistic method, we can measure the co-occurrence activities (i.e. user sessions) in terms of probability estimations to capture the underlying relationships among Web users, pages as well as latent tasks. Analysis of the estimated probabilities leads to build up usage-based user profiles and identify the hidden factors associated with the corre-sponding interests or patterns as well. The discovered usage patterns in the forms of user profiles is used to make collaborative recommendation, in turn, lead to improve the precision and effectiveness of Web recommendation. We have demonstrated the efficiency of our technique through preliminary experiments performed on the real world datasets and comparisons with other existing work.Our future work will focus on the following issues: we intend to identify the primitive task of active user and incorporate Web page categories to predict user potentially visited pages, and implement more experiments to validate the scalability of our approach.424 G. Xu, Y. Zhang, and X. ZhouReferences1 R. Agarwal, C. Aggarwal and V. Prasad, A Tree Projection Algorithm for Generation ofFrequent Itemsets, Journal of Parallel and Distributed Computing, 61 (1999), pp. 350-371.2 R. Agrawal and R. Srikant, Mining Sequential Patterns, in P. S. Y. a. A. S. P. Chen, ed.,Proceedings of the International Conference on Data Engineering (ICDE), IEEE Com-puter Society Press, Taipei, Taiwan, 1995, pp. 3-14.3 A. P. Dempster, N. M. Laird and D. B. Rubin, Maximum likelihood from incomplete datavia the EM algorithm, Journal Royal Statist. Soc. B, 39 (1977), pp. 1-38.4 E. Han, G. Karypis, V. Kumar and B. Mobasher, Hypergraph Based Clustering in High-Dimensional Data Sets: A Summary of Results, IEEE Data Engineering Bulletin, 21 (1998), pp. 15-22.5 J. Herlocker, J. KONSTAN, A. BORCHERS and J. RIEDL, An Algorithmic Frameworkfor Performing Collaborative Filtering, Proceedings of the 22nd ACM Conference on Re-searchand Development in Information Retrieval (SIGIR'99), Berkeley, CA., 1999.6 J. L. Herlocker, J. A. Konstan, L. G. Terveen and J. T. Riedl, Evaluating collaborative fil-tering recommender systems, ACM Transactions on Information Systems (TOIS), 22 (2004), pp. 5 - 53.7 X. Jin, Y. Zhou and B. Mobasher, A Unified Approach to Personalization Based on Prob-abilistic Latent Semantic Models of Web Usage and Content, Proceedings of the AAAI 2004 Workshop on Semantic Web Personalization (SWP'04), San Jose, 2004.8 T. Joachims, D. Freitag and T. Mitchell, Webwatcher: A tour guide for the world wideweb, The 15th International Joint Conference on Artificial Intelligence (ICJAI'97), Na-goya, Japan, 1997, pp. 770-777.9 J. Konstan, B. Miller, D. Maltz, J. Herlocker, L. Gordon and J. Riedl, Grouplens: Apply-ing Collaborative Filtering to Usenet News, Communications of the ACM, 40 (1997), pp.77-87.10 B. Mobasher, Web Usage Mining and Personalization, in M. P. Singh, ed., PracticalHandbook of Internet Computing, CRC Press, 2004.11 B. Mobasher, H. Dai, M. Nakagawa and T. Luo, Discovery and Evaluation of AggregateUsage Profiles for Web Personalization, Data Mining and Knowledge Discovery, 6 (2002), pp. 61-82.12 M. Perkowitz and O. Etzioni, Adaptive Web Sites: Automatically Synthesizing WebPages., Proceedings of the 15th National Conference on Artificial Intelligence, AAAI, Madison, WI, 1998, pp. 727-732.13 G. Xu, Y. Zhang and X. Zhou, A Latent Usage Approach for Clustering Web Transactionand Building User Profile, The First International Conference on Advanced Data Mining and Applications (ADMA 2005), Springer, Wuhan, china, 2005, pp. 31-42.14 G. Xu, Y. Zhang and X. Zhou, Using Probabilistic Semantic Latent Analysis for WebPage Grouping, 15th International Workshop on Research Issues on Data Engineering: Stream Data Mining and Applications (RIDE-SDMA'2005), Tokyo, Japan, 2005.。