An approach for evaluating User Model Data in an interoperability scenario

An approach for evaluating User ModelData in an interoperability scenarioCarmagnola Francesca a1, Cena Federica aa Department of Computer Science, University of Torino, ItalyAbstract.Since more and more interactions take place between humans and differentuser-adapted applications in daily life, there is the great opportunity to acquire a lot ofknowledge about a user and use it in order to reach a better adaptation. A common visionstates that user model should be interoperable and shareable across user-adaptiveapplications in different contexts and environments. The core idea of our approach is thatthe interoperability of user model data lets to reach more effective adaptation results only ifthe exchanged data are reliable. In this paper we discuss how this goal can be achieved by asemantic annotation of the exchanged data, providing the requestor with the possibility toevaluate i) the reliability of the exchanged value and ii) the reputation of the providersystem. To do this, we propose a User-Adapted System Meta-Description which identifies auser-adapted system enriched with a set of semantic meta-information regarding theprovider system, the exchanged value, the reasoning strategies used to define the value.Keywords. User model, user modeling, interoperability, user-adapted system, semanticwebIntroductionNowadays the idea of personalisation is regarded as crucial in many areas, such as e-commerce, e-learning, tourism and cultural heritage, digital libraries, travel planning, interaction in instrumented environments, etc. As a consequence a large number of user-adapted systems have been developed. Typically, user-adapted applications build a model of the user (e.g. profiling user with information entered by herself upon registration, clustering user in stereotype categories, tracking user behaviour and reasoning about her interests and competencies, learning from the dialog with the user, allowing the user to inspect and change the model the system built, etc) [3]. Then, the systems implement some reasoning strategies, defined as heuristic rules, decision trees, Bayesian networks, production rules, inductive reasoning, etc., to derive user knowledge, update the model and decide about adaptation strategies based on the model.Since users can interact with a great number of personalised systems, there is the great opportunity to share user knowledge across applications to obtain a higher understanding of the user [8]. Thus, sharing user knowledge across applications leads to better adaptation results. This is due to the “increased coverage” which means that1Corresponding Author: Francesca Carmagnola, Department of Computer Science, Corso Svizzera 185, 10149, Torino, Italy; E-mail: carmagnola@di.unito.itmore knowledge can be covered by the aggregated user model, because of the varietyof the contributing systems [9].The interoperability of user model data among user-adapted applications that interact with the same user can be useful in several situations, e.g.:− the system is non able to acquire a user feature by itself (neither asking/observing the user, nor inferring this knowledge from other data); consequently the adaptation process may be incomplete;− the system provides a value for the user feature but this value can be reliable only after a lot of sessions of interaction; consequently the adaptation process may face the “cold start problem” [16] (because at the beginning of the interaction the user model has very few data about the user);− the system provides a value for the user feature but this value is not reliable since it is inferred through a weak process of inference, with a low level of certainty;consequently the adaptation process may be incorrect.In all these situations the reusability of distributed user knowledge constitutes the central point for the definition of a complete user model representation. Reusability and interoperability have been in the focus of recent attempts to develop architectures that integrate user-adaptive components. There are two major directions to address reusability. Generic user modeling systems and user model servers (e.g. [12]; [11]; [4]) have made a significant advancement by offering flexible server-client architectures where a user model, stored as a central repository, is maintained by and shared across several applications. The second major advancement is the development of service-oriented architectures for user-adaptive systems (e.g.[2]; [10]). They enable complex personalization functionality to be implemented based on simple blocks (services) that implement specific adaptation procedures.The next major step towards user modeling reusability is semantic-awareness, which draws upon Semantic Web technologies that enable information exchange, aggregation, and interoperability [13]. Initial steps have already been made by [4] that proposed an ontology-based distributed architecture for adaptation in an e-learning scenario, while [9] proposed a framework for ubiquitous user modeling focused on the exchange and the semantic integration of partial user models. Therefore all the above-mentioned approaches foresee the possibility of exchanging user model features across adaptive systems in order to acquire more knowledge abouta user. From this point of view the applications can be providers when they partake of user model data, and requestors when they acquire this shared user knowledge in orderto perform their adaptation goals.The challenge of exchanging user model data among applications leads to many issues, which regard:a) the modalities to make the exchange of knowledge among applications possible.Thus: how can the exchange of user knowledge be achieved?b) the evaluation of the user model data exchanged by the applications. Thus: howcan an application be sure that the user model values provided by another applications are reliable? How can it be sure that the applications themselves arereliable? [14].Our approach moves from these considerations; however in this paper we give an attempt to especially address the second issue. The aim is to demonstrate that the usefulness of exchanging user model data across user-adapted systems is related with the chance of evaluating the trustworthiness of i) the reputation of the provider system, and ii) the reliability of the exchanged value.We state that exchanging only the value of the requested user feature is not sufficient, in a scenario of interoperability since it does not allow the evaluation of i) and ii). As defined in [15] we need a protocol for encoding information about users, to allow user-adapted system to benefit from others. For this reason we introduce the notion of User-Adapted System Meta-Description, which contains a set of meta-information about the system that provides the user feature, and about the exchanged user feature value. The entire set of the meta-information are provided by each application that want to exchange user knowledge, and are stored in a public registry we called Trustworthiness Evaluation Registry (TER).In the following section we illustrate the User-Adapted System Meta-Description providing a specification of the intended meta-information; in Sec. 2 we describe the Trustworthiness Evaluation Registry; while in Sec. 3 we provide an example of a possible use of the TER by means of a real use case. The last section concludes the paper with the future steps of our work.1. The User-Adapted System Meta-DescriptionAs stated in the previous section, in order to evaluate the reputation of the provider-system and the value reliability, we define the User-Adapted System Meta-Description,a meta-representation that provides information regards the user-adapted system.[3] defines a user-adapted system as a system that creates “an explicit user model that represents user knowledge, goals, interests, and other features that enable the system to distinguish among different users. [..] The user model is used to provide an adaptation effect, that is tailor interaction to different users [..]”.Fig. 1 User-Adapted System Meta-DescriptionAs shown in Fig 1, the User Model used by user-adapted systems is composed by a User Model Component and by a User Modeling Component [17]. [17] defines these concept as:Definition 1.1 (User Model Component) A user model is a knowledge source in a system which contains explicit assumptions on all the aspects of the user that may be relevant to the behavior of the system.The User Model Component, in the Semantic Web approach, is usually enriched by the use of ontologies. As pointed out in [7], ontologies provide a common understanding of a domain that can be shared between people and spread systems. Since in the field of artificial intelligence the ontologies have been developed to facilitate knowledge sharing and reuse, they should have an important role for the task of exchanging user knowledge [9].Definition 1.2 (User Modeling Component) A User Modeling Component is that part of a system whose function is to incrementally construct a user model; to store, update and delete entities; to maintain the consistency of the model; and to supply other components of the system with assumptions about a user.To summarize, the User Model Component includes the list of user features with the correspondent values; while the User Modeling Component includes the list of reasoning strategies to both derive user knowledge and manage the model.The core idea of our approach is that the interoperability of user model data lets to reach more effective adaptation results only if the exchanged data are reliable. This goal can be achieved by a semantic annotation of the exchanged data, providing the requestor with the possibility to evaluate i) the reliability of the exchanged value and ii) the reputation of the provider system. To do this, we propose a User-Adapted System Meta-Description which identifies a user-adapted system enriched with a set of meta-information regarding:1. the provider systemThis first set of meta-information is aimed at giving the requestor more knowledge about the provider system. In particular they include both information for the identification of the provider and information related with how it produces the user data it supplies. Notice that the evaluation of the provider derives by both two types of information; information regarding its identification can be used also for make the communication between systems possible.2. the user feature and the correspondent value (meta-information for the User ModelComponent)3. the reasoning strategies used to define the user feature value (meta-information forthe User Modeling Component)The second and the third set of meta-information allow the requestor to acquire more knowledge about the requested user feature value.The introduction of the semantic meta-information about the User Modeling Component is motivated by the idea that the reasoning strategies used in deriving the value need to be known by the requestor for a complete evaluation of the trustworthiness of the user model value [5]. We consider all the aspects as endorsement,thus reasons for believing or disbelieving statement, as Cohen [6] states in his model2.[6] defines the endorsements as “the structured objects that represent reasons for believing or disbelieving the proposition to which they are associated”. In the same way, in our approach we state that the evaluation of the final value depends on the evaluation of the intermediate values that lead to the value.The following table shows a detailed description of the intended meta-information.META-INFORMATION3Definition1. Meta-information for the identification of theprovider<dc:publisher> The entity responsible for making the resourceavailable<mum:url> Url of the provider systemMeta-information about the production of theprovided data<dc:creator> System that stores the user value<dc:date> Date of creation of the system<dc:source> Source that created the system2. Meta-information for the evaluation of thevalue -User Model Component-<dc:identifier> Un unambiguous reference to the resource within agiven context<dc:subject> The topic of the content of the resource<mum:certainty> Certainty value of the feature<dc:date> Date of creation of the feature<mum:last_update> Date of the last update of the featuretime span of how long the value is<mum:temporal_validity> Thequantitativevalidquantitative measure of how the value of the<mum:interaction_dependency> Thefeature is related to the interaction of user with system 3. Meta-information for the evaluation of thevalue -User Modeling Component-2The model of endorsement is an AI approach to represent and reason with heuristic knowledge under uncertainty. It discriminates kinds of evidence and distinguishes the importance of different evidence-gathering situations.3The definition of the meta-information is based on Dublin Core (/documents/dces), a set of metadata elements for describing document-like network object or for locating information resources. We extend it with some elements necessary in our approach defining the <mum> namespace (meta user model).<mum:engine> InferenceEngine used to compute data<mum:language> Languageusedto express derived knowledge<mum:reasoning> Methodology used to derive knowledge starting fromgiven premises<mum:derivation><mum:derivation>imported</mum:derivation> <mum:derivation>observed</mum:derivation> <mum:derivation>declared</mum:derivation> <mum:derivation>inferred</mum:derivation> Modality of production of the user feature- Imported: come from other providers- Observed: observed by user behavior with the system -Declared: directly declared by the user- Inferred: inferred by other featuresTab.1 List of meta-informationFollowing our approach the requestor receives, together with the requested user features value, the whole set of the above described meta-information. But how can it use them?In our perspective each requestor is furnished with a set of heuristics that need to be applied to the i) meta-information about the system, and ii) the meta-information about User Model and User Modeling Components to define respectively i) the system reputation, and ii) the value reliability.It is hardly possible to provide examples of heuristics that can be valid for all the applications, since their definition is closely dependent by the individual choices of each system. In fact, as claimed by [6], “any domain will have a characteristic set of endorsements”. For instance:Type 1: concerning the attribute <dc:date>, one application can affirm that “the trustworthiness increases if the provider system has been created recently” because it probably uses more updated technology” (recentness as a positive endorsement), while another may think that the older is the system , the more its user feature values are reliable because they derive from a longer-period interaction with the user (recentness as a negative endorsement).Type 2: concerning the attribute <dc:last_update>, one system can state that “the trustworthiness is high if the last update of the provided user feature is recent”, while for another system with different goals the evaluation of the endorsements and the heuristics can be different.Type 3: concerning the attribute <mum:derivation>, one application can believe that if derivation is “observed” then the trustworthiness in user feature value is high; if derivation is “imported” then the trustworthiness in user feature value is low. On the contrary another system, which has different experiences, may have different reputations and believes and thus it can define different heuristics.For the reasons highlighted in the above examples we don’t define a generic set of heuristics to be used in all the systems but we state that every system has to define its own set of heuristics.Actually by means of the heuristics applied on the meta-information each requestor will be able not only to identify the provider system and the provided value, but also to evaluate if both the system reputation and the value reliability are enough high to use the imported datum to perform its adaptation goals.2. The Trustworthiness Evaluation RegistryAll the above described meta-information are not included in the repository of each provider, but they are included in what we called TER (Trustworthiness Evaluation Registry), which is published in the network.The idea at the basis of the registry is that all the user-adapted systems that want to take part in the process of exchanging user knowledge should register themselves in the TER and provide all the meta-information required.To allow a common understanding of the exchanged user features, a requirement is that the user knowledge in all the systems of the environment use the same user ontology (for more detailed motivations see [5]). Moreover, each application has to decide which user features can be defined as public and recorded in the TER, and thus exchanged across applications.This choice is functional to the management of privacy regulations in the exchange of user sensitive data.4The requestor, that looks for the value of the feature x for the specific user X, queries the registry. The TER gives as answer not only the searched value, bur also the full set of the meta-information. The meta-information is defined in RDF5 since it lets to specify semantics for data in a standardized interoperable manner.Below is reported a portion of the TER.<?xml version="1.0"?><rdf:RDF xmlns:rdf="/1999/02/22-rdf-syntax-ns#"xmlns:dc= "/dc/elements/1.1/"><rdf:RDF xmlns:rdf="/1999/02/22-rdf-syntax-ns#"xmlns:mum="http://www.di.unito.it/MUM/mum#"><rdf:Description rdf:about="x"><um:value>any</um:value></<um:certainty>any um:certainty><dc:description>any</dc:description>or><dc:creat any</dc:creator><dc:date>any</dc:date><dc:publisher>any</dc:publisher><dc:source>any</dc:source><mum:url>any</mum:url><dc:title>x</dc:title><dc:subject>x</dc:subject><dc:identifier>#UMFeaturex/html</dc:identifier><mum:user_id>#X</mum:user_id><mum:last_update>any</mum:last_update><mum:temporal_validity>any</mum:temporal_validity><mum:interaction_dependency>any</mum:interaction_dependency>ng><mum:reasoni any</mum:reasoning></<mum:engine>any mum:engine><mum:language>any</mum:language><mum:derivation>inferred</mum:derivation><mum:input_parameter>input1</mum:input_parameter>4A different solution is proposed in [9] where the problem of privacy in the exchange of user data is managed through the introduction of the “Privacy Box” which contains a set of meta attributes for the acceptance of sensitive data.5 A semantic data model that supports the interoperability among system(/RDF/)<um:value>any</um:value></u<um:certainty>any m:certainty><mum:last_update>any</mum:last_update><mum:temporal_validity>any</mum:temporal_validity><mum:interaction_dependency>any</mum:interaction_dependency><mum:derivation>any</mum:derivation><mum:input_parameter>input2</mum:input_parameter><um:certainty>any</um:certainty><mum:last_update>any</mum:last_update><mum:temporal_validity>any</mum:temporal_validity><mum:interaction_dependency>any</mum:interaction_dependency><mum:derivation>any</mum:derivation></rdf:Description></rdf:RDFBeside the information needed by the requestor to evaluate the system reputation and the value reliability, some further meta-information are contained in the TER in order to provide the requestor with addition information useful to support communication (e.g. for the identification of the provider <Url>, the feature and its value <Title, Value, Identifier>, and correspondent user <Subject>).Notice that if <mum:derivation> is “inferred”, the registry will contain also the input parameters (thus other user features) that compete in the definition of the requested output in order to allow the requestor to evaluate them as well (according to the above-presented model of endorsement). For all the input components, the TER provides a sub-set of the meta-information above-described, and in particular the meta-information functional to the application of the heuristics. To this purpose, <mum:temporal_validity> and <mum:interaction_dependency> are particular relevant since they let the requestor to understand if the inferred feature derives from features i) valid in the particular moment of the query, and ii) not too much close to the context of interaction and so hardly reusable in different situations (for instance because too strongly related to the used device). Notice that the process of the input parameters evaluation will be obviously repeated every time <mum:derivation> of each input parameter is set as “inferred”.3. Use case of use of the registry by an applicationConsider the use case of an adaptive system (S1) that provides Carlo with a personalised list of TV programs. It needs to know Carlo’s ability to read6, in order to present him content he is able to read. Having no means of collecting this information, it has to import this value from another system Carlo interacted with.To do that, S1 has to perform the following activities:step 1. search for the requested feature into the TERstep 2. evaluate the system reputation (step 2.1) and the value reliability (step 2.2).6It is a class of GUMO ontology () and identifies the physical capacity to read.Step 1. S1 accesses the TER to obtain the value for the requested user feature. It asks to select “ability to read” where “user ID=12” using SPARQ language7 to query the RDF.PREFIX mum: <http:// www.unito.it/MUM/mum#/>SELECT $AbilityToRead mum:valueWHERE {:12 mum:user_id $user_id}As answer to the query, the TER provides S1 with a portion of RDF code which contains the requested user feature value, together with the list of the correspondent meta-information. All the data come from UbiquiTO [1], an adaptive tourist guide Carlo is used to interact with.In the following the portion of TER registry about “ability to read” is reported.<?xml version="1.0"?><rdf:RDF xmlns:rdf="/1999/02/22-rdf-syntax-ns#"xmlns:dc= "/dc/elements/1.1/"><rdf:RDF xmlns:rdf="/1999/02/22-rdf-syntax-ns#"xmlns:mum="http://www.di.unito.it/MUM/mum#"><rdf:Description rdf:about="/#AbilityToRead"><mum:value>0.4</um:value><mum:certainty>0.6</um:certainty><dc:description> physical capacity to read</dc:description><dc:title> AbilityToRead </dc:title><dc:creator>UbiquiTO</dc:creator><dc:date>2001-09-01</dc:date><dc:publisher>CSP</dc:publisher><dc:source>CSP</dc:source><mum:url>http://www.ubiquito.it</mum:url><dc:identifier>#UMFeature11</dc:identifier><dc:subject>AbilityToRead</dc:subject><mum:user_id>user#ID12:user_id></mum</mum:<mum:last_update>2005-10-01last_update><mum:temporal_validity>2006-10-01</mum:temporal_validity>n_dependency><mum:interactio high</mum:interaction_dependency>ng><mum:reasoni production rule</mum:reasoning><mum:engine>jess v.6.1</mum:engine><mum:language>java expert system</mum:language><mum:derivation>inferred</mum:derivation><mum:input_parameter>Age</mum:input_parameter><mum:value>60</um:value><mum:certainty>1.0</um:certainty><mum:last_update>2005-10-01</mum:last_update><mum:temporal_validity>one year</mum:temporal_validity><mum:interaction_dependency>none</mum:interaction_dependency><mum:derivation>declared</mum:derivation><mum:input_parameter Screen Size</mum:input_parameter>></<mum:value>small um:value></um:cert<mum:certainty>0.8ainty><mum:last_update>2005-10-01</mum:last_update><mum:temporal_validity>any</mum:temporal_validity><mum:interaction_dependency>any</mum:interaction_dependency><mum:derivation>inferred</mum:derivation><mum:input_parameter>Device</mum:input_parameter><mum:value>PDA</um:value><mum:certainty>0.8</um:certainty>7 /TR/rdf-sparql-query/<mum:last_update>2005-10-01/mum:last_update><<mum:temporal_validity>a day m:temporal_validity></mu<mum:interaction_dependency>high</mum:interaction_dependency><mum:derivation>observed</mum:derivation></rdf:Description></rdf:RDFStep 2. The requestor, applying its own heuristics to the meta-information, rates the trustworthiness of the user feature value calculating both the system reputation (SR) and the value reliability (VR) (step 2.2). The following table shows a mirror of the heuristics (expressed in natural language) used by S1.S1 Heuristics Trust computationSystem rating (step 2.1)If <dc:creator> is known, then SR is 0.8 else 0.2UbiquiTO: known then SR 0.8If <dc:date> is recent, then SR is 0.7 else 0.3If <dc:source> is known, then SR is 0.8 else 0.2 2001-09-01: not recent then SR0.7 CSP: known then SR 0.8Value rating ( step 2.2- a)If <mum:certainty> is > “0.6” then VR is 0.7 else0.30.6: < 0.7 then VR 0.3If <mum:last_update> is recent then VR 0.8 else0.22005-10-01: not recent then VR 02If current date is before <mum:temporal_validity> then VR 0.6 else 0.4If <mum:interaction_dependency> is “high” then VR 0.2, if medium then 0.3 else 0.5 2006-10-01: before current date then VR 0.6 Interaction_dependency: high then VR 0.2Reasoning strategies rating ( step 2.2- b)If <mum:derivation> is “declared” then VR is 0.4, if“observed” then VR 0.4, if “imported” then VR 0.1else VR 0.1Derivation: inferred then VR 0.1Tab.2 A list of heuristics used by S1Being the <mum:derivation> “inferred”, S1 has to evaluate all input parameters that compete to the definition of the requested output. The value of ability to read depends by the values of two other features: user “Age” and device “Screen Size”. While in UbiquiTO the age is declared by Carlo himself, the screen size is inferred by the device he uses during the interaction with the system. Following our approach, S1 has to apply its heuristics to these features as well, in order to compute a final overall value that considers the meta-information associated with each input parameter.4. Conclusion and Future workThis research represents the subject of the two Ph.D thesis of the authors. They focus on investigating the issues a) and b) outlined in the introduction, thus: (a) how can the exchange of user knowledge be achieved (b) how can the exchanged user feature value be evaluated.。