Bogers, T., Thoonen, W., & van den Bosch, A. (2006). Expertise classification: Collaborative classification vs. automatic extraction. Social Classification: Panacea or Pandora? 17th Annual SIG/CR Classification Research Workshop. Retrieved from http://www.slais.ubc.ca/users/sigcr/sigcr-06bogers.pdf.
Guy, M., & Tonkin, E. (2006). Folksonomies: Tidying up Tags? D-Lib Magazine, 12(1). Retrieved November 10, 2006, from http://www.dlib.org/dlib/january06/guy/01guy.html.
Iverson, L. (2006). Thomas Vander Wal on Folksonomy. Blog posting. Retrieved from http://www.ece.ubc.ca/~leei/weblog/2006/03/thomas_vander_wal_on_folksonom.html.
Lin, X., Joan, B., Yen, B., et al. (2006). Exploring characteristics of social classification. Social Classification: Panacea or Pandora? 17th Annual SIG/CR Classification Research Workshop. Retrieved from http://www.slais.ubc.ca/users/sigcr/sigcr-06lin.pdf.
Long, K. (2005). Leveraging folksonomy - flickr clusters at ExperienceCurve. Blog posting. Retrieved November 10, 2006, from http://blog.experiencecurve.com/archives/leveraging-folksonomy-flickr-clusters.
Marlow, C., Naaman, M., boyd, D., et al. (2006). HT06, Tagging Paper, Taxonomy, Flickr, Academic Article, . Hypertext 06. Retrieved from http://www.danah.org/papers/Hypertext2006.pdf.
Sen, S., Shyong K., L., Cosley, D., et al. (2006). tagging, community, vocabulary, evolution. Proceedings of CSCW 2006. Retrieved from http://www.grouplens.org/papers/pdf/sen-cscw2006.pdf.
Tonkin, E. (2006). Searching the long tail: Hidden structure in social tagging . Social Classification: Panacea or Pandora? 17th Annual SIG/CR Classification Research Workshop. Retrieved from http://www.slais.ubc.ca/users/sigcr/sigcr-06tonkin.pdf.
Vander Wal, T. (2005). Explaining and Showing Broad and Narrow Folksonomies :: Personal InfoCloud. Blog posting. Retrieved November 13, 2006, from http://www.personalinfocloud.com/2005/02/explaining_and_.html.
Monday, November 13, 2006
Wednesday, October 25, 2006
Information seeking, teachers and what a digital repository could offer?
In context of digital learning repositories we often times offer users a few ways to search learning resources without thinking much of why and for what purpose they would be using the repository.
Too many times repositories are developed with a glorious idea "if we build it, they will come". But to start with, do we really ask teachers whether they need a repository for digital learning resources?
Building a repository or a federation of them without asking this "raison d'être" is like creating a product to sell without a need or demand for it. That happens often, without doubt, but with the difference that there is a big marketing budget to create the need. Thus, teachers are stuck with bad looking search interfaces that hide potentially interesting technology without any needs and desires to use it.
To backtrack a bit, one should look at what teachers are doing in their work and how they are doing it. Ok, they teach, right. They have a goal, usually laid out in a curriculum, that they are set out to fulfill. They might have different requirements for the material to use; in some countries teachers have more freedom on choosing educational material than in others. To facilitate their job, teachers like to use the material that they are comfortable with and know well. They might use ready-made lesson plans by school book publishers or prefer to create their own material from bits and pieces.
Moreover, there are the routines to save time and efforts - information seeking, like learning, is a fundamental and high level cognitive process (Marchionini, 1995). People like to resort to something that they know does the job. Teachers are social animals, too, maybe a few hints from a colleague will settle a teacher for the day's task. Thus, the needs are rather contextualised (the given age, curriculum) and socially supported (accepted and negotiated educational practices, hints from colleague teachers,.. ).
Where does the digital learning material stand in this picture and how do teachers see the need for using a repository to better do their teaching? On the long term we are interested in looking at teachers' tasks in their teaching and see how digital learning resources repositories could better support teachers in their daily quest to make the learners better learners. Two main areas are concerned, namely information seeking and social information retrieval. We will argue that digital repositories should do more to offer social and contextual support for teachers and learners to discover relevant resources from a repository. But this blab is about looking into information seeking and the review on existing literature.
Information seeking, teachers and what a digital repository could offer?
When we think of a teacher preparing for a new lesson, we can think that it is part of larger information seeking behaviour, which is contextually driven, i.e. there is the national or regional curriculum, its topics, goals and learning activities to fulfill. The current information seeking research defines information seeking as a conscious effort to acquire information in response to a need or gap in your knowledge (Case, 2002).
For a teacher there is a diversity of support material available to fulfill the task, some of it being paper-based teaching material, some digital and others might relay on human resources. A teacher might also address a digital learning repository for the purpose of finding suitable digital learning resources. This teacher might already know exactly what he is looking for, a piece of material that he has seen before, or he might look for some motivational piece of information or some assessment material, for example.
However, often times digital repositories are not well prepared to serve different individual tasks that teachers might have at hand when they come to a repository. For that reason, more job-level analyses would be needed to look into individual tasks that teachers are to perform when they are using a learning resources repository. In general, the repositories have very little observation on patterns across tasks and contexts. So we ask, what are those general patterns that we can find across tasks that teachers are set out to perform at a digital repository?
Moreover, information seeking, in some cases, can be a social activity. Wilson (2005), for example claims that more information is communicated by word of mouth than is ever retrieved from databases. Many other researchers in the field of information seeking talk about its social layer. Hargittai & Hinnant (2006), who lay out a social framework for information seeking, argue that an “important factor influencing users’ information-seeking behavior concerns the availability of social support networks to help address users’ needs and interests. People’s information behavior does not happen in isolation of others.“
Thus, when we are looking into teachers information seeking tasks at the learning repository, it becomes important to think of the support for such social networks to tap onto. We can think of these networks in two different ways, as human resources themselves (such as getting in touch with an expert in a given field, etc) or as secondary support to help the teacher to find the suitable resource for the lesson. Or, like Peter Morville (2004) says; We use people to find content. We use content to find people. Information seeking behavior and social network analysis go hand in hand.
Information seeking ranges from forming question to gathering, synthesising and using information. It is usually cyclic and iterative process from seeking to gathering, refining questions, to evaluating and synthesising information to using it. A holistic view of information seeking process comes near to ideas of inquiry learning, both emphasising an iterative question-driven process of finding, managing and evaluating information. (Lallimo, et al. 2004).
Moreover, Kuhlthau (Wilson, 2004) also talks about search process in similar terms as educationalists, introducing the notion of the 'Zone of Intervention', similar to Vygotsky's Zone of Proximal development (Vygotsky, 1978), where the learner, when engaging in collaborative problem-solving with a guidance of an adult or more experienced student, can reach better level than without. Kuhlthau talks about five intervention zones, in some of which the advancement is dependent of collaborating with others, such as the librarian providing the quick reference or someone helping discovering potentially useful information resources.
We should further investigate how these zones of interventions could be supported in a digital learning repository when a teacher is looking for learning resources, or when a learner is there with his own information seeking intention to attain a task. We are interested in looking into supporting users in different ways, by designing better tools and interfaces, but also to build in support from fellow users.
This support could appear in different ways, such as "leaving traces" of information seeking patterns by other users or by creating social connections between users where they did not previously exist. The following can be envisaged: leveraging the previous search histories of other users; tapping into similarities in interest displayed by bookmarking action; looking into subjective relevance judgements such as annotations, tagging and end-user evaluations and ratings. Moverover, the use of existing social networks (such as expressed by using FOAF or FXML) should be supported, but more importantly, also the emerging ones, that could be detected by using social network analysis should be investigated.
---------
Case, D.O. (2002). Looking for informaiton: A Survey of Research on Information Seeking, Needs and Behavior, San Diego: Academic Press.
Hargittai, E. and Hinnant, A. (2006). Toward a Social Framework for Information Seeking. In New Directions in Human Information Behavior by Amanda Spink and Charles Cole.
Järvelin, K., Ingwersen, P. (2004). Information seeking research needs extension towards tasks and technology. Information Research, 10(1) paper 212 [Available at http://InformationR.net/ir/10- 1/paper212.html]
Lallimo, J., Lakkala, M. and Paavola, S. (2004) How to Promote Students' Information Seeking? ERNIST Answers archive, European Schoolnet.
Marchionini, G. (1995). Information Seeking in Electronic Environments, Cambridge, UK: Cambridge University Press.
Morville, P. (2004). Ambient Findability. http://www.digital-web.com/articles/ambient_findability/
Vygotsky, L.S. (1978). Mind and society: The development of higher mental processes. Cambridge, MA: Harvard University Press.
Wilson, T.D. (2004) Review of: Kuhlthau, C.C. Seeking meaning: a process approach to library and information services. 2nd. ed. Westport, CT: Libraries Unlimited, 2004. Information Research, 9(3), review no. R129 [Available at: http://informationr.net/ir/reviews/revs129.html]
Wilson, T.D. (2005). Review of: Ingwersen, P. and Järvelin, K. The turn: integration of information seeking and retrieval in context. Dordrecht, The Netherlands: Springer, 2005. Information Research, 11(1), review no. R189 [Available at: http://informationr.net/ir/reviews/revs189.html]
Links to other things I'm reading about the topic: http://www.furl.net/members/vuorikari/info_seeking
Too many times repositories are developed with a glorious idea "if we build it, they will come". But to start with, do we really ask teachers whether they need a repository for digital learning resources?
Building a repository or a federation of them without asking this "raison d'être" is like creating a product to sell without a need or demand for it. That happens often, without doubt, but with the difference that there is a big marketing budget to create the need. Thus, teachers are stuck with bad looking search interfaces that hide potentially interesting technology without any needs and desires to use it.
To backtrack a bit, one should look at what teachers are doing in their work and how they are doing it. Ok, they teach, right. They have a goal, usually laid out in a curriculum, that they are set out to fulfill. They might have different requirements for the material to use; in some countries teachers have more freedom on choosing educational material than in others. To facilitate their job, teachers like to use the material that they are comfortable with and know well. They might use ready-made lesson plans by school book publishers or prefer to create their own material from bits and pieces.
Moreover, there are the routines to save time and efforts - information seeking, like learning, is a fundamental and high level cognitive process (Marchionini, 1995). People like to resort to something that they know does the job. Teachers are social animals, too, maybe a few hints from a colleague will settle a teacher for the day's task. Thus, the needs are rather contextualised (the given age, curriculum) and socially supported (accepted and negotiated educational practices, hints from colleague teachers,.. ).
Where does the digital learning material stand in this picture and how do teachers see the need for using a repository to better do their teaching? On the long term we are interested in looking at teachers' tasks in their teaching and see how digital learning resources repositories could better support teachers in their daily quest to make the learners better learners. Two main areas are concerned, namely information seeking and social information retrieval. We will argue that digital repositories should do more to offer social and contextual support for teachers and learners to discover relevant resources from a repository. But this blab is about looking into information seeking and the review on existing literature.
Information seeking, teachers and what a digital repository could offer?
When we think of a teacher preparing for a new lesson, we can think that it is part of larger information seeking behaviour, which is contextually driven, i.e. there is the national or regional curriculum, its topics, goals and learning activities to fulfill. The current information seeking research defines information seeking as a conscious effort to acquire information in response to a need or gap in your knowledge (Case, 2002).
For a teacher there is a diversity of support material available to fulfill the task, some of it being paper-based teaching material, some digital and others might relay on human resources. A teacher might also address a digital learning repository for the purpose of finding suitable digital learning resources. This teacher might already know exactly what he is looking for, a piece of material that he has seen before, or he might look for some motivational piece of information or some assessment material, for example.
However, often times digital repositories are not well prepared to serve different individual tasks that teachers might have at hand when they come to a repository. For that reason, more job-level analyses would be needed to look into individual tasks that teachers are to perform when they are using a learning resources repository. In general, the repositories have very little observation on patterns across tasks and contexts. So we ask, what are those general patterns that we can find across tasks that teachers are set out to perform at a digital repository?
Moreover, information seeking, in some cases, can be a social activity. Wilson (2005), for example claims that more information is communicated by word of mouth than is ever retrieved from databases. Many other researchers in the field of information seeking talk about its social layer. Hargittai & Hinnant (2006), who lay out a social framework for information seeking, argue that an “important factor influencing users’ information-seeking behavior concerns the availability of social support networks to help address users’ needs and interests. People’s information behavior does not happen in isolation of others.“
Thus, when we are looking into teachers information seeking tasks at the learning repository, it becomes important to think of the support for such social networks to tap onto. We can think of these networks in two different ways, as human resources themselves (such as getting in touch with an expert in a given field, etc) or as secondary support to help the teacher to find the suitable resource for the lesson. Or, like Peter Morville (2004) says; We use people to find content. We use content to find people. Information seeking behavior and social network analysis go hand in hand.
Information seeking ranges from forming question to gathering, synthesising and using information. It is usually cyclic and iterative process from seeking to gathering, refining questions, to evaluating and synthesising information to using it. A holistic view of information seeking process comes near to ideas of inquiry learning, both emphasising an iterative question-driven process of finding, managing and evaluating information. (Lallimo, et al. 2004).
Moreover, Kuhlthau (Wilson, 2004) also talks about search process in similar terms as educationalists, introducing the notion of the 'Zone of Intervention', similar to Vygotsky's Zone of Proximal development (Vygotsky, 1978), where the learner, when engaging in collaborative problem-solving with a guidance of an adult or more experienced student, can reach better level than without. Kuhlthau talks about five intervention zones, in some of which the advancement is dependent of collaborating with others, such as the librarian providing the quick reference or someone helping discovering potentially useful information resources.
We should further investigate how these zones of interventions could be supported in a digital learning repository when a teacher is looking for learning resources, or when a learner is there with his own information seeking intention to attain a task. We are interested in looking into supporting users in different ways, by designing better tools and interfaces, but also to build in support from fellow users.
This support could appear in different ways, such as "leaving traces" of information seeking patterns by other users or by creating social connections between users where they did not previously exist. The following can be envisaged: leveraging the previous search histories of other users; tapping into similarities in interest displayed by bookmarking action; looking into subjective relevance judgements such as annotations, tagging and end-user evaluations and ratings. Moverover, the use of existing social networks (such as expressed by using FOAF or FXML) should be supported, but more importantly, also the emerging ones, that could be detected by using social network analysis should be investigated.
---------
Case, D.O. (2002). Looking for informaiton: A Survey of Research on Information Seeking, Needs and Behavior, San Diego: Academic Press.
Hargittai, E. and Hinnant, A. (2006). Toward a Social Framework for Information Seeking. In New Directions in Human Information Behavior by Amanda Spink and Charles Cole.
Järvelin, K., Ingwersen, P. (2004). Information seeking research needs extension towards tasks and technology. Information Research, 10(1) paper 212 [Available at http://InformationR.net/ir/10- 1/paper212.html]
Lallimo, J., Lakkala, M. and Paavola, S. (2004) How to Promote Students' Information Seeking? ERNIST Answers archive, European Schoolnet.
Marchionini, G. (1995). Information Seeking in Electronic Environments, Cambridge, UK: Cambridge University Press.
Morville, P. (2004). Ambient Findability. http://www.digital-web.com/articles/ambient_findability/
Vygotsky, L.S. (1978). Mind and society: The development of higher mental processes. Cambridge, MA: Harvard University Press.
Wilson, T.D. (2004) Review of: Kuhlthau, C.C. Seeking meaning: a process approach to library and information services. 2nd. ed. Westport, CT: Libraries Unlimited, 2004. Information Research, 9(3), review no. R129 [Available at: http://informationr.net/ir/reviews/revs129.html]
Wilson, T.D. (2005). Review of: Ingwersen, P. and Järvelin, K. The turn: integration of information seeking and retrieval in context. Dordrecht, The Netherlands: Springer, 2005. Information Research, 11(1), review no. R189 [Available at: http://informationr.net/ir/reviews/revs189.html]
Links to other things I'm reading about the topic: http://www.furl.net/members/vuorikari/info_seeking
Tuesday, October 10, 2006
An interesting new acquaintance: the field of information seeking and retrieval
I read a review by T.D. Wilson of the book The turn: integration of information seeking and retrieval in context, and decided to buy it for future reading.
This book introduces a new field called"information seeking and retrieval", which combines two existing ones, namely the research in information seeking and information retrieval. I have a feeling that this is something important for my studies, as I am not only interested in information retrieval in the context of a LOR, but I think the information seeking task at hand has important implications.
The reviewer explains information seeking being
Another important idea from the book, that the reviewer underlined, is that research should not be too narrowly system-oriented - otherwise it might run into the risk of being development of technology with no carefully analyzed use contexts. That is something that I have to also keep in mind, not to be too focused on one system that I study, but keep my mind and door open for further applicability. Looking forward reading the book!
Furthermore, I was reading about the book called Ambient Findability by Peter Morville, that my promoter wanted to bring into my attention. In an article dating in 2004 with the same name (what a stupid name, btw) he goes like this:
Very interestingly, when he says "Information seeking behavior and social network analysis go hand in hand." makes me nod my head. Yes, that is the way it goes and we need tools that help those two to better work together, and add information retrieval into it, maybe from the cognitive approach, as Ingwersen and Järvelin suggest. But for that, I have to read the book to know more.
Wilson, T.D. (2005). Review of: Ingwersen, P. and Järvelin, K. The turn: integration of information seeking and retrieval in context. Dordrecht, The Netherlands: Springer, 2005. Information Research, 11(1), review no. R189 [Available at: http://informationr.net/ir/reviews/revs189.html]
This book introduces a new field called"information seeking and retrieval", which combines two existing ones, namely the research in information seeking and information retrieval. I have a feeling that this is something important for my studies, as I am not only interested in information retrieval in the context of a LOR, but I think the information seeking task at hand has important implications.
The reviewer explains information seeking being
concerned with the discovery of the appropriate information for tasks, research, everyday life, etc., regardless of the way that information is packaged.
Another important idea from the book, that the reviewer underlined, is that research should not be too narrowly system-oriented - otherwise it might run into the risk of being development of technology with no carefully analyzed use contexts. That is something that I have to also keep in mind, not to be too focused on one system that I study, but keep my mind and door open for further applicability. Looking forward reading the book!
Furthermore, I was reading about the book called Ambient Findability by Peter Morville, that my promoter wanted to bring into my attention. In an article dating in 2004 with the same name (what a stupid name, btw) he goes like this:
It is this subtle power of context that intrigues me in the realm of networked information environments. We use people to find content. We use content to find people. Information seeking behavior and social network analysis go hand in hand. In today’s knowledge economy, learning and finding are powered by all sorts of invisible links between and among people and documents.
Very interestingly, when he says "Information seeking behavior and social network analysis go hand in hand." makes me nod my head. Yes, that is the way it goes and we need tools that help those two to better work together, and add information retrieval into it, maybe from the cognitive approach, as Ingwersen and Järvelin suggest. But for that, I have to read the book to know more.
Wilson, T.D. (2005). Review of: Ingwersen, P. and Järvelin, K. The turn: integration of information seeking and retrieval in context. Dordrecht, The Netherlands: Springer, 2005. Information Research, 11(1), review no. R189 [Available at: http://informationr.net/ir/reviews/revs189.html]
Wednesday, October 04, 2006
Virtual co-learners may provide keys to faster, deeper learning
“The findings were that people learned much more with a supportive agent,” observes Nass. ”Some of our other findings indicate that a smarter co-learner (or agent), the one who gets the answer right, helps people learn more than dumber agents. It is clear that in any teaching or learning situation it is worthwhile to have a co-learner – someone else who appears interested.”
http://scil.stanford.edu/news/virtual10.htm
Yes, I want an agent based co-learner who would learn everything that I learn and never forget! Isn't it annoying when reading a text, half way through you say: well, I think I've read this text! That would never happen with the agent.
I already see myself having conversations with my agent: "Really, I know this already? So you mean I don't need to learn this anymore?".
In the future I might just kick back and let my virtual agent do all the communication and tackle the situations where I'm supposed to do something that I have already learnt.
Tuesday, October 03, 2006
school innovation
Learning styles are like a quick fix to understand something as complex as learning, and teaching, for that matter too. They are darlings of corporate training and something that managers look into when they are calculating ROI for professional learning and continuous training.
E-learning area seem to be the other domain where learning styles pop up often. Many times it is claimed that e-learning allows personalised learning, e.g. learner is presented with material that marches his/her learning styles. Commonly we see references to VAKT (Visual, auditory, kinaesthetic and tactile) or some dimensions like holistic vs. analytic or linear, etc.
I took a quick (fix) review on learning styles after a short discussion that I had with a colleague of mine. I, totally mistakenly (of course, not) mentioned something along the lines of learning styles, where my colleague mentioned "aren't they already so passe". Uuhmm, yeah, sure...
So I duck up some literature on the Web and realised: which learning style? There sure are many of them, Goffield et al. (2004), for example, identified 71 in the literature, out of which his team chose 13 most influential and potentially influential models of learning styles for a systematic and critical review.
After reading "Learning styles and pedagogy in post-16 learning;
A systematic and critical review" one becomes humble about quick assumptions regarding learning styles. Table 44. presents these 13 Learning styles models matched against minimal criteria that was used in the review (p.139). Findings...
Oookey, seems like there is really something in this area of learning styles that hints that one should be rather wary and critical about quick fixes. Moreover, the plethora of models in the area should probably ring a bell. Along Coffield et al.
Coffield et al. are truly critical about this field of research, however, they don't, all together, through it to the waste-basket. They actually endorse some of the models which, instead of simplifying learning styles as anything like "deep-sealed features of the cognitive structure" or "components of a relatively stable personality type", see them more related to "learning preferences" or "learning approaches, strategies, orientations and conceptions of learning". They embrace these tools to help learners to gain more self-awareness and become more familiar with their metacognition, e.g. how to enhance their learning.
also
All right, now we are getting somewhere. Seems like it would be acceptable to say that people have learning preferences or "individual dispositions which influence the reactions of learners to their learning opportunities, which include the teaching style of their teachers." According to Bloomer and Hodkinson (2000) dispositions are both psychological and social. It is notable, however, that these individual dispositions constitute only a minor part of what can effect on learning.
To enlighten other effects or intervention on learning, Hattie (1992, 1999) synthesised 630 studies. If individualised learning means offering learning according to students' learning styles, the average effect size is not significant for individualised teaching in schools (significant<0 .40=".40" br="br">
So, where does all this leave e-learning? Are we all just armchair psychologist looking for a quick fix to talk about how different ways of personalisation that ICT and multimedia offer can enhance learning? Maybe not, as the Coffield report leaves a back door open by saying that the potential of ICT to support individualised instruction "has not been fully evaluated".
Interestingly, this leads me where I want to go: look what ICTs can do. I will continue these notes with some reviews on papers on adaptive learning systems. For example, I'll look at this "Reappraising cognitive styles in adaptive web applications" that used Felder-Solomon Inventory of Learning Styles (ILS) instrument (which did not even make it to the 13 models studied by Coffield et al.) Oops, they say: "Contrary to previous findings by other researchers, we found no significant differences in performance between matched and mismatched students. Conclusions are drawn about the value and validity of using cognitive styles as a way of modelling user preferences in educational web applications." WoW!
It might be reasonable to note that in my research I'm not interested in adaptive learning or any of that, but I'm just doing this for the literature review to make my case of social information retrieval.
F Coffield, D Moseley, E Hall, K Ecclestone - Learning and Skills (2004). Learning styles and pedagogy in post-16 learning: A systematic and critical review. Research Centre, Wiltshire, UK.
Bloomer M and Hodkinson P (2000). Learning careers: continuing and change
in young people’s dispositions to learning. British Educational Research Journal, 26, 583–597.
Hattie J. 1999 speach where the table is extracted by Coffield et al.
E-learning area seem to be the other domain where learning styles pop up often. Many times it is claimed that e-learning allows personalised learning, e.g. learner is presented with material that marches his/her learning styles. Commonly we see references to VAKT (Visual, auditory, kinaesthetic and tactile) or some dimensions like holistic vs. analytic or linear, etc.
I took a quick (fix) review on learning styles after a short discussion that I had with a colleague of mine. I, totally mistakenly (of course, not) mentioned something along the lines of learning styles, where my colleague mentioned "aren't they already so passe". Uuhmm, yeah, sure...
So I duck up some literature on the Web and realised: which learning style? There sure are many of them, Goffield et al. (2004), for example, identified 71 in the literature, out of which his team chose 13 most influential and potentially influential models of learning styles for a systematic and critical review.
After reading "Learning styles and pedagogy in post-16 learning;
A systematic and critical review" one becomes humble about quick assumptions regarding learning styles. Table 44. presents these 13 Learning styles models matched against minimal criteria that was used in the review (p.139). Findings...
Only three of the 13 models – those of Allinson and Hayes, Apter and Vermunt – could be said to have come close to meetingthese criteria. A further three – those of Entwistle, Herrmann and Myers-Briggs met two of the four criteria. The Jackson model is in a different category, being so new that no independent evaluations have been carried out so far.
Oookey, seems like there is really something in this area of learning styles that hints that one should be rather wary and critical about quick fixes. Moreover, the plethora of models in the area should probably ring a bell. Along Coffield et al.
These central features of the research field – the isolated research groups, the lack of theoretical coherence and of a common conceptual framework, the proliferating models and dichotomies, the dangers of labelling, the influence of vested interests and the disproportionate claims of supporters – have created conflict, complexity and confusion. They have also produced wariness and a growing disquiet among those academics and researchers who are interested in learning, but who have no direct personal or institutional interest in learning styles. After more than 30 years of research, no consensus has been reached about the most effective instrument for measuring learning styles and no agreement about the most appropriate pedagogical interventions. p. 137
The main charge here is that the socio-economic and the cultural context of students’ lives and of the institutions where they seek to learn tend to be omitted from the learning styles literature. Learners are not all alike, nor are they all suspended in cyberspace via distance learning, nor do they live out their lives in psychological laboratories. Instead, they live in particular socio-economic settings where age, gender, race and class all interact to influence their attitudes to learning. Moreover, their social lives with their partners and friends, their family lives with their parents and siblings, and their economic lives with their employers and fellow workers influence their learning in significant ways. All these factors tend to be played down or simply ignored in most of the learning styles literature.
Coffield et al. are truly critical about this field of research, however, they don't, all together, through it to the waste-basket. They actually endorse some of the models which, instead of simplifying learning styles as anything like "deep-sealed features of the cognitive structure" or "components of a relatively stable personality type", see them more related to "learning preferences" or "learning approaches, strategies, orientations and conceptions of learning". They embrace these tools to help learners to gain more self-awareness and become more familiar with their metacognition, e.g. how to enhance their learning.
One of the main aims of encouraging a metacognitive approach is to enable learners to choose the most appropriate learning strategy from a wide range of options to fit the particular task in hand; but it remains an unanswered question as to how far learning styles need to be incorporated into metacognitive approaches. (p.132)
also
The positive recommendation we are making is that a discussion of learning styles may prove to be the catalyst for individual, organisational or even systemic change.
All right, now we are getting somewhere. Seems like it would be acceptable to say that people have learning preferences or "individual dispositions which influence the reactions of learners to their learning opportunities, which include the teaching style of their teachers." According to Bloomer and Hodkinson (2000) dispositions are both psychological and social. It is notable, however, that these individual dispositions constitute only a minor part of what can effect on learning.
To enlighten other effects or intervention on learning, Hattie (1992, 1999) synthesised 630 studies. If individualised learning means offering learning according to students' learning styles, the average effect size is not significant for individualised teaching in schools (significant<0 .40=".40" br="br">
So, where does all this leave e-learning? Are we all just armchair psychologist looking for a quick fix to talk about how different ways of personalisation that ICT and multimedia offer can enhance learning? Maybe not, as the Coffield report leaves a back door open by saying that the potential of ICT to support individualised instruction "has not been fully evaluated".
Interestingly, this leads me where I want to go: look what ICTs can do. I will continue these notes with some reviews on papers on adaptive learning systems. For example, I'll look at this "Reappraising cognitive styles in adaptive web applications" that used Felder-Solomon Inventory of Learning Styles (ILS) instrument (which did not even make it to the 13 models studied by Coffield et al.) Oops, they say: "Contrary to previous findings by other researchers, we found no significant differences in performance between matched and mismatched students. Conclusions are drawn about the value and validity of using cognitive styles as a way of modelling user preferences in educational web applications." WoW!
It might be reasonable to note that in my research I'm not interested in adaptive learning or any of that, but I'm just doing this for the literature review to make my case of social information retrieval.
F Coffield, D Moseley, E Hall, K Ecclestone - Learning and Skills (2004). Learning styles and pedagogy in post-16 learning: A systematic and critical review. Research Centre, Wiltshire, UK.
Bloomer M and Hodkinson P (2000). Learning careers: continuing and change
in young people’s dispositions to learning. British Educational Research Journal, 26, 583–597.
Hattie J. 1999 speach where the table is extracted by Coffield et al.
Friday, September 29, 2006
"Hello YouTubists...!"
Ok, I must say that I missed some of the YouTube's glamour, but I'm discovering some interesting things about it now. I though it was all about "lonelygirl15" and kids posting their skateboarding videos.
Today I came across "geriatric1927" (http://www.youtube.com/profile_videos?user=geriatric1927), number 1927 probably being his date of birth. This grandpa has found YouTube, and an amazing audience there, to tell about his life, and especially about his life during the years of World War II in England. I find it adoring, I thought that only the young generation uses YouTube, and there he is: this nearly 80 year old man, sitting in front of his computer, telling us all about his life. And that he learned about editing music and putting images in his video, and all!
It's worth watching a few of his babbles, some of which have been watched over 100 000 times by YouTubists. He seems to always start by saying "Hello YouTubists...!", he talks about war times, his youth, life after war and such. It's like anyone's grand-dad; taking time to tell you a story in his own time, space and pace. Very touchy! He also goes into talking about the media exposure that he's received, about communicating with people who post him messages and who send him mails, etc. It's gotta have taken his life into totally new dimension!
I love when new media is used in a new context and especially when its take-up reaches new groups of users that we never thought it might. Imagine the designers of YouTube, early in the day when they thought about setting it up, creating use cases for this! "Use case no 12: the 3rd generation using YouTube to record their experiences of life - hey, this might also help to integrate the oldies into the Web 2.0...". Well, I don't think!
This post is just to remind myself about wonders of the Web!
Today I came across "geriatric1927" (http://www.youtube.com/profile_videos?user=geriatric1927), number 1927 probably being his date of birth. This grandpa has found YouTube, and an amazing audience there, to tell about his life, and especially about his life during the years of World War II in England. I find it adoring, I thought that only the young generation uses YouTube, and there he is: this nearly 80 year old man, sitting in front of his computer, telling us all about his life. And that he learned about editing music and putting images in his video, and all!
It's worth watching a few of his babbles, some of which have been watched over 100 000 times by YouTubists. He seems to always start by saying "Hello YouTubists...!", he talks about war times, his youth, life after war and such. It's like anyone's grand-dad; taking time to tell you a story in his own time, space and pace. Very touchy! He also goes into talking about the media exposure that he's received, about communicating with people who post him messages and who send him mails, etc. It's gotta have taken his life into totally new dimension!
I love when new media is used in a new context and especially when its take-up reaches new groups of users that we never thought it might. Imagine the designers of YouTube, early in the day when they thought about setting it up, creating use cases for this! "Use case no 12: the 3rd generation using YouTube to record their experiences of life - hey, this might also help to integrate the oldies into the Web 2.0...". Well, I don't think!
This post is just to remind myself about wonders of the Web!
Wednesday, September 20, 2006
Is rating broken?
Yahoo!, You Tube and Netflix all use ratings on their services to better gear towards users' needs. We are talking about huge numbers here; Yahoo! gets some 5 million ratings a day for artists, albums, songs and videos; Netflix has currently 2 million ratings per day from its 5M customers; and within a day after the launch of lonelygirl15's My First Kiss- clip on YouTube she's got over 5000 ratings. Clearly, rating is something not to neglect, it is an easy way for users to input their opinion, as well as rather a straight way to compute affinities in relation to some other type of data such as search history, demographies, etc..
However, it seems like there is more to rating than meets the eye, and it becomes increasingly complicated for services to make the best use out of it. In the Recommenders06 conference issues with ratings were mentioned, but not discussed thoroughly. To me this seems a highly important issue, as current services are using rating as a primary input for their recommenders and many of the algorithms work based on ratings.
The following issues came up with ratings (unordered list):
Ok, I think it's broken, but the question is can it be fixed? Well, that looks like a long list of issues to deal with, but I'm sure nothing has gone beyond repairing.
However, there are many remaining questions: How to help people who don't rate? How to better understand users behaviour, what do they like and what not, and get that information in a more implicit way? The following remedies were mentioned in the conference:
Some more ways that I could think of:
However, it seems like there is more to rating than meets the eye, and it becomes increasingly complicated for services to make the best use out of it. In the Recommenders06 conference issues with ratings were mentioned, but not discussed thoroughly. To me this seems a highly important issue, as current services are using rating as a primary input for their recommenders and many of the algorithms work based on ratings.
The following issues came up with ratings (unordered list):
- Semantics of rating are pretty unclear; what does a user actually mean with 3.5?
- Meaning of ratings is very subjective; does my 2.5 mean the same as your 2.5?
- Ratings are straight out unclear; on the scale 1 to 5, does one (1) mean that I really don't want to ever see it again or does it mean that I just quite don't like it?
- What does a single attribute actually mean when rating for example one movie; is it about the plot, the actor, soundtrack? What if I like the plot but hate the main actor, how to express that?
- Love/Hate-ratings: many services are getting more and more ratings only on the far dimentions of the scale; rating value distribution is large.
- Binaries like thumbs up and down have issues too; How do I interpret something that has 10 thumbs-ups and 10 downs? Am I going to take the risk to either really like it or really hate it?
- Rating variance; how does 10 up and 10 down rating effect on people's choice? Do people go for the middle way? Apparently not, see Jolie's presentation.
- Only a few have rated many, many have rated a few – distribution of ratings is very sparse. It is hard to recommend something for those many with few ratings.
- Rating distribution between genres: some genres are more predictable than others thanks to user ratings. How to recommend the ones not so ofter rated? In Netflix comedies and drama are more predictable than musicals, for example. The presentage of 4-5 start movies rented has increased, as prediction accuracy becomes better.
- Do users understand what ratings are for? Whether users really understand what ratings can do for them when using Yahoo! Music stuff or Netflix? It's about the trade-off between control gained over the service but yielding to users' convenience that they give up when taking time to rate.
- Feedback loop between rating and recommendations can become self-promoting. If I rate something good, the recommender keeps recommending that or similar items to me (also known as similarity-trap). There is the popularity bias: at the end, everything is related to Britney Spears.
- Knowing the users' intentions: wanna buy or listen?
- Ratings depend on when the item was rated (Netflix found out that ratings done immediately after watching the movie vary from the ones made at the later stage!).
- Ratings are vulnerable to chilling, intentionally bad ratings, want to lift some music up on the list by rating it high, etc. (influencing the vote is relatively easy using some algorithms whereas hybrids might be more robust against manipulation, see Mr Mobasher’s persentation).
- Does the user feel home with the other raters? Am I sure that I belong to this group of users and tastes? For example Last.fm started as a rather geek service, thus lots of users have rated items that match geeky music taste! This becomes really important when we think about internationalisation of recommender services, can my taste match with white-male-middle-class American taste?
- Computing affinities with userprofile, editorial rankings, etc can take a long time, for example some Yahoo! services are only updated weekly since it's so computing intensive.
Ok, I think it's broken, but the question is can it be fixed? Well, that looks like a long list of issues to deal with, but I'm sure nothing has gone beyond repairing.
However, there are many remaining questions: How to help people who don't rate? How to better understand users behaviour, what do they like and what not, and get that information in a more implicit way? The following remedies were mentioned in the conference:
- Going beyond rating for data input for recommenders by monitoring the play events in an online radio.
- Uploaded playlists by users can yield important information about sequencing music, moods that they are played in, etc.
- Netflix talked about encoding traits of movies that predict emotional responses, for example. Maria, one of the students, talked about combining personality traits and mood settings to further personalise and contextualise recommendations.
- Prof.Riedl talked about letting users know the value of their rating to the community, e.g. how important rating one given item is to make better recommendations for this given group. It seems like people care about others, they are willing to make ratings to help other people similar to them finding better items.
- Using social networks to better make and find recommendations.
- Imporving ROI for users; with fewer inputs get more valuable outputs like playlists, concerts, videos, music news, etc.
Some more ways that I could think of:
- If binary types of ratings are something that people do, let's just use thumbs up and thumbs down.
- If scales are used, be explicit about them. No one really knows what the stars mean in iPod! Say clearly: O means “never play again”.
- Multi-attribute ratings: if you allow people to rate, give them also options to be more clear about it; I think the plot is good, but acting sucks. There are people who love to do ratings and evaluations (just look at Amazon.com with their reviewers lists!) and many times they are good in doing it.
- Leverage on re-using ratings from other services: Netflix, Yahoo!, MovieLense and God knows how many other services rate the movies. Think about webservices or harvesting those ratings and get rid of the sparsity problem! There should be some interoperability between user ratings and other evaluations between services.
- I want a meta-recommender! Would be good to know if my music taste matches with other people's taste in a given service or whether I should hang out somewhere else to get favourable recommendations.
- Anyway, those services are too focused only getting people to use that one and only service, by pooling up and letting users to take advantage of their profile in place a in place b would be convenient for me! Maybe Attention metadata could become to help here. Attention XML and Attention Metadata: Collecting, Managing and Exploiting of Rich Usage Information at International ACM Workshop
Monday, September 18, 2006
Anousheh Ansari: currently in orbit
What an inspiring story, what an inspirational person! and yeah, a woman :)
Not that I want to overemphises that latter fact, but I must say that it really makes me feel shivers and smile proudly - we need this kind of examples to inspire us. A quote from BBC world:
Touché!
Her spaceblog is at: http://spaceblog.xprize.org/
Not that I want to overemphises that latter fact, but I must say that it really makes me feel shivers and smile proudly - we need this kind of examples to inspire us. A quote from BBC world:
The Star Trek fan, who spent her early childhood in pre-revolutionary Iran, has spoken of the nights on the balcony gazing at the stars and a longing to become an astronaut.Imagine that! And now she's in space, blogging away, after making it to Fortune magazine's "40 under 40", ha! Kinda person that makes me want to achieve something too. Being an example to someone else and make a positive impact on them.
Touché!
Her spaceblog is at: http://spaceblog.xprize.org/
Wednesday, September 13, 2006
A meta recommender of movies and music
Would be fun to play around with a meta-recommender that compares recommendations from different services, for example, for a movie. So, I would be in one service, say Netflix and had hard time choosing a film between the big choice of movies. Instead of knowing what other Netflix users thought about the movie, I could also see how users in Yahoo!, MovieLens, etc rated it.
As we found out today, ratings, that are widely used in recommenders, are quite tricky things. There is a lot of sparsity problems, variations in ratings, interpretations of a rating scale, not enough criteria or too much of them, etc. So, it might be useful (or then totally not), to see how MovieLens and Amazon users rated the movie, to compare whether there is a deep variation between them, and eventually find a community whose tase is similar to yours.
Of course the datamodels are not the same and there are different rating scales, but some normalisation could be done or thumbs up/down, or so. Besides, comparing the evaluation data across different applications would probably yeld very interesting results!
As we found out today, ratings, that are widely used in recommenders, are quite tricky things. There is a lot of sparsity problems, variations in ratings, interpretations of a rating scale, not enough criteria or too much of them, etc. So, it might be useful (or then totally not), to see how MovieLens and Amazon users rated the movie, to compare whether there is a deep variation between them, and eventually find a community whose tase is similar to yours.
Of course the datamodels are not the same and there are different rating scales, but some normalisation could be done or thumbs up/down, or so. Besides, comparing the evaluation data across different applications would probably yeld very interesting results!
Recommenders06 - round table and attention
As the round table discussion drifted from one thing to another, some things penetrated my attention - specially when the discussion was about business models and future trends. It was mentioned that convergences between things are found, lilke the pact with Apple and Nike.
It made me think further about the labour intensiveness of the social applications on the web; you have to add your user profile, friends, music taste, etc to make it fun to work with (some automatisation exist, but not quite..). The same thing with recommenders, first I spend time to rate movies in MovieLens to get some recommendations. Then I go to Amazon.com to buy some movies or books, and they don't know shit about my taste as I haven't been shopping exclusively there.
Someone mentioned "Universal profile", which apparently has been thought of at some point in the past. To me that sounds like a really bad idea. That should be a mother big profile, a real monster to keep all info about me and my intentions and attentions.
The idea of using Attention metadata (AttentionTrust.org) kind of solution would be better. I own my attention, and I choose how much of it I want to share with a commercial entity to get better recommendations.
Some current work going on Attention XML and Attention Metadata here: Collecting, Managing and Exploiting of Rich Usage Information at International ACM Workshop http://ariadne.cs.kuleuven.be/cama2006/
It made me think further about the labour intensiveness of the social applications on the web; you have to add your user profile, friends, music taste, etc to make it fun to work with (some automatisation exist, but not quite..). The same thing with recommenders, first I spend time to rate movies in MovieLens to get some recommendations. Then I go to Amazon.com to buy some movies or books, and they don't know shit about my taste as I haven't been shopping exclusively there.
Someone mentioned "Universal profile", which apparently has been thought of at some point in the past. To me that sounds like a really bad idea. That should be a mother big profile, a real monster to keep all info about me and my intentions and attentions.
The idea of using Attention metadata (AttentionTrust.org) kind of solution would be better. I own my attention, and I choose how much of it I want to share with a commercial entity to get better recommendations.
Some current work going on Attention XML and Attention Metadata here: Collecting, Managing and Exploiting of Rich Usage Information at International ACM Workshop http://ariadne.cs.kuleuven.be/cama2006/
Tuesday, September 12, 2006
Recommenders 06
Yesterday after arrival to Bilbao, Spain, some people from the conference got together to go around the town. There was Claudio, Nikos, Jolie and Iilja. So I already had a chance to talk with Claudio Baccigalupo (IIIA-CSIC, Spain) about his research already.
(btw; Blog postings with presentations are found at http://blog.recommenders06.com/)
It's about automatically generating playlists based on Case-based Reasoning (CBR). You can find a demo at http://labs.mystrands.com. As a case base for his recommender he uses uploaded playlists by MyStrands users who have decided to upload them. Then the system analyses them for regarding the order, and when a user says that she wants a playlist with F.Sinatra, the recommender creates one based on previous playlists. Of course, for the system to work well it is desirable to assume that the case base is perfect, i.e. hopefully the playlists are created by semi-professional deejays, and not just with people without any understanding of how songs actually best fit together to have a nice flow.
Fun, will play around with it!
(btw; Blog postings with presentations are found at http://blog.recommenders06.com/)
It's about automatically generating playlists based on Case-based Reasoning (CBR). You can find a demo at http://labs.mystrands.com. As a case base for his recommender he uses uploaded playlists by MyStrands users who have decided to upload them. Then the system analyses them for regarding the order, and when a user says that she wants a playlist with F.Sinatra, the recommender creates one based on previous playlists. Of course, for the system to work well it is desirable to assume that the case base is perfect, i.e. hopefully the playlists are created by semi-professional deejays, and not just with people without any understanding of how songs actually best fit together to have a nice flow.
Fun, will play around with it!
Friday, September 08, 2006
Notes and ideas on “tagging, communities, vocabulary, evoution”
Shilad Sen, Shyong K. Lam, Dan Cosley, Al Mamunur Rashid, Dan Frankowski, Franklin Harper, Jeremy Osterhouse, John Riedl. tagging, community, vocabulary, evolution. To appear in Proceedings of CSCW 2006.
This paper focuses on users and tags from the user point of view, not from the object-tag point of view.
The paper refers to the nine tag classes (feels kinda contrary to talk about classes for tags...) presented by Golder et al. (2006), and present three classes system with
The authors conducted their research on MovieLens system where they had introduced a tagging system that was tested over a period of a month. The distribution over the tags was
Some worth mentioning finding and something to look for or compare with in learning resources experiments
About half of the tags were tags that the user had previously applied, thus they conclude that clearly habit and investment influence tagging behaviour and grows stronger as users apply more tags (I'm not sure whether they mean that as more tags (bigger variety of tags) are used by the tagger or as more items are tagged in general (with using a small variety of tags..). However they state that habit and investment aren't the only factors that contribute to vocabulary evolution.
On the act on tagging
Users who view tags by other people before tagging their first tag are more likely to have their tags influenced by other taggers community. So community affects user's personal vocabulary and it is stronger on user's first tag if they have been exposed for others' tags. This is to consider when designing a tool!
On the convergence of vocabularies
Seems like (quite self evidently) that if people see other's (e.g. they are proposed, are autofilled when typing, ...) tags while they are tagging vocabularies are more likely to converge than if users are working on their own. This is important to think of when we are designing our tool: are we going to show only user's own tags, all other users' tags, only most used tags or make a ready-made set of desirable tags on a given resource. Plus, how will the thesaurus term affect on the tagging culture. This could actually provide an interesting research possibility: to have different tagging interfaces for users and see how tags would differ!
On tags and how they are useful for different user's tasks
Different classes of tags approve useful for different tasks that the user has. The “taskonomy” is: self-expression (helps to express opinion), organising, learning about the given movie, finding, decision support.
It seems like personal tags are found useful only for self-organising (which hints to the direction that I think of their usefulness for PKM), whereas factual tags are good for learning about the movie and to help finding it. Subjective tags are found overwhelmingly good for self-expression, but also to support decision making process (!) (although only 1/3 of people who did not tag thought so).
All in all, all tags were mostly found useful in self-expressing and organising. Note, 23% of people who did not tag found them also helpful for organising! What is interesting and worth noting for our development is that people did not like seeing other people's personal tags, to which the authors mention that maybe a design decision needs to be taken on whether to have some way to choose to keep tags only private to the tagger, i.e. how to strike balance between other benefits of tagging and the privacy that people might need.
Also, another design question is related to people who did not tag, there were overwhelming more non-taggers in the group that had not seen examples of tags than in the one that had seen them in their tagging interface. To remember, though, pre-existing tags affect future tagging behaviour.
Moreover, the authors suggest that it could be useful to try to foresee some way to classify tags in those above mentioned classes, both automated ways to infer tags and interface designs should be considered.
Good references to check:
[4] C. Cattuto, V. Loreto, and L. Pietronero. Semiotic dynamics in online social communities. In The European Physical Journal C (accepted). Springer-Verlag, 2006.
[5] R. B. Cialdini. Influence Science and Practice. Allyn and Bacon, MA, USA, 2001.
[6] D. Cosley, S. K. Lam, I. Albert, J. Konstan, and J. Riedl. Is seeing believing? How recommender system interfaces affect users’ opinions. In CHI, 2003.
[9] S. Golder and B. A. Huberman. The structure of collaborative tagging systems. Journal of Information Science (accepted), 2006.
[10] M. Guy and E. Tonkin. Tidying up Tags? D-Lib Magazine, 12(1):1082–9873, 2006.
[11] T. Hammond, T. Hannay, B. Lund, and J. Scott. Social bookmarking tools : A general review. D-Lib Magazine, 11(4), April 2005.
This paper focuses on users and tags from the user point of view, not from the object-tag point of view.
The paper refers to the nine tag classes (feels kinda contrary to talk about classes for tags...) presented by Golder et al. (2006), and present three classes system with
- factual tags (Golder: item topics, kinds of item, category refinements)
- subjective tags (Golder: item qualities)
- personal tags (Golder: item ownership, self-reference, tasks organisation)
The authors conducted their research on MovieLens system where they had introduced a tagging system that was tested over a period of a month. The distribution over the tags was
- 63% factual
- 29% subjective
- 3% personal
- 5% other
- names were related to the broad subject area such as biology or mathematics (factual);
- the name additionally had indication about the intended audience (factual);
- the name only had indication of the intended audience (factual);
- the name indicated a sub-area such as “trigonometria,” a precise theme such as “brain” or a pluridiciplinary subject such as “watter” (factual);
- also names that identified other things were used such as “easy” (subjective),
- many indicated names of people and other acronymes whose meaning was not identifiable for an “outsider” (personal).
- names were mostly marked in the language of the user.
Some worth mentioning finding and something to look for or compare with in learning resources experiments
About half of the tags were tags that the user had previously applied, thus they conclude that clearly habit and investment influence tagging behaviour and grows stronger as users apply more tags (I'm not sure whether they mean that as more tags (bigger variety of tags) are used by the tagger or as more items are tagged in general (with using a small variety of tags..). However they state that habit and investment aren't the only factors that contribute to vocabulary evolution.
On the act on tagging
Users who view tags by other people before tagging their first tag are more likely to have their tags influenced by other taggers community. So community affects user's personal vocabulary and it is stronger on user's first tag if they have been exposed for others' tags. This is to consider when designing a tool!
On the convergence of vocabularies
Seems like (quite self evidently) that if people see other's (e.g. they are proposed, are autofilled when typing, ...) tags while they are tagging vocabularies are more likely to converge than if users are working on their own. This is important to think of when we are designing our tool: are we going to show only user's own tags, all other users' tags, only most used tags or make a ready-made set of desirable tags on a given resource. Plus, how will the thesaurus term affect on the tagging culture. This could actually provide an interesting research possibility: to have different tagging interfaces for users and see how tags would differ!
On tags and how they are useful for different user's tasks
Different classes of tags approve useful for different tasks that the user has. The “taskonomy” is: self-expression (helps to express opinion), organising, learning about the given movie, finding, decision support.
It seems like personal tags are found useful only for self-organising (which hints to the direction that I think of their usefulness for PKM), whereas factual tags are good for learning about the movie and to help finding it. Subjective tags are found overwhelmingly good for self-expression, but also to support decision making process (!) (although only 1/3 of people who did not tag thought so).
All in all, all tags were mostly found useful in self-expressing and organising. Note, 23% of people who did not tag found them also helpful for organising! What is interesting and worth noting for our development is that people did not like seeing other people's personal tags, to which the authors mention that maybe a design decision needs to be taken on whether to have some way to choose to keep tags only private to the tagger, i.e. how to strike balance between other benefits of tagging and the privacy that people might need.
Also, another design question is related to people who did not tag, there were overwhelming more non-taggers in the group that had not seen examples of tags than in the one that had seen them in their tagging interface. To remember, though, pre-existing tags affect future tagging behaviour.
Moreover, the authors suggest that it could be useful to try to foresee some way to classify tags in those above mentioned classes, both automated ways to infer tags and interface designs should be considered.
Good references to check:
[4] C. Cattuto, V. Loreto, and L. Pietronero. Semiotic dynamics in online social communities. In The European Physical Journal C (accepted). Springer-Verlag, 2006.
[5] R. B. Cialdini. Influence Science and Practice. Allyn and Bacon, MA, USA, 2001.
[6] D. Cosley, S. K. Lam, I. Albert, J. Konstan, and J. Riedl. Is seeing believing? How recommender system interfaces affect users’ opinions. In CHI, 2003.
[9] S. Golder and B. A. Huberman. The structure of collaborative tagging systems. Journal of Information Science (accepted), 2006.
[10] M. Guy and E. Tonkin. Tidying up Tags? D-Lib Magazine, 12(1):1082–9873, 2006.
[11] T. Hammond, T. Hannay, B. Lund, and J. Scott. Social bookmarking tools : A general review. D-Lib Magazine, 11(4), April 2005.
Tuesday, September 05, 2006
Questions and notes on Making Recommendations Better: An Analytic Model for Human-Recommender Interaction
S.M. McNee, J. Riedl, and J.A. Konstan. "Making Recommendations Better: An Analytic Model for Human-Recommender Interaction". In the Extended Abstracts of the 2006 ACM Conference on Human Factors in Computing Systems (CHI 2006) [to appear], Montreal, Canada, April 2006.
The paper start from an healthy self-assertion that recommenders do not always generate good recommendations for users. Authors propose a Human-Recommender Interaction (HRI) as a framework and a methodology to understand users, their tasks and how do they relate to recommender algorithms. They propose that HRI can be a bridge between user information seeking tasks and recommender algorithms, when applied in the HRI Analytic Process Model, it can become a constructive model to help the process to design a recommender.
As the information density grows, users have more specific needs for their information seeking. HRI can be used to describe these needs, thus firstly, thinking about myself and my research area, I have to describe user types (probably I could use the LRE logs to deduce this) and typical domain tasks (this would have to be some guestimates that I test with a focus group). The authors suggest Hackos 1998 for this, but looks pretty old. A detailed analysis of these tasks will allow us to link task to specific HRI Aspects.
HRI Aspects, the three pillars:
The HRI Analytic Process Model can help to analyse and redesign recommenders to better meet user information needs. (Can it also help to design them in the first place?) For example it could help to understand whether a user would be contented with risky recommendations or more like the ones that affirm her information seeking needs.
Moreover, the authors say that by looking at which HRI aspects are important to which task, some metrics can be designed (I would be very interested in those metrics!) to categorise the differences between tasks. These metrics could be used to benchmark the known algorithms, and thus help to choose the proper one for the task. Rather, as the authors state, a recommender should have a set of algorithms to use instead of being “one for all users”-type of set.
Questions for Mr. Riedl
More on HRI, a PhD thesis by McNee: http://www-users.cs.umn.edu/~mcnee/mcnee-thesis-preprint.pdf
Research statement by the above: http://www-users.cs.umn.edu/~mcnee/mcnee-research-statement.pdf
The paper start from an healthy self-assertion that recommenders do not always generate good recommendations for users. Authors propose a Human-Recommender Interaction (HRI) as a framework and a methodology to understand users, their tasks and how do they relate to recommender algorithms. They propose that HRI can be a bridge between user information seeking tasks and recommender algorithms, when applied in the HRI Analytic Process Model, it can become a constructive model to help the process to design a recommender.
As the information density grows, users have more specific needs for their information seeking. HRI can be used to describe these needs, thus firstly, thinking about myself and my research area, I have to describe user types (probably I could use the LRE logs to deduce this) and typical domain tasks (this would have to be some guestimates that I test with a focus group). The authors suggest Hackos 1998 for this, but looks pretty old. A detailed analysis of these tasks will allow us to link task to specific HRI Aspects.
HRI Aspects, the three pillars:
- the Recommendation Dialog, the act of giving information and recieving one recommendation list from a recommender. This contains aspects like Correctness, Transparency, Saliency, Serendipity Quantity, Usefulness, Spread and Usability. The authors argues that recommender's purpose is to generate salient recommendations that strike an emotional response (the awe factor!)
- the Recommender Personality (uh, I don't like that term), the user's perception of the recommender over a period of time. Aspects are such like personalisation, boldness, adaptability, trust/first impression, risk taking/aversion, affirmation, pigeonholing and freshness.
- User Information Seeking Tasks, the reason the user came to the recommender system. Aspects such as Concreteness of task, task compromising, recommender appropriateness, expectations of recommender usefulness, recommender importance in meeting needs. Check out Case 2002.
The HRI Analytic Process Model can help to analyse and redesign recommenders to better meet user information needs. (Can it also help to design them in the first place?) For example it could help to understand whether a user would be contented with risky recommendations or more like the ones that affirm her information seeking needs.
Moreover, the authors say that by looking at which HRI aspects are important to which task, some metrics can be designed (I would be very interested in those metrics!) to categorise the differences between tasks. These metrics could be used to benchmark the known algorithms, and thus help to choose the proper one for the task. Rather, as the authors state, a recommender should have a set of algorithms to use instead of being “one for all users”-type of set.
Questions for Mr. Riedl
- What are the metrics, any exaples?
- What are the outcomes of the simulations against the well-known algorithms, did the mapping between the tasks and algorithms materialise, and if not, how well? More information available? In the paper it mentiones that they are submitted, under review. Whom to contact?
More on HRI, a PhD thesis by McNee: http://www-users.cs.umn.edu/~mcnee/mcnee-thesis-preprint.pdf
Research statement by the above: http://www-users.cs.umn.edu/~mcnee/mcnee-research-statement.pdf
Sunday, September 03, 2006
On Social Network Analysis and Recommenders
An interesting area where I drifted today is the crossing point of recommender systems and social network analysis (SNA). I read a few papers in a row about it (Rashid et al., 2005; Korfiatis et al. 2006, not published; Carcia-Barriocanal&Sicilia, 2005),
The other day I chatted up with my PhD study-buddy on SNA and it was actually quite enlightening. I was seeking to understand what is the difference between what most (old-school) recommenders do, and what does SNA have to offer to this. SNA are used to better figure out what groups do and how do they form, etc, but I lacked the understanding of how to use this for something that I want to do, i.e. enhance the discovery and re-use of LOs in a repository.
I am an avid believer and lover of social bookmarks. That's it, it's out. I think we could do so many things better just doing that. I, of course, just have to prove that in my PhD, and find a way to prove it, so it helped to talk with my buddy who is researching stuff somewhere between behavioural economics and social network theory. He had the words that I was lacking for social navigation – you get in a social space and you don't have any clues of what is out there. What do people do. They follow others, they need a guide. Say, you see other people going one way and you follow. That is what I see social bookmarks offer you, a guide to go ahead, a direction, a pointer to start. But there is also another aspect to it, bookmarks offer connections, relations between me and things I like, and then again, between things I like and other people who like the same things.
Which brings me to - how can we leverage this for information retrieval (IR). Sicilia and Garcia, 2005 and Korfiatis et al. 2006 (not published) talked about this: to bridge the areas of Social Network Analysis (SNA) and Information Retrieval. In a way, already the famous PageRank is about social networks, who endorses whom in a form of a hyperlink. The only problem being is that we do also link to things that we don't care about...but back to recommenders...
Until somewhat recently recommenders were about ratings and explicit values that people gave to items. The big deal was inferring those values for users who had not explicitly done that or even interacted with the item. Nowadays we are moving into using all other kinds of data as an input for recommenders, like the context-aware attention metadata that my colleagues are looking into.
The idea of Contextual Attention Metadata-framework is that it would log data from different application that a user is using for the e-learning purposes. The fact is, that nowadays we are getting further and further away (at least mentally) from single big Learning Management Systems (LMS) and are more and more looking into using small “comfi” tools (IM, bookmarks, wikis, blogs,..) for learning purposes too. All these tools can generate attention metadata, and a framework like CAM could track that. A step ahead from conventional data-mining from separate and sparse log-files.
So, now are are looking into contextual attention metadata that can arch across application boundaries and tell us stuff like: after watching that educational movie, the learner 3 contacted a tutor by IM and then spent an hour working on a text editor while surfing on the Web using x and y keywords. From that we can try to deduce things (like how the learner actually uses the learning tools and material) that we could use to make more personalised recommendations.
What I find more interesting, though, is the social context, like PeopleRank (Carcia-Barriocanal&Sicilia, 2005; Korfiatis, 2006 n-y-p), that could be used to compliment something like PageRank. PeopleRank would use the social ties, i.e. the links that people have expressed in a FOAF-file to compliment the “conventional” the PageRank algorithm. That's cool, all right, although, just right from the bat I feel like I prefer the Yahoo's MyRank, that also uses a FOAF-description on top of their conventional search algorithm. Moreover, I would be interested in finding some other ways to use the FOAF-file, which I'm trying to think of. Maybe some more interesting things could, in deed like suggested by Carcia-B..&co, come from the use of foaf to express relations between organisation or group (schools, educational projects,.- like we could use it in our EUN-context), instead of individuals.
Well, back to my bookmarks and tags: I'm interested in observing on what happens in a repository of LOs where users can bookmark learning resources, socially navigate them in other people's collections, when tags are used and when people can rate and evaluate LOs that they have in their collections. Furthermore, we like to facilitate the creation of lesson plans, like one would create play lists in iTunes.
Recommending educational material to teachers and learners, automatically sequencing course material or aggregating learning resources and delivering personalised learning has in many research oriented projects relied on pedagogical concepts, on learners learning styles, on assessment of previous knowledge and skills, etc. This is probably very useful and has undoubtedly many potentials. (First we only need kind of standardised testing to assess skills and then plentiful pool of varied learning resources that comply to any different learning style, oh yeah, and which definition of learning styles are we going to use...).
Instead, I'm interested in tapping into the social power of a group of educators and their knowledge about what learning resources to use and in what case. Instead of looking into personalisation-side of things, I want to see what happens if we just look into socialisation-side of things. Do like others have done-kinda idea. If other people cross the street here, maybe I should cross it here too.
Of course we would have to assume that there are some personalisation going on, each case is unique, after all. But still many cases do resemble one another. And maybe looking into social navigation in an educational context can help us to unlock the problematic and labour intensive questions of recommending educational material.
Additionally, bookmarking comes with tagging, user-generated keywords that people can assign for material to find it later. That's the personal knowledge management side of things. Tags can also create communities, people interested in same things eventually end up using similar names/tags and thus a link is formed. Tags also make us understand better the different meanings and ways that people can understand “a thing”, etc..In the LOR-context tags could make explicit some of the teachers' "folk pedagogy" type of knowledge. Folk pedagogy can be an accumulated set of beliefs, conceptions and assumptions that professors personally hold about the practice of teaching (Bruner, 1996). Maybe this can also unlock something that we don't know of, yet.
Well, some thoughts that I've decided to write down to keep track of my thought.
The other day I chatted up with my PhD study-buddy on SNA and it was actually quite enlightening. I was seeking to understand what is the difference between what most (old-school) recommenders do, and what does SNA have to offer to this. SNA are used to better figure out what groups do and how do they form, etc, but I lacked the understanding of how to use this for something that I want to do, i.e. enhance the discovery and re-use of LOs in a repository.
I am an avid believer and lover of social bookmarks. That's it, it's out. I think we could do so many things better just doing that. I, of course, just have to prove that in my PhD, and find a way to prove it, so it helped to talk with my buddy who is researching stuff somewhere between behavioural economics and social network theory. He had the words that I was lacking for social navigation – you get in a social space and you don't have any clues of what is out there. What do people do. They follow others, they need a guide. Say, you see other people going one way and you follow. That is what I see social bookmarks offer you, a guide to go ahead, a direction, a pointer to start. But there is also another aspect to it, bookmarks offer connections, relations between me and things I like, and then again, between things I like and other people who like the same things.
Which brings me to - how can we leverage this for information retrieval (IR). Sicilia and Garcia, 2005 and Korfiatis et al. 2006 (not published) talked about this: to bridge the areas of Social Network Analysis (SNA) and Information Retrieval. In a way, already the famous PageRank is about social networks, who endorses whom in a form of a hyperlink. The only problem being is that we do also link to things that we don't care about...but back to recommenders...
Until somewhat recently recommenders were about ratings and explicit values that people gave to items. The big deal was inferring those values for users who had not explicitly done that or even interacted with the item. Nowadays we are moving into using all other kinds of data as an input for recommenders, like the context-aware attention metadata that my colleagues are looking into.
The idea of Contextual Attention Metadata-framework is that it would log data from different application that a user is using for the e-learning purposes. The fact is, that nowadays we are getting further and further away (at least mentally) from single big Learning Management Systems (LMS) and are more and more looking into using small “comfi” tools (IM, bookmarks, wikis, blogs,..) for learning purposes too. All these tools can generate attention metadata, and a framework like CAM could track that. A step ahead from conventional data-mining from separate and sparse log-files.
So, now are are looking into contextual attention metadata that can arch across application boundaries and tell us stuff like: after watching that educational movie, the learner 3 contacted a tutor by IM and then spent an hour working on a text editor while surfing on the Web using x and y keywords. From that we can try to deduce things (like how the learner actually uses the learning tools and material) that we could use to make more personalised recommendations.
What I find more interesting, though, is the social context, like PeopleRank (Carcia-Barriocanal&Sicilia, 2005; Korfiatis, 2006 n-y-p), that could be used to compliment something like PageRank. PeopleRank would use the social ties, i.e. the links that people have expressed in a FOAF-file to compliment the “conventional” the PageRank algorithm. That's cool, all right, although, just right from the bat I feel like I prefer the Yahoo's MyRank, that also uses a FOAF-description on top of their conventional search algorithm. Moreover, I would be interested in finding some other ways to use the FOAF-file, which I'm trying to think of. Maybe some more interesting things could, in deed like suggested by Carcia-B..&co, come from the use of foaf to express relations between organisation or group (schools, educational projects,.- like we could use it in our EUN-context), instead of individuals.
Well, back to my bookmarks and tags: I'm interested in observing on what happens in a repository of LOs where users can bookmark learning resources, socially navigate them in other people's collections, when tags are used and when people can rate and evaluate LOs that they have in their collections. Furthermore, we like to facilitate the creation of lesson plans, like one would create play lists in iTunes.
Recommending educational material to teachers and learners, automatically sequencing course material or aggregating learning resources and delivering personalised learning has in many research oriented projects relied on pedagogical concepts, on learners learning styles, on assessment of previous knowledge and skills, etc. This is probably very useful and has undoubtedly many potentials. (First we only need kind of standardised testing to assess skills and then plentiful pool of varied learning resources that comply to any different learning style, oh yeah, and which definition of learning styles are we going to use...).
Instead, I'm interested in tapping into the social power of a group of educators and their knowledge about what learning resources to use and in what case. Instead of looking into personalisation-side of things, I want to see what happens if we just look into socialisation-side of things. Do like others have done-kinda idea. If other people cross the street here, maybe I should cross it here too.
Of course we would have to assume that there are some personalisation going on, each case is unique, after all. But still many cases do resemble one another. And maybe looking into social navigation in an educational context can help us to unlock the problematic and labour intensive questions of recommending educational material.
Additionally, bookmarking comes with tagging, user-generated keywords that people can assign for material to find it later. That's the personal knowledge management side of things. Tags can also create communities, people interested in same things eventually end up using similar names/tags and thus a link is formed. Tags also make us understand better the different meanings and ways that people can understand “a thing”, etc..In the LOR-context tags could make explicit some of the teachers' "folk pedagogy" type of knowledge. Folk pedagogy can be an accumulated set of beliefs, conceptions and assumptions that professors personally hold about the practice of teaching (Bruner, 1996). Maybe this can also unlock something that we don't know of, yet.
Well, some thoughts that I've decided to write down to keep track of my thought.
Tuesday, August 29, 2006
Notes and comments on: Accurate is not always good: How Accuracy Metrics have hurt Recommender systems
S.M. McNee, J. Riedl, and J.A. Konstan. "Being Accurate is Not Enough: How Accuracy Metrics have hurt Recommender Systems". In the Extended Abstracts of the 2006 ACM Conference on Human Factors in Computing Systems (CHI 2006) [to appear], Montreal, Canada, April 2006
The paper starts by informally arguing that "the recommender community should move beyond the conventional accuracy metrics and their associated experiment methodologies. We propose new user-centric directions for evaluating recommender systems".
The paper states that the current accuracy metrics, such as MAE (Herlocker 1999), measure recommender algorithm performance by comparing the algorithm's prediction against a user's rating of an item. They continue saying that this means, in essence, that a recommender that recommends places to a user where she has already visited would be rewarded rather than a recommendation on new places that might be of interest. Clearly, if that is the case, there is something rotten.
The paper proposes three aspects; similarity, recommendation serendipity and the importance of user needs and expectations in a recommender, and suggests how they could be improved.
A) Similarity
- the item-item collaborative filtering algorithm can trap users in a "similarity hole" only giving similar recommendations. This becomes more problematic when there is less data, for example, for a new user in a system.
The authors go on to discuss about the accuracy metrics that don't recognise this problem, because they are designed to judge the accuracy of individual items and not the list of items. However, the authors argue, "the recommendation list should be judged for its usefulness as a complete entity, not just as a collection of individual items." There was evidence in a user testing that the lists that had performed badly on conventional accuracy measures were the ones preferred by users. These lists had used the Intra-List Similarity Metrics and the process of Topic Diversification for recommendation lists (Ziegler 2005).
Authors go on saying that depending on the user's intentions, the makeup of items appearing on the list affected the user's satisfaction with the recommender. Here, in my opinion, it becomes important to remember the user intentions as provided by Swearingen & Sinha (2001)
- Reminder recommendations, mostly from within genre (“I was planning to read this anyway, it’s my typical kind of item”)
- More like this” recommendations, from within genre, similar to a particular item (“I am in the mood for a movie similar to GoodFellas”)
- New items, within a particular genre, just released, that they / their friends do not know about
- “Broaden my horizon” recommendations (might be from other genres)
B) Serendipity
This is how unexpected the recommendation is for the user and how novel it is. This is hard to measure. The authors approach the issue by its opposite: the ratability of received recommendations, and this, they say, is easy to measure by using the "leave-n-out" approach. However, the assumption that users are interested in the highest ratable items is not always true for recommenders. They give an example of recommending Beatle's White Album to users of a music recommender as a bad idea, as it almost adds no value.
The same example, I remember, was somewhere else on recommending to people buy bananas when they go shopping, however, apparently people almost always buy bananas anyway, thus no commercial value there..This could, though, have some value, when building people's trust on a recommender.
Moreover, the authors point out that different algorithms give different recommendations, and that people preferred one over another depending on their current task (think again about Swearingen/Sinha). To conclude on serendipity, the authors say that other metrics could be needed to judge a variety of algorithm aspects - no direction given on this one, though.
c) User experiences and expectations
New users have different needs from experienced users. Rashid (2001) has shown that the choice of algorithm for a new user greatly affects the experience (really?!) and also, apparently, the native language is greatly preferred (Torres 2004), wonder what kind of language groups were in question there..
- Moving forward
Authors don't suggest that the old-school metrics should be thrown away, but not only be used alone, we need to think of the users who want meaningful recommendations (!!).
Firstly, it is recommended that instead of looking at each item on the list of recommendations, one should pay more attention on the integrity of the list, using metrics like Intra-List Similarity metrics, and more of such kinds.
We should test more what kind of search algorithms users like and given them those.
Users have a purpose for expecting a recommendation, so we would need to know better what actually are the user needs when they come to see a recommendation (Zaslow 2002).
Well, well, if this is where we are at with recommender usability studies, it is not much. However, it is great that important people such as Grouplens researchers tell us this, so maybe it makes the general audience more susceptible for new things to come.
References:
Swearingen & Sinha (2001)
Torres, R., McNee, S.M., Abel, M., Konstan, J.A., and Riedl, J. Enhancing digital libraries with
TechLens+. In Proc. of ACM/IEEE JCDL 2004, ACM
Press (2004) 228-236.
Ziegler, C.N., McNee, S.M., Konstan, J.A., and Lausen, G., Improving Recommendation Lists through Topic Diversification. In Proc. of WWW 2005, ACM Press (2005), 22-32.
Zaslow, J. If TiVo Thinks You Are Gay, Here's How To Set It Straight --- Amazon.com Knows You, Too, Based on What You Buy; Why All the Cartoons? The Wall Street Journal, sect. A, p. 1, November 26, 2002.
Wednesday, August 23, 2006
Why Google is not a content-based recommender
Yesterday in the HMDB-bookclub, that we run in my unit, we read and discussed a paper on the recommender systems (Adomavicius & Tuzhilin, 2005, Toward the Next Generation of Recommender Systems). As this is my topic of research I was very eager to hear how my study-buddies perceived the issue and what did they have to say.
The discussion lingered into understanding the two main trends to produce recommendations: the content-based (CB) and collaborative recommendation. There were questions and attempts to answer them which left me unsatisfied after the session. Mainly, we left with the impression that Google, or any information retrieval system, would be, at the end of it, just a content-based recommender. I was somewhat troubled with this though and set my self for the quest to understand better what is there to discover.
Let’s go first by definition: Konstan et al. (2005) say:
When Adomavicius et al (2005) talk about CB approach, they state that it has its roots in information retrieval and filtering research, but
In the regular Google search there is no account, whereas to produce both content-based (CB) and collaborative filtering (CF) recommendations we need an account that we can assign to the user. An individual user profile is build based upon this.
In the CB recommendation a user is recommended items similar to the ones preferred in the past. This means that we need a search history, i.e. a user profile, where we can identify what the user has preferred in the past.
Thus, to generate a rather complete user profile that can find similarities between items (not people!) things like a history of viewed paged, bookmarked pages, the purchase history, “wish list”, and things like heurestic text analysis, etc. are important (implicit rating/input). Conventionally, especially with the first generation of recommenders the explicit ratings were the top notch:
Additionally, many times the CB systems would use additional information such as demographic, specific interests, location, etc that is part of the user’s self-manifested profile for the input. Maybe in the future this type of information could be extracted from some other sources, such as blog-postings, as were suggested during the session.
So, to get closer to the answer to the question, whether Google is just a content-based recommender, we can say that if used anonymously, it is not, although probably many of the techniques are the same. However, if we think of Google Personalized Search (beta) it for sure gets to be one.
The second somewhat baffling issues was the name of collaborative filtering, as it turns out, there is no collaboration between the users to produce any recommendations. In the CF recommendation the user is recommended items that people with similar tastes and preferences liked in the past. This means that we need a history for this person, too, in order to find out similarities within tastes and past experiences.
The strength of the CF approach at this stage is that even if you personally haven’t seen a link, product or what ever object we are talking about, or indicated the system what you liked about it, there most likely is someone in your nearest neighbourhood who has indicated that. Thus, in CF the values used to compute the recommendations are inferred based on similarities on the profiles, and you don’t need to have necessarily done it yourself. So, here lies the one main divider between CB and CF as for the input for the recommender: CB only uses YOUR history, whereas CF uses other users’ search history to better understand, or guesstimate, your history.
Well, this is stuff explained in short, more and better arguments are found in the papers and in my links at: http://www.furl.net/members/vuorikari/recommendation
Adomavicius & Tuzhilin, 2005, Toward the Next Generation of Recommender Systems
J.A. Konstan, N. Kapoor, S.M. McNee, and J.T. Butler. "TechLens: Exploring the Use of Recommenders to Support Users of Digital Libraries". A Project Briefing at the Coalition for Networked Information Fall 2005 Task Force Meeting, Phoenix, AZ, December 2005.
http://www.grouplens.org/papers/pdf/CNI-TechLens-Final.pdf
The discussion lingered into understanding the two main trends to produce recommendations: the content-based (CB) and collaborative recommendation. There were questions and attempts to answer them which left me unsatisfied after the session. Mainly, we left with the impression that Google, or any information retrieval system, would be, at the end of it, just a content-based recommender. I was somewhat troubled with this though and set my self for the quest to understand better what is there to discover.
Let’s go first by definition: Konstan et al. (2005) say:
Unlike ordinary keyword search systems, recommenders attempt to find items that match user's tastes and the user’s sense of quality, as well as syntactic matches on topic or keyword. For example, a music recommender will use an individual’s prior taste in music to identify additional songs or albums that may be of interest.
When Adomavicius et al (2005) talk about CB approach, they state that it has its roots in information retrieval and filtering research, but
the improvements over the traditional information retrieval approaches comes from the use of user profiles that contain information about user’s tastes, preferences, and needs. The profiling information can be elicited from users explicitly, e.g., though questionnaires, or implicitly- learned from their transactional behavior over time.
In the regular Google search there is no account, whereas to produce both content-based (CB) and collaborative filtering (CF) recommendations we need an account that we can assign to the user. An individual user profile is build based upon this.
In the CB recommendation a user is recommended items similar to the ones preferred in the past. This means that we need a search history, i.e. a user profile, where we can identify what the user has preferred in the past.
Thus, to generate a rather complete user profile that can find similarities between items (not people!) things like a history of viewed paged, bookmarked pages, the purchase history, “wish list”, and things like heurestic text analysis, etc. are important (implicit rating/input). Conventionally, especially with the first generation of recommenders the explicit ratings were the top notch:
...mid-1990s when researchers started focusing on recommendation problems that explicitly rely on the ratings structure. In its most common formulation, the recommendation problem is reduced to the problem of estimating ratings for the items that have not been seen by a user...Once we can estimate ratings for the yet unrated items, we can recommend to the user the items(s) with the highest estimated ratings(s) (Adomavicius, 2005)
Additionally, many times the CB systems would use additional information such as demographic, specific interests, location, etc that is part of the user’s self-manifested profile for the input. Maybe in the future this type of information could be extracted from some other sources, such as blog-postings, as were suggested during the session.
So, to get closer to the answer to the question, whether Google is just a content-based recommender, we can say that if used anonymously, it is not, although probably many of the techniques are the same. However, if we think of Google Personalized Search (beta) it for sure gets to be one.
The second somewhat baffling issues was the name of collaborative filtering, as it turns out, there is no collaboration between the users to produce any recommendations. In the CF recommendation the user is recommended items that people with similar tastes and preferences liked in the past. This means that we need a history for this person, too, in order to find out similarities within tastes and past experiences.
The strength of the CF approach at this stage is that even if you personally haven’t seen a link, product or what ever object we are talking about, or indicated the system what you liked about it, there most likely is someone in your nearest neighbourhood who has indicated that. Thus, in CF the values used to compute the recommendations are inferred based on similarities on the profiles, and you don’t need to have necessarily done it yourself. So, here lies the one main divider between CB and CF as for the input for the recommender: CB only uses YOUR history, whereas CF uses other users’ search history to better understand, or guesstimate, your history.
Well, this is stuff explained in short, more and better arguments are found in the papers and in my links at: http://www.furl.net/members/vuorikari/recommendation
Adomavicius & Tuzhilin, 2005, Toward the Next Generation of Recommender Systems
J.A. Konstan, N. Kapoor, S.M. McNee, and J.T. Butler. "TechLens: Exploring the Use of Recommenders to Support Users of Digital Libraries". A Project Briefing at the Coalition for Networked Information Fall 2005 Task Force Meeting, Phoenix, AZ, December 2005.
http://www.grouplens.org/papers/pdf/CNI-TechLens-Final.pdf
Wednesday, August 16, 2006
Yet Another Summer school: The Present and Future of Recommender Systems
September 12-13, 2006 Bilbao, Euskadi
Now I am very exited, I was accepted to a summer school, well, rather a kinda corporate conference, on "The Present and Future of Recommender Systems".
It'll be kick-ass, even Chris Anderson, The Long Tail-guy and John Riedl from the GroupLens will be among speakers!
The event is organised by MyStrands, an online music service, a recommender for music with all the gadgets; using tags, recommending related music, tracking trends, etc. The easy way to input the system is to allow it to hook to your iTunes. They'll check all the music you have there, and recommend something to you that you probably like.
I'm testing it a bit, so now when I turn my iTunes on, it turns the recommender on, and I get some 5 to 10 items that the system thinks I'd like. You can rate them, tag songs, buy them (oh really?!) and also find similar profiles of other users. The site looks good and seems to work pretty well. However, I guess I should really use it more than a few times to know if it really works, and I guess I should at least try and buy a few tunes - just to see whether the match is made in heaven! And, ...just to see whether they can really persuade to me the desired action from their side - to consume.
There's been some interesting usability studies on recommender system by Kirsten Swearingen & Rashmi Sinha a few years back. These were somewhat out of the general strand of the research in the field, as they asked for users' opinions (really!) on the usability issues and most importantly, whether they liked the recommendations. It seems to me, when reading the literature review, that most studies don't even give a hec whether the systems will ever be used by end-users, but they are busy proving the accuracy of algorithms in some bizarre mathematical ways. They come with stuff - genre - yes, this recommender works, it recommends to the user books by her favourite author. Right, just like one didn't know that...
This brings me to think what is it actually that people might want from a recommender system? Do they want it to help them to discover new items that they are not aware of, or just give good secure recommendations on the items that they already feel comfortable with?
Kirsten Swearingen & Rashmi Sinha go for "Different Strokes for Different Folks"-approach on user's needs:
Kirsten Swearingen & Rashmi Sinha
ps. the pic was done by Adam, Jehad's son. I will have to ask him for cc-licence, so for now it's copyrighted.
Now I am very exited, I was accepted to a summer school, well, rather a kinda corporate conference, on "The Present and Future of Recommender Systems".
It'll be kick-ass, even Chris Anderson, The Long Tail-guy and John Riedl from the GroupLens will be among speakers!
The event is organised by MyStrands, an online music service, a recommender for music with all the gadgets; using tags, recommending related music, tracking trends, etc. The easy way to input the system is to allow it to hook to your iTunes. They'll check all the music you have there, and recommend something to you that you probably like.I'm testing it a bit, so now when I turn my iTunes on, it turns the recommender on, and I get some 5 to 10 items that the system thinks I'd like. You can rate them, tag songs, buy them (oh really?!) and also find similar profiles of other users. The site looks good and seems to work pretty well. However, I guess I should really use it more than a few times to know if it really works, and I guess I should at least try and buy a few tunes - just to see whether the match is made in heaven! And, ...just to see whether they can really persuade to me the desired action from their side - to consume.
There's been some interesting usability studies on recommender system by Kirsten Swearingen & Rashmi Sinha a few years back. These were somewhat out of the general strand of the research in the field, as they asked for users' opinions (really!) on the usability issues and most importantly, whether they liked the recommendations. It seems to me, when reading the literature review, that most studies don't even give a hec whether the systems will ever be used by end-users, but they are busy proving the accuracy of algorithms in some bizarre mathematical ways. They come with stuff - genre - yes, this recommender works, it recommends to the user books by her favourite author. Right, just like one didn't know that...
This brings me to think what is it actually that people might want from a recommender system? Do they want it to help them to discover new items that they are not aware of, or just give good secure recommendations on the items that they already feel comfortable with?
Kirsten Swearingen & Rashmi Sinha go for "Different Strokes for Different Folks"-approach on user's needs:
- Reminder recommendations, mostly from within genre (“I was planning to read this anyway, it’s my typical kind of item”)
- More like this” recommendations, from within genre, similar to a particular item (“I am in the mood for a movie similar to GoodFellas”)
- New items, within a particular genre, just released, that they / their friends do not know about
- “Broaden my horizon” recommendations (might be from other genres)
Kirsten Swearingen & Rashmi Sinha
ps. the pic was done by Adam, Jehad's son. I will have to ask him for cc-licence, so for now it's copyrighted.
Friday, August 04, 2006
Showcase demonstration on the absurdity of software patents
This week's buzz has been the press release by Blackboard Inc. that announced, well, that Blackboard actually has invented e-learning or at least, the virtual learning environments. S.Downes gives a good run down on the blog postings on the issue here and in his today's OLDaily.
Like many have noted and protested against, there is plethora of cases of prior-art on what Blackboard Inc. claims to have invented, and what the US. Patent authority has granted them.
A great initiative called "History of virtual learning environments" has started in Wikipedia that currently is collecting and documenting our e-learning history in a form of cases of prior-art in the area of virtual learning environments. This will be an indispensable source of information, sort of poor man's portfolio of counter arguments, whenever it comes to a patent litigation in court over this given issue. Which, it seems, could be anticipated; it sounds like Blackboard is giving some indication that it might be using its 30-global-patents-and-patents-pending-portfolio aggressively (from the FAQ:
Moreover, the saga continues to other countries and continents where Blackboard Inc. has deposit patent claims for the same patent. To be precise, we here at the EU-land have also had our share of attention: the European Patent Office's database has a record of a pending claim on "EP1192615: Software Patent: Internet-based education support system and methods".
How will that effect on our life is still in the mist. It is a known fact that the EU is still giving a priority to strengthen Intellectual Property Rights, but where do software patents stand on that seem pretty cryptic to a common citizen.
After the last Public Consultation and public hearing on future patent policy in Europe in July, it seems that the two DGs in the European Commission can't find a common message to sent out, whereas the European patent litigation agreement (EPLA) is being set up all without the Commission's involvement by the the European Patent Convention (EPC).
Hey, little things count: we started collecting names on the petition to raise awareness against possible e-learning patents in Europe last Spring. This was to flag our concerns to people who are preparing the Public Consultation and public hearing on future patent policy in EU.
Keep discussing about this issue and ask your colleagues and friends to sign the petition online! It's good to show to our policy-makers and corporate folks that there are many people who do not want to be part of the software patent hell, but just get on with our work.
The petition to sign: http://flosse.dicole.org/?item=don-t-allow-software-patents-to-threaten-technology-enhanced-learning-in-europe
eusoftpat.
Like many have noted and protested against, there is plethora of cases of prior-art on what Blackboard Inc. claims to have invented, and what the US. Patent authority has granted them.
A great initiative called "History of virtual learning environments" has started in Wikipedia that currently is collecting and documenting our e-learning history in a form of cases of prior-art in the area of virtual learning environments. This will be an indispensable source of information, sort of poor man's portfolio of counter arguments, whenever it comes to a patent litigation in court over this given issue. Which, it seems, could be anticipated; it sounds like Blackboard is giving some indication that it might be using its 30-global-patents-and-patents-pending-portfolio aggressively (from the FAQ:
"My institution doesn't use a Blackboard system but uses a competitor’s course management system. How are we affected?"Just imagine all the CMS providers freaking out on this! Another story is whether anyone can afford opposing this patent in court, as potential targets might be open source initiatives like Moodle, Sakai, etc. and the educational institutions using them. However, like Mr. Attwell notes: let's hope that big companies will take care of the fight: SAP apparently has pending e-learning patents, too.
Answer: Evaluating patents can be complex and because we don’t know the specifics of how your system works, we would encourage you to consult with your CMS provider for answers."
Moreover, the saga continues to other countries and continents where Blackboard Inc. has deposit patent claims for the same patent. To be precise, we here at the EU-land have also had our share of attention: the European Patent Office's database has a record of a pending claim on "EP1192615: Software Patent: Internet-based education support system and methods".
How will that effect on our life is still in the mist. It is a known fact that the EU is still giving a priority to strengthen Intellectual Property Rights, but where do software patents stand on that seem pretty cryptic to a common citizen.
After the last Public Consultation and public hearing on future patent policy in Europe in July, it seems that the two DGs in the European Commission can't find a common message to sent out, whereas the European patent litigation agreement (EPLA) is being set up all without the Commission's involvement by the the European Patent Convention (EPC).
Hey, little things count: we started collecting names on the petition to raise awareness against possible e-learning patents in Europe last Spring. This was to flag our concerns to people who are preparing the Public Consultation and public hearing on future patent policy in EU.
Keep discussing about this issue and ask your colleagues and friends to sign the petition online! It's good to show to our policy-makers and corporate folks that there are many people who do not want to be part of the software patent hell, but just get on with our work.
The petition to sign: http://flosse.dicole.org/?item=don-t-allow-software-patents-to-threaten-technology-enhanced-learning-in-europe
eusoftpat.
Monday, July 17, 2006
If the White Lady is not visible, at least WiFi in Estonia is
Finding an open wireless network is a piece of cake in Estonia (currently 820 available), almost in every village there is some available, not even to mention bigger cities.
This photo is from the Haapsalu castle where the famous White lady sometime appears. If she's not around, look for a hot spot.
This photo is from the Haapsalu castle where the famous White lady sometime appears. If she's not around, look for a hot spot.
Requirements for Learning repositories
I was recently in a conference where this project was brought to my attention. Requirement laid out include support for Creative Commons, Learning Design, being distributed, etc.
The OpenDock project sees a repository as a thin layer which sits between your file system and the rest of the net, controling access and providing information on what is being shared. T
The Bazaar - Bazaar project » What do we want from a FLOSS repository?
technorati tags:LOR, opensource
Blogged with Flock
Subscribe to:
Posts (Atom)
