[Sm]all things considered by r.vuorikari

My favourite teen girls blogging

2012-01-30T22:37:00.005+01:00

I was checking quickly the web for references on teens, especially girls, and how do they use social media nowadays. Having an opportunity to spend time with my nieces every once in a blue moon, it's fun to see how they embrace social media.

14% of online teens now say they blog, down from 28% of teen internet users in 2006. (Pew, 2010)

Teens are now beginning to resemble their elders in their likelihood of blogging, as about 12% of adults have consistently reported blogging since February 2007. This decline is also reflected in the decline of the number of teens (Pew, 2010)

Last summer I helped my god-daughter (11), to start her own blog. Today, I received an email saying she had started a new one and had deleted the old one. This one is about her favourite receipts, she likes to bake muffins. A few hours later came a mail from her older sister (15) about her new blog, on nails, a topic she is, ehem, rather obsessed with.

Even before starting their own blogs, they've been reading blogs from other teens a lot. Mostly about hobbies, like pets and nails. It came out last summer, kind of a roundabout way, that my god-daughter would maybe be interested in starting her own blog. I asked her if she wanted us to do it together, and so we sat down to get it started, check how to post, choose templates, etc. By now, she's taught it to her older sister, who after a long time contemplating on the name of her blog (this was a major discussion item over the Christmas), finally got started with hers.

I enjoy seeing them as "doers" on the Web rather than as just readers and passive receivers. It's also fun that it's something that I can help them with. I think it's funny, though, that they are interested in blogs! No one blogs these days anymore, after all, everything can be expressed in 140 characters! - but I guess teenage girls - and me...wonder if this is a start of a new trend?

Open and Social Technologies for Networked Learning (OST '12)

2012-01-05T12:40:00.001+01:00

30.July – 3.August 2012
Tallinn University, Tallinn, Estonia

Read more: http://ifip-ost12.tlu.ee

In 2011, Tallinn (Estonia) is the European Cultural Capital. In 2012, Tallinn will host an IFIP open conference on “Open and Social Technologies for Networked Learning”. Sponsored by IFIP Working Group 3.4 (Professional Education), the conference is jointly organized by Tallinn University and University of Tampere (Finland).

Open and Social Technologies play an increasingly important role in many educational settings. Social technologies are naturally entering primary, secondary and higher education where they blur the boundaries between formal and informal learning. Social technologies also enter
the workplaces where they connect learners and bridge the boundaries between individual learning and organizational knowledge processes.

Not only do these technologies connect learners independent of place and time, they have also been found to exert emergent properties. For example, wikis or social tagging environments are increasingly used for collaborative knowledge construction where new knowledge emerges
from a large scale interaction of individuals. These properties and their impact on individual, group and organizational learning have only started to be researched.

Open Source Software (OSS) and Technologies have received extensive
research attention due to some favorable properties contrasting with a
traditional understanding of software development and the use of those
systems. Many OSS issues are motivations for OSS developers and
licensing bodies. However, important research areas in OSS are product
and implementation success and the use of OSS in different educational
and enterprise settings. OSS can also serve as a platform for
providing services to user communities. Especially in developing
countries, OSS provides an attractive opportunity.

We invite Contributions for the following topics

Social Technologies in Education (Weblogs, Tagging, Wikis, Microblogging, Social Networking)
Open Technologies in Education (Open Source Software, Standards, Licensing, Linked open Data)
Technologies for networked learning (Personal Learning environments, Virtual Learning environments, Mobile Learning)
Collaborative and networked learning (Collaborative Knowledge Building, Community-based Learning)
Socio-Technical Systems and Digital Ecosystems in Education,
Models and Networks (Learning Analytics and User Modelling, Emergent Properties, Social Network Analysis, Educational uses of data mining)
Design, Development and Evaluation Methodologies for Open and Social Systems
Application of Technologies in Educational Settings (Pre-School, Primary and Secondary Schools, Higher Education)
Application of Technologies in Professional Education and the Workplace (Workplace Learning, Learning in Business Networks and Alliances, Learning and Knowledge Management, Learning and Innovation)

Revised full and short papers will be published in the Conference Book
to be published by Springer Publishers as a special volume in
accordance with the high standards associated with IFIP publications.

Important dates

15 January 2012: Submission of long (8 pages) and short (4 pages)
papers, submission of symposia (2 page description)

28 February 2012: Notification of acceptance for long and short papers
and for symposia

15 March 2012: Submissions for Doctoral Consortium, Posters, Demos,
Discussion Groups

15 April 2012: Submission of Revised Camera Ready long and short
papers, Notification of acceptance Doctoral Consortium, Posters,
Demos, Discussion Groups

29 July 2012: Meeting of the Doctoral Consortium

30 July - 03 August 2012: Conference at Tallinn University

Conference Chairs

* Tobias Ley, Tallinn University, Estonia
* Mikko Ruohonen, University of Tampere, Finland

Local Organisation Chair

* Mart Laanpere, Tallinn University, Estonia

Chair of the Editorial Board

* Arthur Tatnall, Victoria University, Melbourne, Australia

Workshop on Learning Technology for Education in Cloud (LTEC'12)

2012-01-05T12:38:00.000+01:00

In conjunction with KMO'12 Conference
11th -13th July, 2012
Salamanca, Spain

Read the full call: http://ltec.usal.es/

The use of technology for learning has grown tremendously in the last decade. The need for continuous just-in-time training has made learning technology an indispensible part of life for workers. Learning technology is a type of system that provides educational services to students. Cloud Computing is perfectly placed to enable the learning technology providers to have an infrastructure that enables growth and at the same time save them considerable cost. Furthermore, it allows providers to deliver affordable solutions to universities along with the tools students require to maximise their time and their employability. Nowadays, we are living in a world of increased mobility where proliferation of mobile technologies is creating a host of new anytime and anywhere contexts. The emerging social media of Web 2.0 are more flexible, sociable and more visually attractive. We live and learn in a connected world. Schools, colleges and universities must change to adapt to these new needs and expectations. This highlights the need for innovative solutions in education and learning.

A day in 2061 - TEDxBrussels

2011-11-22T23:00:00.004+01:00

What a day in TEDxBrussels- 35 speakers of a huge variety talking about the deep future, namely what will it be like in 2061.

Very inspiring, but requires lots of thinking and making connections between things that did not exist before or did not click together before (e.g. much talk about singularity, so I really had to re-fresh my memories about Kurtzweil's book, a good thing that Matt was there to help me!).

Here is my top talks:

Leila Janah Absolutely the best "TEDTalk" of the day, just like what you would expect. She's smart, has a dream, then a vision and an idea, and able to make things happen. This talk was actually very inspiring for me, I really dig her idea and the fact that she was able to make it happen (and make it to TEDxTalks :)

Peter Hinssen This guy had a challenge to be the 34th speaker of the day - boy, we were pretty tired at that point! He was good and able to energise us - and he talked about health care (about the most boring topic on my list). Regardless, he made it interesting and he made strong statements, like that he's the S-curve guy (as opposed to exponential curve, which was the underlying message of the remaining 30 talks).

Rudy Rucker (video not yet online when I posted, must be coming soon) I like some of the science fiction just because it's so freaking innovative and mind-blowing. So this sweet looking, gray-haired dude turns out to be one of the early CyberPunks - and he is still totally out there! Check his predictions for year 3000 - we no longer produce things, we grow them.

Rob Spence One particular thing about TEDTalks is that they are always sooo personal. They always tell something touchy about themselves, which after a few rounds gets pretty boring. Not so with this guy. After loosing his eye, he now replaced it with a video camera and he's talk is about pimping up humans with machines (the cyborg stuff) - really cool.

Andrew Hassel A talk about science and biology in 2061. Very well composed and making a point about open science.

Eileen Bartholomew Her talk is informative if you are interested in snooping out what the next X PRIZE challenge might be.

Probably any of the other videos are also worth of your 8 or 18 minutes. Some of the space travel stuff was fascinating too.

About dual licencing: Creative Commons and commercial work

2011-11-07T09:47:00.010+01:00

Can I make money with Creative Commons?

A FAQ answer by CC
Creative Commons Plus (CC+), initially it's the same as dual licencing in open source world
Example of dual licencing on Flickr with Getty image.
Example of dual licencing on Magnatunes.
Examples of making money in education
E.g. offering print-on-demand (POD) copies at reasonable prices: MoveMeOn-book: pdf is made available for free and printed book at a price
See a video by a commercial textbook publishing company that is leveraging CC licenses as part of their business model (Eric Frank, co-founder of Flat World Knowledge)
A nice slideshare about Creative Commons and Open Educational Resources

Examples of how other people can use your photos under Creative Commons (CC-BY-SA)

My travel photo from Prague used in an online travel guide
My travel photos from South-Pacific
Btw, I was asked if I wanted to be part of this
Other examples that I was not aware of, but they were allowed to use my picture.

custom license

Workshop: Social and Personal Computing for Web-Supported Learning Communities (SPeL 2011)

2011-08-12T16:12:00.001+02:00

The website with the call for papers is at: http://software.ucv.ro/~epopescu/spel2011 and the schedule for SPeL 2011 is as follows:

September 7, 2011 Abstract submission
September 10, 2011 Workshop paper submission
October 5, 2011 Workshop paper notification
October 15, 2011 Camera ready paper & Registration
December 8-10, 2011 ICWL 2011 conference

The workshop follows the previous SPeL 2008, SPeL 2009 and SPeL 2010 workshops. The general topic of the workshop is the social and personal computing for web-supported learning communities.

Web-based learning is moving from centralized, institution-based systems to a decentralized and informal creation and sharing of knowledge. Social software (e.g., blogs, wikis, podcasts, media-sharing services) is increasingly being used for e-learning purposes, helping to create novel learning experiences and knowledge. In the world of pervasive Internet, learners are also evolving: the so-called "digital natives" want to be in constant communication with their peers, they expect an individualized instruction and a personalized learning environment, which automatically adapt to their individual needs.

Call: Datasets and Data Supported Learning in Technology-Enhanced Learning

2011-07-15T12:22:00.000+02:00

************************************************

CALL FOR JOURNAL PAPERS

Special Issue on dataTEL
“Datasets and Data Supported Learning in Technology-Enhanced Learning”

International Journal of Technology Enhanced Learning (IJTEL)
ISSN (Online): 1753-5263 - ISSN (Print): 1753-5255

Deadline of submissions: 25 October 2011

************************************************

SCOPE

The prospect of great growth of open and linked data in the knowledge society creates opportunities for new insights through advanced analysis methods based on e.g., information extraction, filtering, and retrieval technologies. Educational institutions also create and own large datasets on their students’ and course activities. The analytic use of such data, however, is very limited, when considering new educational services, recommending suitable peers or content or processes or goals, and improving the personalization of learning. Nevertheless, personalized learning is expected to have the potential to create more effective learning experiences, and accelerate learners’ time-to-competence. In the educational world, the literature is sparse on how to build upon today’s very limited public datasets and how to accommodate the lack of agreed quality standards on the personalization of learning.

The special issue on dataTEL in IJTEL aims to address this issue by collecting high value research papers to develop a body of knowledge about data-based personalization of learning. So far, there is no consensus on algorithms that can be successfully applied to make reliable analyses of data in a specific learning setting. Having an initial collection of datasets, coupled with case studies of their use in TEL, could be a first major step towards a theory of personalisation within TEL that can be based on empirical experiments with verifiable and valid results.

However, data driven research confronts researchers with a new set of challenges, for instance, a lack of common dataset formats or policies to share educational datasets, a huge variety of different evaluation methods for comparing diverse personalization techniques, and new ethical and privacy issues that arise from the ability to link and mine information.

Therefore, the objective of this special issue is to explore suitable datasets for TEL – with a specific focus on recommender and information filtering systems that can take advantage of these datasets. In this context, new challenges emerge like unclear legal protection rights and privacy issues, suitable policies and formats to share data, required pre-processing procedures and rules to create sharable data sets, common evaluation criteria for recommender systems in TEL and how a data set driven future in TEL could look like.

TOPICS

Relevant topics include, but are not limited to:

- descriptions of datasets that can be used for experimentation

- descriptions of data experiments (methods or results of experiments)

- experiences with those datasets

- dealing with legal protection rights towards datasets on a European level

- privacy preservation for educational datasets

- methods of effective anonymisation of educational datasets

- management and pre-processing procedures for educational datasets

- future scenarios for educational datasets

- impact of educational datasets for learners, teachers, and parents

- mash-ups based on educational datasets

- recommender approaches that are based on educational data

- evaluation methodologies and metrics for educational recommender systems

SPECIAL ISSUE CO-EDITORS

Hendrik Drachsler, Open University, The Netherlands

Katrien Verbert, K.U. Leuven, Belgium

Miguel-Angel Sicilia, University of Alcalá, Spain

Nikos Manouselis, Agro-Know Technologies, Greece

Stefanie Lindstaedt, KnowCenter, Austria

Martin Wolpers, Fraunhofer Institute for Applied Information Technology, Germany

Riina Vuorikari, European Schoolnet, Belgium

SUBMISSIONS

Authors are invited to submit original unpublished research as papers. All submitted papers will be peer-reviewed by at least two members of the program committee for originality, significance, clarity, and quality. In addition, the authors are asked to contribute short abstracts of their submissions to the dataTEL group space at TELeurope.

Submission will be available through the EasyChair submission system:

http://www.easychair.org/conferences/?conf=datatel2011

Details of the journal, manuscript preparation are available on the here: http://www.inderscience.com/www/authorguide.pdf

Any questions and submissions should be sent to: hendrik.drachsler@ou.nl

REVIEW COMMITTEE (to be confirmed)

Erik Duval, K.U. Leuven, Belgium

Seda Gurses, K.U. Leuven, Belgium

Abelardo Pardo, University Carlos III of Madrid, Spain

Julià Minguillón, Open University of Catalonia, Spain

Olga Santos, aDeNu, Spanish National University for Distance Education, Spain

Julien Broisin, Université Paul Sabatier, France

Christoph Rensing, TU Darmstadt, Germany

Shlomo Berkovsky, CSIRO, Australia

John Stamper, Datashop, Pittsburgh Science of Learning Center, USA

Eelco Herder, Forschungszentrum L3S, Germany

Martin Memmel, DFKI, Germany

Xavier Ochoa, Escuela Superior Politécnica del Litoral, Ecuador

Fridolin Wild, KMI, Open University, UK

Wolfgang Reinhardt, University of Paderborn, Germany

Wolfgang Greller, Open Universiteit, The Netherlands

Marco Kalz, Open Universiteit, The Netherlands

Adriana Berlanga, Open Universiteit, The Netherlands

Peter Sloep, Open Universiteit, The Netherlands

Ralf Klamma, RWTH Aachen, Germany

Pythagoras Karampiperis, NCSR Demokritos, Greece

Giannis Stoitsis, IEEE, Greece

IMPORTANT DATES

Submission of manuscripts: 25 October 2011

Completion of first review: 30 November 2011

Submission of revised manuscripts: 15 January 2011

Final decision notification: 10 February 2012

Publication date (tentative): February 2012

SUBMISSION GUIDELINES

The manuscripts should be original, unpublished, and not in consideration for publication elsewhere at the time of submission to the International Journal on Technology-Enhanced Learning and during the review process.

Please carefully follow the author guidelines at http://www.inderscience.com/mapper.php?id=31 while preparing your manuscript. To get familiarity with the style of the journal, please see a previous issue at http://www.inderscience.com/browse/index.php?journalID=246

All manuscripts will be subject to the usual high standards of peer review. Each paper will undergo double blind review.

Extended deadline: The 10th International Conference on Web-based Learning

2011-06-15T11:18:00.001+02:00

Full Paper Submission Deadline: Jun 30, 2011

The conference program consists of high quality technical papers that are reviewed and selected by an international program committee. Papers are solicited on all technical aspects of web-based learning and related technologies, including but not limited to the following topics:

Technology Enhanced Learning
Personalized and Adaptive Learning
Computer Support for Intelligent Tutoring
Intelligent Tools for Visual Learning
Web-based Learning for Oriental Languages Learning
Game-based Learning
Personal Learning Environments (PLE)
Computer Supported Collaborative Learning
Web 2.0 and Social Learning Environments
Intelligent Learner and Group Modeling
Human Factors and Affective Computing for Learning
E-Learning Platforms and Tools
Design, Model and Framework of e-Learning Systems
Deployment, Organization and Management of Learning Objects
E-Learning Metadata and Standards
Semantic Web and Ontologies for E-learning
Mobile, Situated and Blended Learning
Pedagogical Issues
Practice and Experience Sharing

Authors are invited to submit original papers reporting on research results or novel applications in web-based learning. All accepted full papers presented in the conference will be published as a volume in Springer Lecture Notes in Computer Science (LNCS). Selected papers will be recommended for possible publication in some international journals, including IEEE Trans. on Learning Technologies. Papers for submissions should be formatted in single column of no more than 10 pages in single line spacing according to the Springer LNCS Authors Guideline for a double blind review. Any identification information about the authors should NOT be included in the manuscript to facilitate double blind review. All papers should be submitted in PDF format. Please refer to the conference website (http://www.hkws.org/conference/icwl2011/) for more information and send us email at icwl2011@cs.cityu.edu.hk for any inquiry.

Building awareness in learning networks

2011-05-26T16:10:00.002+02:00

"I just joined the PC of the workshop "Awareness and Reflection in Learning Networks" at #ectel11 #arnets11 http://teleurope.eu/arnets11". What a great way to "build awareness", I was invited to join the PC and at the same time to let ppl know about it! Way to go, very clever ! :)

Questions for eTwinning conference in Slovenia

2011-04-13T18:11:00.002+02:00

Questionnaire: http://www.zoomerang.com/Survey/WEB22C8CJBYPCX/

Why we need big, authentic datasets for education?

2010-10-08T16:34:00.003+02:00

In the world of consumer recommender systems, it is a common practise to use different data sets as benchmarks to evaluate new recommender systems algorithms (e.g. MovieLens, Book-Crossing, EachMovie data set).

In TEL, there are no standardised data sets publicly available, so that the outcomes of different recommender systems within TEL are hardly comparable.

**********************************************
CALL FOR CONTRIBUTIONS

Workshop: dataTEL- Data Sets for Technology Enhanced Learning
Date: March 30th to March 31st, 2011
Location: Ski resort La Clusaz in the French Alps, Massif des Aravis
Funding: Food and lodging for 3 nights for 10 selected participants
Submissions: For being funded, please send extended abstracts to
http://www.easychair.org/conferences/?conf=datatel2011

Deadline for submissions: October 25th, 2010
**********************************************

1st workshop on ëData Sets for Technology Enhanced Learningí at the 2nd STELLAR Alpine Rendez-Vous in La Clusaz, France

SCOPE
Personalisation and analysis of user interaction data is a key approach to overcome the plethora of information in the knowledge society. It is expected that personalised learning has the potential to reduce delivery costs, to create more effective learning environments and experiences, to accelerate study time, and to increase collaboration between learners.

Recommender systems and information filtering are some of the promising technologies to support people in finding most suitable information and peer learners. They are increasingly applied in Technology-Enhanced Learning (TEL) in various European projects in order to personalise learning content and connect suitable peer learners according to their context (e.g., individual needs, preferences, and learning goals).

In the world of consumer recommender systems, it is a common practise to use different data sets as benchmarks to evaluate new recommender systems algorithms (MovieLens, Book-Crossing, EachMovie data set). In TEL, there are no standardised data sets publicly available, so that the outcomes of different recommender systems within TEL are hardly comparable.

So far, no universally valid knowledge exists on algorithm that can be successfully applied in a certain learning setting. Having such data sets could be a first major step towards a theory of personalisation within TEL that can be based on empirical experiments with verifiable and valid results.

Therefore, the objective of this workshop is to explore suitable data sets for TEL ñ with a specific focus on recommender and information filtering systems that can take advantage of these data sets. In this context, new challenges emerge like unclear legal protection rights and privacy issues, suitable policies and formats to share data, required pre-processing procedures and rules to create sharable data sets, common evaluation criteria for recommender systems in TEL and how a data set driven future in TEL could look like.

Relevant topics include, but are not limited to:

publicly available data sets for educational systems
dealing with legal protection rights towards data sets on a European level
privacy preservation for educational data sets
methods of effective anonymisation of educational data sets
management and pre-processing procedures for educational data sets
future scenarios for educational data sets
impact of educational data sets for learners and teachers
mash-ups based on educational data sets
recommender approaches that are based on educational data
evaluation methodologies and metrics for educational recommender systems

SUBMISSIONS
Authors are invited to submit original unpublished research as papers (4-8 pages). Demonstrations and Hands-on sessions are explicitly encouraged. All submitted papers will be peer-reviewed by at least three members of the program committee for originality, significance, clarity, and quality. Re-worked versions of accepted submissions will be published in the Interdisciplinary Journal of E-Learning and Learning Objects (IJELLO). In addition, the authors are asked to contribute short summaries of their submissions to the dataTEL group space at TELeurope to encourage early information sharing and discussion also with third persons.

Submission will be available through the EasyChair submission system:
http://www.easychair.org/conferences/?conf=datatel2011

All questions and submissions should be sent to: dataForTEL@gmail.com

WORKSHOP STRUCTURE
Based on workshop submissions, the organisers will identify 4 most pressing research challenges. For each of the challenges, we will allow around 2 hours discussion, started off through about two short submission presentations or keynotes. The final 2 hours of the last workshop day will be utilised to bring the results of the individual discussions together and to chart a vision of the future of TEL amplified by publicly available data sets.

FUNDING
Participants pay for their own travel and lodging. But we are able to fund food and lodging for 3 nights as well as the community event on Tuesday, March 29th for 10 participants. Please indicate your funding needs with your submission.

ORGANIZERS
Katrien Verbert (K.U.Leuven, BE)
Riina Vuorikari (European Schoolnet, BE)
Stefanie Lindstaedt (KnowCenter, AT)
Martin Wolpers (Fraunhofer Institute for Applied Information Technology, DE)
Miguel-Angel Sicilia (University of Alcal·, ES)
Nikos Manouselis (Greek Research and Technology Network, GR),
Hendrik Drachsler (Open University of the Netherlands, NL)

IMPORTANT DATES
Submission of extended abstract (1000-2000 words): October 25th, 2010
Notification of acceptance of abstracts: November 15th, 2010
Submission of papers: December 17th, 2011
Workshop: March 30th to March 31st, 2011

IMPORTANT WEBSITES
dataTEL group space: http://www.teleurope.eu/pg/groups/9405/datatel/
The Alpine Rendez-Vous: http://www.stellarnet.eu/programme/wp3/rendez-vous
La Clusaz: http://www.laclusaz.com/hiver-winter10/index.php?lang=_uk

VENUE DETAILS
The second STELLAR Alpine Rendez-Vous (ARV) will take place in the French Alps in the Massif des Arravis at a beautiful ski resort called La Clusaz (hotel Alpen Roc). There will be some free time in the afternoons for TEL community building during winter activities. Funding for lodging and food will be available for a limited number of participants.

THE ALPINE RENDEZ-VOUS
This 2011 STELLAR Alpine Rendez-Vous is the second event of a series. It is organized and funded by STELLAR, a European network of excellence on learning technologies (http://www.stellarnet.eu/). The ARV is not a standard conference, but a set of independent workshops located at the same time in the same hotel. Four workshops run in parallel during the first part of the week and four during the second part. It's called "Rendez-Vous" because shared events are organized in the middle of the week (Tuesday evening) and because we set up breaks and meals in a way that promotes informal encounters between participants from the different workshops. Finally, it is called "Alpine" because it gathers scientists in the Alps, away from their workplace routines, in a place where snow is used as social facilitator."

Interesting opportunity: Professor of Digital Ecosystems

2010-06-04T16:46:00.003+02:00

I've always enjoyed the drive of Tallinn University. This job application shows that they have both a good hunch of what is going on, but also style. Application deadline already on June 10, 2010. Tallinn - why not?

The Institute of Informatics invites applications for a position of Professor of Digital Ecosystems.

http://htk.tlu.ee/~priit/ecosystems_prof/prof.html

The Institute of Informatics has undertaken a major development by evolving international programmes and research groups in ICT.

The significant part of ongoing research in the institute is focusing on social media ecosystems in the context of e-learning, e-governance, e-participation and network enterprise. Our international R&D team is using mainly participatory design research approach and intervention studies to envisage and develop prototypes of the next generation social media tools, with the special attention to their semantic interoperability, identity management, user experience design, activity pattern mining and semantic annotation for metadata.

The successful applicant for a professorship is expected to make a significant contribution to the development of the Interactive Media and Knowledge Environments master curriculum as well as of the Information Society Technologies PhD curriculum. (S)he is also expected to supervise PhD and master students, to be a mentor for less experienced teachers of the institute, to develop new initiatives and to start a research programme related to the field of professorship.

The employment period for the position is for 5 years and shall undergo re-elections after that. Tenure at Tallinn University can be achieved after two successful periods. The new position has been established with a significant financial contribution from European Social Fund.

Details of the position and application procedures: Tallinn University Employment Rules Tallinn University, www.tlu.ee. Institute of Informatics, http://www.tlu.ee/informatics Deadline of applications: June 10, 2010. Expected start of the work January 1st, 2011.

Workshop on Recommender Systems for Technology Enhanced Learning (RecSysTEL)

2010-05-31T12:39:00.007+02:00

Deadline extended: July 1st

This year's continuation for our previous work on SIRTEL-workshops is a jointly organised workshop called RecSysTEL in conjunction with:

4th ACM Conference on Recommender Systems (RecSys 2010)
5th European Conference on Technology Enhanced Learning (EC-TEL 2010)

It takes place in Barcelona, Spain, 29-30 September 2010

KEYNOTE SPEAKER: Joseph Konstan, GroupLens Research, University of Minnesota (USA)

AIM & TOPICS

Technology enhanced learning (TEL) aims to design, develop and test socio-technical innovations that will support and enhance learning practices of both individuals and organisations. It is an application domain that generally addresses all types of
technology research & development aiming to support of teaching and learning activities. Information retrieval is a pivotal activity in TEL, and the deployment of recommender systems has attracted increased interest during the past years.

Recommendation methods, techniques and systems open an interesting new approach to facilitate and support learning and teaching. There are plenty a resource available on the Web, both in terms of digital learning content and people resources (e.g. other learners,
experts, tutors) that can be used to facilitate teaching and learning tasks. The challenge is to develop, deploy and evaluate systems that provide learners and teachers with meaningful guidance in order to help identify suitable learning resources from a potentially overwhelming variety of choices.

The aim of the Workshop is to bring together researchers and practitioners that are working on topics related to the design, development and testing of recommender systems in educational
settings as well as present the current status of research in this area and create cross-disciplinary liaisons between the RecSys and EC-TEL communities. Overall, it aims to outline the rich potential of TEL as an application area for recommender systems, as well as expose participants to the challenges of developing such systems in a TEL context.

Topics include but are not limited to:

User tasks to be supported by recommender systems in TEL
Focus of recommendation in TEL
Requirements for the deployment of TEL recommender systems
Publicly available data sets for TEL recommender systems
Recommendation algorithms and systems for TEL
Transfer of successful algorithms and systems from other application areas
Evaluation criteria and methods for TEL recommender systems

IMPORTANT DATES

20 June 2010: Submissions
16 July 2010: Notifications
1 August 2010: Camera-ready of accepted papers
29-30 September 2010: RecSysTEL Workshop in Barcelona

DATATEL CHALLENGE

Published datasets in recommender systems, such as the MovieLens and EachMovie ones, are very often used in experimental testing of new recommendation algorithms.
Very few datasets are publicly made available online for TEL applications.
Thus, it is not possible yet for TEL recommender systems' researchers to apply and benchmark their algorithms on existing, public datasets.

To this end, the dataTEL Theme Team of the European STELLAR Network of Excellence (http://www.teleurope.eu/pg/groups/9405/datatel/) is sponsoring the dataTEL Challenge: a call for TEL datasets that invites research groups to submit existing datasets from TEL applications that can be used as input for TEL recommender systems (e.g. ratings, tags, bookmarks).

The winner of the dataTEL Challenge will receive a best TEL dataset award.

More about the dataTEL Challenge:
http://www.teleurope.eu/pg/pages/view/9519/
http://adenu.ia.uned.es/workshops/recsystel2010/datatel.htm

SUBMISSIONS

The Workshop accepts a variety of submission types:

Full papers: 12 pages
Short papers: 6 pages
System/service demos: 2 pages
TEL Data sets: 2 pages and data set file (specs/format to be announced soon)

Papers should be original and not previously submitted to other venues.
Submission will be available through the EasyChair submission system:
http://www.easychair.org/conferences/?conf=recsystel2010

If you haven't an EasyChair account yet, you'll be asked to create it before you can access the RecSysTEL'10 page.

PUBLICATION

Workshop proceedings will be published in a seperate volume by a publisher
that will be announced soon.

In addition, authors of best full papers will be invited to submit a revised version
of their manuscripts for a Special Issue in a prestigious international journal
such as the IEEE Transactions on Learning Technologies.

STEERING COMMITTEE

Jesus G. Boticario, aDeNu - Spanish National University for Distance Education (Spain)
Peter Brusilovksy, University of Pittsburgh (USA)
Erik Duval, Katholieke Universiteit Leuven (Belgium)
Denis Gillet, Swiss Federal Institute of Lausanne (Switzerland)
Stefanie Lindstaedt, Know-Center Graz (Austria)
Peter Scott, Open University (UK)
Riina Vuorikari, European Schoolnet (Belgium)
Fridolin Wild, Open University (UK)
Martin Wolpers, Fraunhofer FIT (Germany)

PROGRAM COMMITTEE

Liliana Ardissono, Universita di Torino (Italy)
Katrin Borcea-Pfitzmann, Dresden University of Technology (Germany)
Julien Broisin, IRIT Universite Paul Sabatier (France)
Carlos Delgado Kloos, Carlos III University of Madrid (Spain)
Stavros Demetriadis, Aristotle University of Thessaloniki (Greece)
Jon Dron, Athabasca University (Canada)
Rosta Farzan, Carnegie Mellon University (USA)
Alexander Felfernig, Graz University of Technology (Austria)
Rick D. Hangartner, Strands (USA)
Eelco Herder, L3S (Germany)
Tsukasa Hirashima, Hiroshima University (Japan)
Ralf Klamma, RWTH Aachen University (Germany)
Martin Memmel, DFKI GmbH (Germany)
Pedro J. MuÒoz-Merino, Carlos III University of Madrid (Spain)
Brandon Muramatsu, MIT (USA)
Wolfgang Nejdl - L3S & Leibniz Universit‰t Hannover (Germany)
Xavier Ochoa, Escuela Superior Politecnica del Litoral (Ecuador)
Mimi Recker, Utah State University (USA)
Christoph Rensing, TU Darmstadt (Germany)
Hans-Christian Schmitz, Fraunhofer FIT (Germany)
Miguel-Angel Sicilia, University of Alcala (Spain)
Sergey Sosnovsky,DFKI GmbH (Germany)

CO-CHAIRS

Nikos Manouselis, Greek Research & Technology Network (Greece)
Hendrik Drachsler, Open Universiteit Nederlands (The Netherlands)
Katrien Verbert, Katholieke Universiteit Leuven (Belgium)
Olga C. Santos, aDeNu - Spanish National University for Distance Education (Spain)

ABOUT RecSys 2010 and EC-TEL 2010

The 4th ACM Conference on Recommender Systems (RecSys 2010) is the premier annual event
on research and applications of recommender technologies. It will promote a close interaction
among practitioners and researchers, reaching a wider range of participants
including those from Europe and Asia. See http://recsys.acm.org/2010/ for details.

The 5th European Conference on Technology Enhanced Learning (EC-TEL 2010) brings together technological developments, learning models, and implementations of new and innovative approaches to training and education. The conference traditionally explores how the synergy of multiple disciplines can provide new, more effective and more especially more sustainable, technology-enhanced learning solutions to learning problems. See http://www.ectel2010.org for details.

*************************************************************

What does it mean for a school to self-organise?

2010-03-16T22:59:00.007+01:00

I was taken back to this initial idea, can schools self-organise?, after reading an article by Weston & Blain (2010) called "The end of techno-critique: the naked truth about 1:1 laptop initiatives and educational change". The reading of the article is of course related to my current work where we are running a large-scale 1:1 initiative in 6 European countries. At the end of the article, which I absolutely recommend reading, the authors conclude:

...1:1 initiatives can be fertile ground for the creation of new-paradigm schools, the schools that are self-organizing. The widespread availability of laptop computers can be a driver for the more expansive efforts that must happen in order for schools to meet the educational needs of all students....While the original mission of 1:1 laptop computer initiatives did not include shifting of educational paradigm, turning those initiatives toward the creation of self-organizing schools may be the the way forward for techno advocates and critics alike.

It's a lot about "change" in general, and especially the change that has to take place at the whole school level, not only by individual teachers, which still seem to be a trend when looking at the ICT implementations.

In our eTwinning, for example, we have more than 90 000 teachers signed up in Europe, which can be considered a good success (about 1.85% of all European teachers!). The naked truth comes out, though, when you look at the penetration by schools: about 75% of the schools have only one single teachers signed up in eTwinning, whose mission statement is "The community for schools in Europe". The community of schools build by single teachers who do not collaborate within their own schools?

So back to self-organisation - what would it mean for a school to self-organise? The hypothesis that I used for my PhD to study self-organisation within learning resource repositories was this:

The main hypothesis is that the self-organisation aspect of a social tagging system on a learning resource portal helps users discover learning resources more efficiently. Moreover, user-generated tags make the system, which operates in a multilingual context, more robust and flexible.

In this case, we could hypothesise:

The self-organisation aspect of a school helps learners learn more efficiently. Moreover, it makes the school more robust and flexible.

Bonabeau & Meyer (2001) use the following terms when talking about social insects and how they self-organise:

Self-organisation (activities are neither centrally controlled nor locally supervised);
Flexibility (the colony can adapt to a changing environment);
Robustness (even when one or more individuals fail, the group can still perform its tasks).

Adapted to a school, these would be interpreted as:

Self-organisation (activities are neither centrally controlled nor locally supervised);
Flexibility (the school can adapt to a changing environment);
Robustness (even when one or more individuals fail, the school can still perform its tasks).

Wow, that would be powerful, or what!?? Could 1:1 really ever help schools to self-organise? I guess this is one more reason to keep working hard on our 1:1 pilot. The next questions would be; how can a school self-organise? and what is required for a school to self-organise?

When I first started to think of this question, I was attending The Big Ideas Fest in California organised by ISKME. I was thinking of what would be A REALLY big idea for education. I think this is a pretty darn big one...

Is eTwinning socially contagious?

2010-02-08T17:02:00.013+01:00

Last weekend I joined more than 500 eTwinners (www.etwinning.net) in the 5th annual conference that took place in Sevilla. Quite a fiesta! I co-ran 3 workshops which all had something to do with social networks, more or less. I jokingly tell teachers that I'd like them all get an eTwinning virus and spread it around when they go home. This "virus" is, of course, a good one (e.g. innovative use of web in ed.context) and it spreads through the social network that teachers have created by being part of eTwinning. From where my question: is, or can, eTwinning be socially contagious?

Often times nowadays when people talk about social networks, they actually talk about social media tools or web 2.0 stuff, where it is made easy to express your social ties and make them visible to others (can I friend you?). Underneath all that "stuff" lies the structure of the social network which is of interest to me. I consider a network

as a conduit for the propagation of information or the exertion of influence, and an individual's place in the overall pattern of relations determines what information that person has access to or, correspondingly, whom he or she is in a position to influence. A person's social role therefore depends not only on the groups to which he or she belongs but also on his or her position within those groups. (Watts, 2003, p. 48)

Moreover, Watts goes on to explain yet a different way to view the network, namely through weak ties, "which can be thought of as a link between individual- and group-level analysis in that they are created by individuals, but their presence affects the status and performance not just of the individual who "own" them but of the entire group to which they belong." - and this all leads to the new science of networks.

Duncan Watts's book Six Degrees (2003) is one of my favourite science-tainment (like edutainment) book. I always enjoy picking it up and re-reading it, I seem to understand some of the passages in a new light. Today I re-read the stuff about differences of spreading a virus and "social contagion", like a fab that spreads or cascades throughout the whole social network.

There are some similarities, like the fact that each individual has a different threshold (some get the virus easier than others) and that you have people around to spread it to ("to whom she or he pays attention to"). But "social contagion", unlike biological one, does not take place if the network is too well connected!

So when everyone is paying attention to many others, no single innovator, acting alone, can activate any one of them. ... In social contagion, remember, it is the relative number of "infected" versus "uninfected" - active versus inactive - neighbors that matter. (Watts, 2003, p. 240)

Studying fabs or innovations (e.g. the use of web in ed.context), the question is about the moment when the fab stops being a niche thing among early adapters and when it leaps to the larger general population. I too often have a feeling that eTwinning "preaches to the converted". So, I'm keen on understanding how eTwinning can step out of being a nice of early adapters and get the others "contaminated". Someone aired a good comment in the conference, we should not take eTwinners as a representative sample of educational community in general.

Step one: who is infected?

I ran a few analysis to get a better picture. It's hard to find the number of schools or teachers in all eTwinning countries. After some digging I found OECD's Stat extracts which has lots of good data, and was able to find the number of teaching staff for 23 out of 32 countries (close to OECD's definition for EU19). I think that's a pretty good proxy to go by, however, not sure how accurately our data aligns with them. I used the date as described in the image for 2007.

The data says there was 6 210 411.57 teachers (I love the 0.57 teacher!) working in those countries, out of which 76 367 have registered in eTwinning by this date. On average, each country has 1.83% of their teachers infected by the "eTwinning virus", median was 1.42%. The countries above median in descending order are: Estonia, Iceland, Slovak Republic, Czech Republic, Slovenia, Finland, Greece, Poland (still above average, too!), Spain, Luxembourg, Portugal and Sweden. Are you surprised? I am a bit...

The countries below median were: the United Kingdom, Turkey, Norway, Netherlands, Italy, Ireland, Hungary, Germany, France, Belgium and Austria.

Well, my question is not about the success of eTwinning initiative itself, but it's more about understanding what are the conditions (globally and locally), what are the individual thresholds and when can we expect the cascading effect to take place (it's all about one or more vulnerable neighbors who have one or more vulnerable neighbors who have...).

These are typical questions that people interested in the new science of networks ask, and I want to know more how it happens within educational context. I think eTwinning is a good virus to study that!

We've just stated the TeLLNet project with some top-notch partners, so I'm looking forward to dwell into this problematic later again!

From what are big educational ideas made?

2009-12-06T20:40:00.006+01:00

Silicon valley is an iconic example when people talk about innovation. To breed innovation, it's important to remember that it's not only about individuals with ideas, but about the right conditions, like a "large number of cutting-edge entrepreneurs, engineers and venture capitalists" (wikipedia), in the case of the Valley. On top of that, you add a flavour of Californian sea, sun and wine, and you season it with a bit of pioneering attitude, and voila!

I'm today in California for theBig Ideas Fest.

The three-day Big Ideas Fest is an immersion into collaboration and design with the focus on inspiring and modeling cutting-edge thinking in K-20 education.

The programme and the names of speakers are pretty impressive! I think there will be an interesting buzz in the air, as the event brings together cutting-edge educationalists, "doers and shakers" who are interested in brainstorming on big ideas that are needed to move education in the right direction. The session is starting in a few seconds, so I'll be reporting back later.

My degree of Doctor at Open Universiteit Nederland

2009-11-16T17:22:00.008+01:00

"..you hereby receive all rights associated with the degree of Doctor either by law or custom"

My Doctorate Board and me with the degree and a huge smile - what a day!

From left Prof. dr. G. Conole, Prof. dr. A. Littlejohn, Prof. dr. B. Berendt , me, Prof. dr. P.B. Sloep, Prof. dr. E.J.R. Koper (supervisor), Prof. dr. J. van Marle (chairman) and Beadle E. Vinken. And no, even if I'm a doctor now, I don't get to wear a funny hat!

I thank everyone involved in my research and all who sent me kind wishes and congratulations!

my 10 minutes of fame

View more presentations from Riina Vuorikari.

Slides and PhD download

Future privacy and security resaerch challenges in online social networks

2009-11-09T12:28:00.003+01:00

I attended this workshop a couple of days ago. It was an interesting mix of ppl with different backgrounds; computer scientists, researchers, lawyers, some practitioners & users like a SN provider, some artists who run social network services, and my interest was of course teachers' social network services like our eTwinning platform (more than 70000 teachers).

I enjoyed the keynote by Ronald Leenes from Tilburg University. A few projects that he mentioned, PrimeLife (Bringing sustainable privacy and identity management to future networks and services) and BROAD (Broadening the Range Of Awareness in Data protection).

EUN is now running a http://www.dataprotectionday.eu/ campaign, here is my favourite film about it!

Most scholarly opponent - PhD Ceremony in OUNL Nov 13 2009

2009-10-22T23:35:00.006+02:00

It's pretty exiting to prepare for the PhD ceremony. There are all kinds of little things to think about. I've never gotten married, but it sure sounds like the same thing. In the Netherlands, for example, we need to have paranymphs at the PhD ceremony.

Yes, I know, it sounds like I need to have two dwarfs by my side there, but apparently they are like a bride's maid or a best man. With a twist that in case there was a heated fight between me and the opponents during the defence, the paranymphs would defend me with swords. Or, you know, something similar. So far, all what I've seen is that they hand over a class of water, but - we've ain't seen nothing yet..

Another thing is the manner of speech, I am, for example, to address the members in my committee by saying "most scholarly opponent", if they are a professor, otherwise just "scholarly opponent" is fine. And they call me "esteemed candidate". Pretty theatrical :)

Finally, I needed to prepare 10 statements in advance, they are called stellingen. 4 of them are about my thesis and the rest are more general (some can even be a bit funny, see #10). But in general, they all should be something that I can defend.

The purpose of this is that when the opponent has not had time to read my thesis and come up with a question, they can read one from the list. This way they can have a well formulated question to ask. My advantage - I can prepare for it in advance. Kind of a funny game! It seems to me that the more bold the statement is, the better chances there are that one of the opponents reads it out loud. Let's see...

1. A learning resource portal in a multilingual context can be made more robust and flexible by interrelating conventional metadata and social tags. (this thesis)

2. Socal tags, represented as a triple (user,item,tag), open more sophisticated avenues for resource discovery across contexts, especially when it applies to cross-language and cross-country discoveries. (this thesis)

3. The triple (user,item,tag) can be used as a parameter to measure links between cross-language content that reside on heterogeneous repositories. It can be created a posteriori to content creation and link-setting, and it can be used to support and enhance a new type of link-following behaviour by end-users. (this thesis)

4. The discovery strategies based on Social Information Retrieval (SIR) methods allow users to spend less effort in finding relevant resources on a multilingual portal. (this thesis)

5. The notion of learning resources as content is too limiting.

6. Even if the current trend in information seeking behaviour is the Web, interpersonal ties still drive and support information seeking. Social search should be considered beyond individual’s Web-behaviour.

7. While studying the impact of new information and communication technologies on education and attainment, studying their out-of-school use should be considered as important as their in-school use.

8. In Technology-Enhanced Learning, like in other lo-fi high tech, the next best thing is “good ‘nuf” for most users.

9. Languages both unite and divide people.

10. Anyone involved in decision-making for educational purposes should read science fiction.

Educational take - danah boyd: American Teens & Social Media

2009-10-01T20:55:00.003+02:00

danah boyd has put something really important in words in this little video, it's related to the use of social media (applicable to all ICTs and mobile technologies) in education. If in a hurry, fast-forward to about 5.30 of the video:

Teens need to know boundaries, norms and how society works, and this should be taught through using social media, i.e. the tools that they use anyway .

The reason that "because kids are doing it" it not a good reason to start teaching about social media in schools. "Pedagogy and understanding of the tools, are two key pieces of knowledge that teachers must have before entertaining the idea of bringing social media into the classroom. Information sharing is relevant to education..."

A valid point made for teachers continuous professional development!

Invitation to the public defence of my PhD

2009-09-29T22:30:00.007+02:00

Hurrah, finally the day when I can announce the public defence of my PhD: Nov 13 2009 at 13.30. You are welcome to join the public defence of my PhD at the OUNL premises in Heerlen, NL!

Today I also submitted my manuscript to print. Here you can see the cover of the publication, which is also downloadable here.

Run of the day:
13.30 Public defence in OUNL
15.00 Reception (all welcome, RSVP)
19.30 Dinner in Brussels (RSVP)

Impact of ICT use on educational performance

2009-09-26T18:40:00.004+02:00

The question the top-dog politicians nowadays ask is the impact of ICT on educational performance. Especially in the school sector, this has been the trendy question since a few years, when the policy makers realised that they want to see some return on their investment (i.e. all the hardware put in schools). My favourite EUN report on recent research is The ICT Impact Report (01/2007).

This week at the OECD's New Millennium Learner conference South-Koreans presented another interesting study towards this direction. Prof. Heo's presentation is available here, check the pages from 20 onwards for the results. The study included 10% of the 10th graders in South-Korea (1071 students, random sampling). The design of the study looked at the use of ICT

Place: in-school and out-of-school use of ICT
Purpose: learning vs. entertainment use of ICT
Context: individual use vs. social use of ICT

Educational performance was divided into:

Cognitive domain
Affective domain
Socio-cultural domain

Significant impact was found on educational performance with:

Out-of school ICT use
ICT use for learning
ICT use in individual context

Note, in-school use did not yield any significant impact :/ More interestingly, out-of-school use of ICT for learning purposes had a positive correlation (r=0.520, p= 0.00) with cognitive domain of educational performance, which shows good news for informal context of learning.

Mindmap from Social Information Retrieval for Technology Enhanced Learning workshop

2009-09-20T13:26:00.002+02:00

Best Paper Award in ICWL'09

2009-09-01T16:46:00.007+02:00

The paper that I presented in ICWL'09, Are tags from Mars and descriptors from Venus? A study on the ecology of educational resource metadata, was awarded the Best Paper Award. That was pretty dam cool! The picture below shows how psyched I was to pick up the award at the gala dinner. You'll find the paper from here and a news release from here.

The picture is by Kevin Chen from RWTH, Aachen.

Ok, this is an inside joke: I've named the pic "la vengeance se mange très-bien froide", for those who know the story, you guessed that the timing of this award could not have been better! Thanks for the co-authors and the jury :)

Obama's OER vs. EU's OER

2009-08-17T21:37:00.004+02:00

Checking the news about Obama's announced initiative where a $500-million-dollar online-education plan is outlined with the idea that the money will buy online course material that is made freely available. The person who is pushing the plan within the administration, Mr. Smith, has previously worked with OER in Hewlett Foundation, which has put some $ 70 million in last years to support Open Educational Resources worldwide.

EU has put huge amounts on different research and development progremmes around digital educational resources since late nineteens, and especially under Lisbon 2010 agenda. I checked some figures just for recent times:

FP7 progrmamme for '09-'10 has €151 million euros for digital libraries and TEL
LPP puts € 7 billion for the programme from '07 to '13, more than a billion a year
eContent Plus has significant budgets also

The big difference is that the EU seldom actually puts money in developing the content, but actually innovation and services around the content. I think the reasoning is that the content comes from publishers and more and more from end-users, and they do not want to rock that boat too much. I actually cannot think of a single EU-project where the content creation is paid by EU. For example the big OUUK initiative was also funded by Hewlett Foundations, which nowadays gets the world-wide claim to OER fame.

This setting could offer an interesting comparison study in a few years time: do we see more uptake with content that is professionally developed and made available to educators and learners (i.e. Obama's model), or do we see that the EU model, where the focus is on services, but the content is not always up to par, (finally) produces some uptake?

I wanted to find the journal article mentioned in the post, but could not

In January he published an article in the journal Sciencelaying out the dream of "a 21st-century library" composed of Web-based open courses for high-school and college students. The courses would be laced with multimedia features and personalized with feedback from computer programs that track student performance. The language coming out of the White House and Education Department today echoes some of the concepts in Mr. Smith's article.

Social media: narcissism, ADHA, stalking

2009-07-19T11:35:00.003+02:00

This is a pretty accurate depiction of the what goes on with the users of social media. It makes a good-looking t-shirt!

Personalisation vs. Social

2009-07-17T13:55:00.009+02:00

I've been thinking of this personalisation-thing a lot lately. I quite cannot get my head around it, so this blog post is just to mull over the ideas.

By personalisation it is meant that, for example on a learning resource portal, the offer is tailor made for one of the users of the system. There are three commonly known ways of doing this:

Based on self-proclaimed profile (e.g. you say you teach math, so your services are personlised towards math)
Based on collaborative filtering e.g. ratings (like-minded users, ppl who agree in the past tend to agree in the future) or content-based filtering
Based on behaviour (e.g. other teachers who used this resource, also used xx)

In the cases 1 and 2, the idea is that there is a profile for you, whereas no: 3 can be used for any user (e.g. this is what Amazon does for any user regardless if they are logged in or not). With the first two cases, we can state that this type of personalisation is often cumbersome and labour-intensive, the problem is how to get that information from users (which creates cold start problem for users, and is also related to cold start problem of resources).

Also, not all the users are always interested in doing the work of filling in a profile or rating resources. In one of my studies (Vuorikari, Sillaots, Panzavolta, Koper, 2009) we found very different types of users behaviour, in this case related to how ppl tag (which could be also used for collaborative filtering).

About 33% of ppl tagged content (the arrows going away from the user group in the image), 32% used tags for searching but did not tag themselves (the arrows going towards a user), whereas 35% of ppl did not tag nor used tags at all (in the image the arrow going from LOM towards the group indicating that they used LOM based search methods only).

In this case, I'm interested in the 32% group, who clearly got benefit from tagging that other folks did, but did not do any work themselves (kinda freeriders, if you wish, but I don't mean to be negative).

If we were to use the tagging and bookmarking information to construct a user profile of the users and further use it for personalisation (no: 1 and 2 in the list above), we would be only able to do it for the group of taggers, i.e. 33%. The benefits could only be reaped by that group too, since for the rest of them, we have no profiling information to be used for personalisation.

However, a social tagging system is more about creating "personalisation" for all the users of the system, regardless if we know anything about them and they are willing to put in the time to create a profile and feed it in. It's about making social navigation trails visible to every user of the system, instead of going for "personalisation".

What I think is really cool is that we've shown that we more than doubled the amount of ppl who took advantage of contributions, i.e. from 27% who tag and use tags for navigation to 59% (that is adding the group of 32% who use tags but don't tag).

My assumption is that the rest would not even care about recommendations, etc., as they seem to formulate their searches in a rather acknowledgble way (40% formulate advanced searches; 38% only browse categories and 30% do both).

---------------------------------------------------
Some other thoughts about personalisation:

There are things that I dig, like Amazon, when it tells me "ppl who bought this book also bought xx". Thing thing is, though, that is the stuff that generally makes any user's life better on Amazon. It is not that they personalise the thing for me only, the unique Riina, the one and only, but it is something that makes any users experience on Amazon better.
the problem with personalisation often is that there needs to be a detailed profile of you that is based on detailed user model that is based on some abstract model that some obscure committee came out with in the 70's. Ok, that's maybe a bit exaggerated, but you get the point - there is a model where you are fitted.
What if I don't want to be personalised? What if I want to do the same thing as my buddies do; listen to same music as they do, study the same stuff as they do and go shopping with them? I want to share my life and experiences with other people around me because, guess what, through that type of sharing and doing stuff together, I feel related to them, I have things to talk with them and we form a community together. And it is really important for me to be part of that community, because it's part of who I am and helps me to reflect on what's out there.
Is peronalisation really personalised? It's not actually. By making things personalised to me, what is actually happening is "un-personlisation" of me. My taste is guided to the direction of all the other users, so I am actually being socialised! My personal music recommendations are actually very similar to other listeners, and eventually, it's all going to be the same taste!! Of course, unless there is randomness which offers serendipity.
Lately with all these micro-messaging things where ppl post their "mood" or what they are doing online ( e.g. I should be tweeting right now: "I'm writing a blog post" and simultaneously have it up on my Facebook), it's kinda funny that they feel this urge to yell out to all what they are doing in their über-personlised world.

Palm Pre - I wonder when it might come to Europe?

2009-07-15T21:25:00.005+02:00

Already for a long time I've been ready to say goodbye for Nokia. This is a big statement from someone from Finland whose grown up with Nokia. Mind you, when you kid/growing up you wear Nokia rubber boots. Your bike tires are Nokia. I bet at one point there was also toilet paper that they produced, among other things like cable and TVs. And then - the first kännys, the mobile phones in early 90' (Btw, they were not called gsms back in the day, as we used anotehr standard called NMT with much better coverage in vaste areas like in Lapland).

So, my whole active mobile phone user time (which in my case started already in early 80's, dad had a mobile phone in the car which we used to tell mom to turn on sauna when returning from summer house. You actually had to go by a dispatcher and push to talk!) I've stayed fidel to Nokia, and mostly Communicator, which I have had since '03 or something (when they came up with the 2 model).

Anyway, everything turned sour with the last Communicator that I had (E something). They've cut down the awsome shortcuts that used to be there and everything became too heavey and hard to use. Also, the size did not get any smaller. But I did not want to buy an iPhone either, I think no cool kids use iPhone. All the other phones are infested with all Microsoft this and that from which I prefer to shun away. So when I heard about Pre Palm, I took a look at it with no presumptions of the past (I used to hate palms and despite ppl who choose to use such an inferior technology - go and figure)

I so want it, this review is really good. I wonder if I have to wait until they come to Europe or can I buy it from the US? Well, actually just checked on it, by Christmas - geee, that's some waiting.

Google parsing microformat, e.g. ratings

2009-07-15T20:26:00.010+02:00

In May Google announced that they will start parsing microformats (on a small scale first), similar stuff came out from Yahoo! last year, but even on a smaller scale.

This is pretty huge for the end-user generated ratings! I must say that I did not see it coming in this way, which makes it even more exiting :)

..Google is releasing support for parsing and display of microformat data in their search results. .. anyone who marks their pages up with the appropriate microformat data will be able to make their information understandable by Google. This technology would allow you to explicitly search, for example, for only printers that had an average customer review of 3 stars or higher.

Holy smokes! This is cool, can't wait to see when it will first pop up in my search :)

So, since a long time it's been problematic to get enough ratings on items, this is a known problem especially in the field of Recommender systems. They talk about "sparse data". An example, you want to make a recommendation on music, but the item x has 3 ratings, item y 2 ratings, etc. This is way too little to be used to create recommendations using the algorithms that are out there. Take another example, a camera shop, they let users rate their cameras, but they get very little reviews from users.

Now, however, there are other camera shops who are struggling with the same problem. Essentially, they all are selling the same camera brands, and they all have only a few ratings on it, and at the end, non of them can do much fun with this small amount of rather anecdotal information.

There has been talk about a unique identifier set by industry, for example, so that all camera sellers could use them and thus aggregate all the reviews and ratings together. Yep, you guessed it, there's maybe that one shop down the blog who does not want to use it. I think a couple of years back Yahoo! came up with a very compelling paper reiterating the idea and trying to muster up enough consensus among industry and other players. Not much happened - and then, here is Google and microformats... beautiful :)

Why I'm interested in this is that with the idea of federating learning resource metadata across repositories, we face the same problem. As a result of sharing metadata, the same resource might end up used in many different repositories, where users might be allowed to rate them. But that metadata on ratings or evaluations is VERY seldom shipped back to the mother board.

The same with tags and bookmarking (other other tools that allow users to create collections or playlists). That could be valuable information for the repository who first federated the resource metadata out. By collecting back the varied annotations from different repositories, they could gain interesting information, and eventually overpass the sparse data problem. Moreover, they would gain data about what works and in which context, which makes me think of "travel well" resources.

In action

Here is an example of a search for Palm's new phone that I'm contemplating on. I search for reviews only and the result list shows the ratings, but I cannot yet make a query saying "palm pre" ratings grater than 3. Nice in any case.

I've had a few ideas on this with some colleagues and I really look forward to seeing what Google comes up with that. And how are they going to solve the issue of different rating scales used, and multi-attribute ratings.

Vuorikari, R., Manouselis, N., & Duval, E. (2007). Metadata for social recommendations: storing, sharing and reusing evaluations of learning resources. In D. H. Goh & S. Foo (Eds.), Social Information Retrieval Systems: Emerging Technologies and Applications for Searching the Web Effectively (pp. 87-107). Hershey, PA: Idea Group Inc. Retrieved from http://elgg.ou.nl/rvu/files/20/144/SIR_vuorikari_manouselis_duval_web.pdf.

Manouselis, N., & Vuorikari, R. (2009). What if annotations were reusable: a preliminary discussion. In M. Spaniol (Ed.), Advances in Web-Based Learning - ICWL 2009, Lecture Notes in Computer Science (Vol. 5686, pp. 255–264). Berlin Heidelberg: Springer-Verlag.

Tags and self-organisation: a metadata ecology for learning resources in a multilingual context

2009-07-10T18:57:00.006+02:00

I think I finally came up with a title for my PhD. You know, the type of title that says it all. It's a bit long, but "correct and descriptive", like Matt said. So here it goes: Tags and self-organisation: a metadata ecology for learning resources in a multilingual context.

Here is a wordle, it was extracted from a paper that summaries the research. Looks pretty accurate :)

Study on contexts in tracking usage and attention metadata in multilingual Technology Enhanced Learning

2009-06-30T17:27:00.004+02:00

Just submitted the final version of the paper to a workshop on Exploitation of Usage and Attention Metadata (EUAM 09). Here is a one-pager about it and the link to the paper.

Study on contexts in tracking usage and attention metadata in multilingual Technology Enhanced Learning

“Context” is widely accepted to be important for correctly interpreting user input and for improving predictive and possibly also diagnostic models. But what is context, and how can it be measured? By measuring we mean to operationalise the construct and data gathering to provide values for the desired variables.

In this study, we consider the intersection of the areas of digital learning resource repositories, digital libraries and social tagging systems where users from a variety of countries use technology enhanced learning (TEL) offerings in a variety of languages. We consider usage and attention metadata as an example of the wider notion of context adapting the definition of context as “any information that can be used to characterise the situation of entities” [Dey01]. We give an overview of dimensions of context that are relevant in TEL, specifically arguing that context comprises the usage situation and environment as well as persistent and transient properties of the user. Therefore, distinguishing between the macro-context and the micro-context of TEL is useful.

TEL and the analysis of the data it generates take place in different types of educational settings which we call the macro-context of TEL. We use the term micro-context to denote the context that is relevant for interpreting a specific user input and for designing adequate system responses and other output. The micro-context is subdivided into user models, material/environment models, interaction models, and background knowledge, showing that usage and attention metadata are of different types and play different roles for learning about context.

We then concentrate on teachers using learning-resource repositories as an important use-case example of TEL and focus on language and country as context variables. We describe different ways in which these variables are operationalised, and we outline ways in which TEL use such context information to improve the use and reuse of repositories by supporting users in a multilingual and multicultural context. A key theme of our article is the central role that social tagging can play in this process: on the one hand, tags describe usage, attention, and other aspects of context, on the other, they can help to exploit context data towards making repositories more useful, and thus enhance the reuse.

Riina Vuorikari 1,2, Bettina Berendt3
1 European Schoolnet, Brussels, Belgium,
2 OUNL, Heerlen, Netherlands,
3 KU Leuven, Belgium

My tag paper nominated for best paper award 2009

2009-06-25T22:54:00.004+02:00

I'm pretty exited that one of my papers for ICWL 09 was among the 5 best paper nominees. For a some time now I've been wondering what does it take to write a paper that arises above the general mass of papers. Well, now I have a bit better idea :)

What does it take? Reading tons of research papers, write a few (un)successful ones to practice, a good inspiring topic, some research work with ppl who are truly interested in what they are doing, and voila!

I also like how Celstec, OUNL (where I study), picked it up for their news feed. I think that over all, they have a pretty neat way to recognise what's going on and make others aware of it too. A modest person as I am, I would never make any fuss about it.... right.. ;)

Wiley calls it “dirty secret” of OER

2009-06-22T19:00:00.007+02:00

Just picked up a fresh PhD study by S. M. Duncan from USU, a student of D.Wiley's. The study is called Patterns of Learning Object Reuse in the Connexions Repository. The punch line is that there is very little reuse of LOs among the repository studied.

What new? Similar findings have been discovered here in Europe (end elsewhere) for a while now. Ochoa (2008), for example, found in his PhD dissertation that reuse in general remains low, about 20%, across all sizes of collections. This was interesting not only for how low the reuse is (20%, common!), but also because since forever folks have been saying that resources with smaller granularity are more reusable, as they lack context, etc (insert here the infamous graph of "modular content hierarchy", the most used LO). Well, according to Ochoa (2008), this was not the case.

I also looked at the reuse on 2 different platforms: LeMill and Calibrate from European Schoolnet. My twist was to study the cross-boundary use and reuse, i.e. teachers reusing learning resources that are in a language other than their mother tongue and originate from different countries than they do. I used the same reuse definition as Ochoa (2008), which basically is the same as in Duncan's study.

The finding was that the general reuse was around 20%, but NOT across all collections. For example, in LeMill, "Multimedia material" was used more often, but in Calibrate, the smaller granularity was seldom added to Collections. The cross-boundary reuse was notably less (37% to 55% of it). Moreover, in some of the collections only around 10% of resources were ever added to a collections, which makes you really think hard about the efficiency of this all..

Anyway, the good news in Duncan's study is this:

There was a common author in 3,722 module uses, while there were only 1,013 module uses where there was no common author. This means that modules were included in collections 3.67 times more often when there was at least one person in common with both the module and the collection.p.32

So if people know each other, they are more likely to reuse material from each other! This shows that social is important when we are talking about the use and reuse of learning resources! This is similar to what I am saying in my PhD thesis, which hopefully will come out one day soon. My twist of course is that tags can make those social connections between people, and by taking advantage of these underlying social connections, we can make the learning resource discovery much better - and hopefully also more useful for teachers.

Vuorikari, R., Koper, R. Evidence of cross-boundary use and reuse of digital educational resources. Link to a revised version of the paper, not reviewed yet!

Testing LeMill for embedding content

2009-06-15T18:33:00.004+02:00

LeMill is one of my favourite tools to create online material. It's so simple and easy to use. What I like a lot is that they always follow their time, like here I'm testing how the embedding of my content happens in another platform.

Here is one "Collection" that I've created called "hansin kamaa" = stuff from Hans. It includes two different pieces of content. Apart from exporting the content as a zip-file, I can now also just embed it somewhere, for example in my blog. Pretty neat - and useful!

Testing something else here:

ICT Call 5 info days: European Schoolnet

2009-05-11T00:08:00.006+02:00

I'm attending Call 5 infodays tomorrow for European Schoolnet. Here are a few things that we've been working with lately that could be relevant. The speakers look interesting, check them here.

For Large data sets:

eTwinning schools: more than 60 000 teachers have signed up. Stats available here. Now with eTwinning 2.0, new data will be available for new types of "connections" and "links" that teachers have.

This work is an example: eTwinning Network visualisation tool was created in collaboration with RWTH

Social Bookmarking data by teachers on learning resources residing on a number of different learning resource repositories in Europe. Some ManyEyes visualisation available to see different types of "connections" or "links" created.

Personal sphere

Would be interesting to study how is the personal sphere of a teachers in these days!

Some related slideshows:

Open Educational resources and tags creating connections

Networking learning communities: Open Educational Resources Start from page 16 onwards.
Social networks and networking to support eTwinning teachers

SIRTEL'09: 3rd Workshop on Social Information Retrieval for Technology-Enhanced Learning

2009-05-06T12:03:00.006+02:00

Paper Submission by June 14, 2009
in the International Conference on Web-based Learning (ICWL) 2009
Aachen, Germany, August 21, 2009
http://celstec.org/sirtel

IMPORTANT DATES

Contribution Submission: June 14, 2009
Results Notification: July 13, 2009
Camera Ready Submission: July 31, 2009
Workshop date: August 21, 2009

CALL FOR WORKSHOP CONTRIBUTIONS

We are delighted to welcome exciting new contributions for the 3rd SIRTEL workshop
- Research papers
- System Demos
- Hands-On proposals
- Abstracts for "Pecha Kucha"

RATIONALE

Learning and teaching resource are available on the Web - both in terms of digital learning content and people resources (e.g. other learners, experts, tutors). They can be used to facilitate teaching and learning tasks. The remaining challenge is to develop, deploy and evaluate Social information retrieval (SIR) methods, techniques and systems that provide learners and teachers with guidance in potentially overwhelming variety of choices.

The aim of the SIRTEL’09 workshop is to look onward beyond recent achievements to discuss specific topics, emerging research issues, new trends and endeavors in SIR for TEL. The workshop will bring together researchers and practitioners to present, and more importantly, to discuss the current status of research in SIR and TEL and its implications for science and teaching.

The proceedings from the last years:
· SIRTEL'07 http://ceur-ws.org/Vol-307
· SIRTEL'08 http://ceur-ws.org/Vol-382

TOPICS OF INTEREST (but not limited to):

Recommender systems and collaborative filtering in educational settings
Defining the scope, purpose and objects of social information retrieval in TEL
Novel ways of generating input for recommenders (explicit and implicit methods)
Ranking of search results to support individualised learning needs
Integrating SIR services in existing educational platforms
Folksonomies, tagging and other collaboration-based information retrieval systems
Social navigation processes and metaphors for searching information related to teaching and learning
Social networks and interactions in learning communities to facilitate information sharing and retrieval
Approaches to TEL metadata reflecting social ties and collaborative experiences in the field of education
Pedagogic decisions, recommender systems and how to contextualise recommender system to support learning processes.
Interoperability of SIR systems for TEL
Visualisation techniques in learning and teaching
Semantic annotation and tagging for social information retrieval purposes
Evaluating the performance of SIR systems in educational applications
Measuring the effectiveness of SIR systems in supporting learning and teaching
Evaluation the user satisfaction with SIR systems in supporting learning and teaching

WORKSHOP SUBMISSIONS

The workshop invites several types of contributions which allow a wide level of participation:
· Research papers (upto 8 pages)
· System Demos (upto 2 pages)
· Hands-On proposals (1-pager)
· Abstract for Pecha Kucha (1-pager)

The workshop proceedings will be published as CEUR Workshop Proceedings online at http://ftp.informatik.rwth-aachen.de/Publications/CEUR-WS/. Copy rights will be reserved.

Please use the same template as the one for the main conference with details at http://www.hkws.org/events/icwl2009/submission.html. Workshop paper length is not limited.

All questions and submissions should be sent to: sirtelworkshop@gmail.com

PROGRAM COMMITTEE

Alexander Felfernig, Graz University of Technology, Austria
Brandon Muramatsu, MIT, USA
Frans van Assche, European Schoolnet, Belgium
John Dron, Athabasca University, Canada
Lloyd Rutledge, OUNL, The Netherlands
Markus Strohmaier, Graz University of Technology, Austria
Markus Weimer, Technical University of Darmstadt, Germany
Martin Wolpers, Fraonhofer-Institut, Germany
Miguel-Angel Sicilia, University of Alcala, Spain
Olga Santos, UNED, Spain
Rick D. Hangartner, MyStrands,USA
Rosta Farzan, University of Pittsburgh, USA
Styliani Kleanthous, University of Leeds, UK
Tiffany Tang, Hong Kong Polytechnic University, China
Wolfgang Reinhardt, University of Paderborn, Germany
Xavier Ochoa, Escuela Superior Politecnica del Litoral, Ecuador
Yiwei Cao, RWTH Aachen University, Germany
Zinayida Petrushyna, RWTH Aachen University, Germany

ORGANISERS

* Riina Vuorikari, European Schoolnet (EUN), Belgium and CELSTEC, OUNL, Netherland
* Hendrik Drachsler, CELSTEC, OUNL,The Netherlands
* Nikos Manouselis, Greek Research & Technology Network
* Rob Koper, CELSTEC, OUNL, The Netherlands

ABOUT ICWL 2009

ICWL is an annual international conference on web-based learning. Since the first ICWL was held in Hong Kong in 2002, it has been held in Australia (2003), China (2004), Hong Kong (2005), Malaysia (2006), United Kingdom (2007), and China (2008). The 8th ICWL 2009 will be held in Aachen, Germany, a city with rich culture, high-tech research, and a truly European spirit. ICWL 2009 will be jointly organized by Hong Kong Web Society, RWTH Aachen University, and Max-Planck-Institute for Computer Science.
TAG THIS

Feel free to blog about this and social bookmark the call! Use the tag "sirtel09".
*************************************************************

Challenges and lessons learned from Social tagging in MELT

2009-05-05T11:03:00.006+02:00

Social tagging in MELT, how do we want to take the social tagging work forward

Social tagging of educational resources potentially offers new ways for:

Individuals to
1.1) better manage their digital learning resources that reside in different repositories and platforms, and

1.2) discover and access new resources from different contexts (e.g. different language, educational system) through tags and other users.

LOR managers to
2.1) get third party metadata on learning resources (either the ones that already reside on their repository, or the possible new ones to be added to collections,

2.2) create affinities (e.g. link structure) between separate pieces of resources (either on their own repository, or the ones that reside on other repositories on the federation or on the Web) that were not cross-referenced before.

In the MELT project so far, we have only been able to see the peak of these potentials emerging. We list issues that we see important for future work in the field, for the clarity, we only list one of the main issues for each topic:

1.1 To fully support users in their knowledge management task on digital learning resources, the bookmarks (including title, url and tags) should be exportable in standard Webfeed formats. This would allow users to access and manage their MELT resources as part of their other resources collections, whereas now users need to be logged on to the MELT portal to do this.
1.2 Pivotal browsing of social bookmarks takes advantage of the affinities between the user, resource and tags. In the MELT context, more metadata could also be added to support pivotal browsing, such as the country of the user, interest topics; resource metadata such as multilingual indexing keywords. This would allow novel ways to access resources that other users have already discovered within the federation, and thus build on users’ social interactions and co-construction of knowledge.
2.1 Tags by end-users on the MELT portal have been shown to be of good quality as additional metadata descriptors of resources. We have enumerated possibilities of metadata ecology that the use of multilingual Thesaurus can offer to a federation such as LRE. Apart from working on ways to automatically generate LOM from tags, we urge on using the hierarchical structure and multilingual features to leverage user-generated tags.
2.2 Why not do Google for learning resources? Using PageRank-like algorithms on a learning resource repository or federation has been impossible for a number of reasons, the most important is the lack of a link-structure that cross-references resources. Tags, creating underlying connections between seemingly random pieces of content in different languages, on repositories in different countries and other platforms on the Web, rely on humans’ subjective idea of its importance for a given information seeking task. Using this new, emerging link-structure with tags as “anchor texts” offers totally new ways to “organise the world's learning resources and make them universally accessible and useful”. A new tag line could be “From teachers to teachers”.

Link structure and anchor text

2009-05-04T22:44:00.003+02:00

I read that Brin & Page (1998) paper again. A few guidelines to keep in mind:

..our notion of "relevant" to only include the very best documents since there may be tens of thousands of slightly relevant documents. This very high precision is important even at the expense of recall (the total number of relevant documents the system is able to return).

Two features to produce high quality precision:

Link structure is used to create objective measure of its citation importance that corresponds well with people’s subjective idea of importance. Well, it's that simple..
Anchor text:
..anchors often provide more accurate descriptions of web pages than the pages themselves. Second, anchors may exist for documents which cannot be indexed by a text-based search engine, such as images, programs,..

The point about the anchor text is so interesting, I wonder how well does it apply to tags? I bet really well..

I also found this interesting: "it has location information for all hits and so it makes extensive use of proximity in search"

Differences Between the Web and Well Controlled Collections

extreme variation internal to the documents: documents differ internally in their language (both human and programming), vocabulary (email addresses, links, zip codes, phone numbers, product numbers), type or format (text, HTML, PDF, images, sounds), and may even be machine generated (log files or out putfrom a database).
external meta information as information that can be inferred about a document, but is not contained within it. Examples of external meta information include things like reputation of the source, update frequency, quality, popularity or usage, and citations. Not only are the possible sources of external meta information varied, but the things that are being measuredvary many orders of magnitude as well.

http://www.scribd.com/doc/3208417/The-Anatomy-of-a-LargeScale-Hypertextual-Web-Search-Engine

Cross-language use of the Web; users behaviours and attitudes

2009-05-02T16:13:00.010+02:00

Berendt & Kralisch (2009) A user-centric approach to identifying best deployment strategies for language tools: the impact of content and access language on Web user behaviour and attitudes

The results indicate that non-English languages are under-represented on the Web and that this is partly due to content-creation, link-setting and link-following behavoiur. User satisfaction is influenced both by the cognitive effort of searching and the availability of alternative information in that language.

Cost=time+cognitive effort

Not only capacities to access the site but also opportunities to access it, thus language is only one factor.

Language can be expected to not only influence the total amount of information available to Web users, but also how information sources (i.e. Websites) are linked among each other and therefore how easy/likely it is to find and access a certain Web site.

Bharat et al. and Halavis are first indicators of the potential impact of language: Website in different languages are less connected than sites in the same language (note: studied data aggregated on the national level and therefore only limited insight into the role of language.

"Websites are, in most cases more likely to link to another sitehosted in the same country than to cross national borders. Whenthey do cross national borders, they are more likely to leadto pages hosted in the United States than to pages anywhereelse in the world." (Halavais, A, 2000, p. 7)

Behavioural aspects of information seeking:

Users' information seeking behaviour,
information and information flow on the Web,

Attitudinal aspects of information seeking:

"usefulness", i.e. the language related value of information decreases as more information is offered in that language on the Web. "..value perceptions are also determined by topic; thus a large amount of content on a topic in a native language may also reduce the value of content on that topic in other languages.
"ease of use", i.e. the cost of language processing during information seeking can be expected to affect attitudes in Web search.

Results on behavioural aspects

Non-English languages are under-represented on the Web in terms of the amount of content supplied.
Search engines do not register all pages linking to the site, and many links known to the search engine were not used. This indicates that non-English language s are under-represented on the Web in terms of the links that content creators set to content in those languages.
Users have a clear preference to navigate in their native language when it is available via a link, but if that is not available they accept the necessity to navigate in English.
This all means: behavioural tendencies both of content providers and of content users lead to mutually reinforcing under-representation of non-English languages. Compared to the respective market size or available options, there is less content in these languages, this content is linked to less and the links are followed less often.

Results on Attitudinal stuff:

A complex interplay of English language skills, the perceived saved effort of using native-language content, the perceived overall supply in that language on the Web, and satisfaction:
People who are proficient in English often prefer to navigate in English (even if offered content is their own language) and are more scrutinised of the quality of Web content. Do not care much about whether sites make efforts to provide them with content in their own languages.
People who are not so proficient in English do perceive the (real) scarcity of information in their native language and are highly appreciative of content in this language.

This means that content and search-tool designers should not draw simplistic conclusions based on behaviour alone, because this is not a reliable indicator of attitudes and preferences. In the absence of links and/or content in their native languages, users will acquiesce to English-language content. However, their preference will persist.

Berendt, B., & Kralisch, A. (2009). A user-centric approach to identifying best deployment strategies for language tools: the impact of content and access language on Web user behaviour and attitudes. Inf. Retr., 12(3), 380-399.

HALAVAIS, A. (2000). National Borders on the World Wide Web.New Media Society, 2 (1), 7-28. doi: 10.1177/14614440022225689.

Bharat, K., Chang, B., Henzinger, M. R., and Ruhl, M. 2001. Who Links to Whom: Mining Linkage between Web Sites. In Proceedings of the 2001 IEEE international Conference on Data Mining (November 29 - December 02, 2001). N. Cercone, T. Y. Lin, and X. Wu, Eds. ICDM. IEEE Computer Society, Washington, DC, 51-58.

A touch screen for schools for less than 50€

2009-04-02T16:58:00.003+02:00

I love the do-it-yourself attitude of some e-learning tech support guys! Marko Puusaar just showed to us here in the e-university conference what he hacked together based on http://johnnylee.net/projects/wii.

In the picture you can see the touch screen for less than 50€. It took him about 15min, and that is with all the explanations of parts included!!

The pieces needed are: a Wii remote control, which will work through blue-tooth with your computer, a bit of software (there are pay versions, or free ones for educational use), and a Infra-red pen, which he did himself to look like a normal pen.

The Ada Lovelace pledge "Ms. Mayer"

2009-03-24T20:13:00.007+01:00

Some time ago I pledged to this one: "I will publish a blog post on Tuesday 24th March about a woman in technology whom I admire but only if 1,000 other people will do the same." I do to honor Ada Lovelace.

So here I go: since the first moment I set my eyes on it, I thought there was something that set it apart from the crowd. It must have been sometimes in 2000 or so. The name was catchy too, but what I most admired was the plain, simplistic look, almost too little, and yet, everything was there. Ever since I've admired the almost iconic look of it.

A couple of weeks back when visiting D&D in San Fransisco, we were drinking coffee and reading the Sunday edition of New York Times, I got across an article about Google's design. I learned that "Ms. Mayer controls the look, feel and functionality of the Internet’s most heavily trafficked search engine."

Of course, I thought, that is why the interface looks so DAM GOOD, it's a she!

The article got a few "Hyvä Suomi!" when I learned that since a kid she had admired the design of Marimekko, something that every Finnish kid from the seventies has imprinted in their brain. So, she's got Finnish ancestors too! "Hyvä Suomi!"

Apart from being the employee no: 20 and Google's first female engineer, she seems to like a good party and is comfortable on skis. Petty darn impressive! Thanks for being there!

http://en.wikipedia.org/wiki/Ada_Lovelace
an interesting interview on Marissa Mayer (where she wears an awful shirt, oups...)
http://www.charlierose.com/view/clip/10136

Sneak preview: SIRTEL'09

2009-03-23T14:24:00.007+01:00

Workshop on Social Information Retrieval for Technology-Enhanced Learning (SIRTEL'08) in the International Conference on Web-based Learning (ICWL) 2009 in Aachen, Germany, August 21, 2009
http://www.hkws.org/events/icwl2009/workshops.html

IMPORTANT DATES

Contribution Submission: June 14, 2009
Results Notification: July 13, 2009
Camera Ready Submission: July 31, 2009
Workshop date: August 21, 2009

CALL FOR WORKSHOP CONTRIBUTIONS

We are delighted to welcome exciting new contributions for the 3rd SIRTEL workshop

Research papers
System Demos
Hands-On proposals
Abstracts for "Pecha Kucha"

RATIONALE

Learning and teaching resource are available on the Web - both in terms of digital learning content and people resources (e.g. other learners, experts, tutors). They can be used to facilitate teaching and learning tasks. The remaining challenge is to develop, deploy and evaluate Social information retrieval (SIR) methods, techniques and systems that provide learners and teachers with guidance in potentially overwhelming variety of choices.

The aim of the SIRTEL’09 workshop is to look onward beyond recent achievements to discuss specific topics, emerging research issues, new trends and endeavors in SIR for TEL. The workshop will bring together researchers and practitioners to present, and more importantly, to discuss the current status of research in SIR and TEL and its implications for science and teaching.

The proceedings from the last years:
- SIRTEL'07 http://ceur-ws.org/Vol-307
- SIRTEL'08 http://ceur-ws.org/Vol-382

TOPICS OF INTEREST (but not limited to):

Recommender systems and collaborative filtering in educational settings
Defining the scope, purpose and objects of social information retrieval in TEL
Novel ways of generating input for recommenders (explicit and implicit methods)
Ranking of search results to support individualised learning needs
Integrating SIR services in existing educational platforms
Folksonomies, tagging and other collaboration-based information retrieval systems
Social navigation processes and metaphors for searching information related to teaching and learning
Social networks and interactions in learning communities to facilitate information sharing and retrieval
Approaches to TEL metadata reflecting social ties and collaborative experiences in the field of education
Pedagogic decisions, recommender systems and how to contextualise recommender system to support learning processes.
Interoperability of SIR systems for TEL
Visualisation techniques in learning and teaching
Semantic annotation and tagging for social information retrieval purposes
Evaluating the performance of SIR systems in educational applications
Measuring the effectiveness of SIR systems in supporting learning and teaching
Evaluation the user satisfaction with SIR systems in supporting learning and teaching

WORKSHOP SUBMISSIONS

The workshop invites several types of contributions which allow a wide level of participation:

Research papers (upto 8 pages)
System Demos (upto 2 pages)
Hands-On proposals (1-pager)
Abstract for Pecha Kucha (1-pager)

The workshop proceedings will be published as CEUR Workshop Proceedings online at http://ftp.informatik.rwth-aachen.de/Publications/CEUR-WS/. Copy rights will be reserved.

Please use the same template as the one for the main conference with details at http://www.hkws.org/events/icwl2009/submission.html. Workshop paper length is not limited.

All questions and submissions should be sent to: sirtelworkshop@gmail.com
The website with the call will be up shortly too.

PROGRAM COMMITTEE:
• Alexander Felfernig, University of Klagenfurt, Germany
• Brandon Muramatsu, MIT, USA
• Frans van Assche, European Schoolnet (EUN), Belgium
• John Dron, Athabasca University, Canada
• Lloyd Rutledge, Open University of the Netherlands, NL
• Markus Strohmaier, Technical University of Graz, Austria
• Markus Weimer, Technische Universität Darmstadt, Germany
• Martin Wolpers, Fraonhofer-Institut, Germany
• Miguel-Angel Sicilia, University of Alcala, Spain
• Olga Santos, UNED, Spain
• Rick D. Hangartner, MyStrands, USA
• Rosta Farzan, University of Pittsburgh, USA
• Wolfgang Reinhardt, Universität Paderborn, Germany
• Xavier Ochoa, Escuela Superior Politécnica del Litoral, Ecuador
• Yiwei Cao, RWTH Aachen University, Germany
• Zinayida Petrushyna, RWTH Aachen University, Germany

OER Creating connections: content, users, tags

2009-03-04T22:31:00.004+01:00

I'm in the Hewlett grantees meeting now and marveling all the work that has been done in the area of Open Educational Resources. I had a chance to present some of the work here that we are doing with OER. I focused on creating connections, which I think is one of the most important things for the content. Knowing how it all is connected together is important in order to make sense of all the small pieces of separate content. Here are the slides, I give a few explanations.

Learning Resource Exchange -OER Hewlett Mongerey

View more presentations from Riina Vuorikari. (tags: lre eun)

The LRE is an access point to 16 content providers who make resources available to teachers in Europe and elsewhere. It's pretty much like any conventional content portal, but we have added a social tagging and bookmarking tool on top of it.

A good part of the slides show how we can make connections between pieces of content from different repositories, and how those content pieces can bring users togeher across country and language borders. In the visualisations, the little dots (nodes) are resources that are connected to other resources or users through tags and bookmarks.

Moreover, I give a few pointers to research that we have done on creating better navigation to the content, evaluating whether it is efficient or not, and also looking into the quality of tags.

Are tags from Mars and descriptors from Venus?

2009-02-27T13:15:00.006+01:00

A study on the ecology of educational resource metadata.

I just finished a paper on the tag evaluations that we did in the MELT project. We had lots of fun with the name of the paper :) the main question being which one, tags or descriptors, should be from Venus...?

Anyway, we were able to show that not all the tags are as far from the Thesaurus descriptors as Mars is from Venus. We had different perspectives for evaluations: end-users, expert indexers and repository owners. For me the most interesting thing that came up was that 11% of end-user generated tags are actually terms that we can find in our multilingual Thesaurus! I assume teachers are "better taggers" than average, usually there is lots of talk about the gap between end-users' language and the one deployed by experts.

Abstract. pdf. In this study, over a period of six months, we gathered empirical data from more than 200 users on a learning resource portal with a social bookmarking and tagging feature. Our aim was to look at the tags from different stakeholders’ points of view; end-users, librarians/expert indexers and repository owners. We first look how users tag resources, and then conduct an evaluation with indexers to understand how they perceive the value of tags as descriptors. We then present a case study from a repository owner’s point of view. Lastly, we study users’ clickstream when searching resources. We find that, even though end-users and expert evaluators apply very different strategies when adding metadata, (end-users have a rather synthetic approach whereas expert indexers an analytical one) there is an overlap in the information in tags and the official descriptors, this overlap is even up to 51%, creating an ecology of metadata.

Keywords: Learning resource metadata, tags, folksonomy, clickstream,
thesaurus, evaluation.

"Thesaurus-tags"

2009-02-09T22:12:00.004+01:00

One of the particularities of the MELT portal is that apart from being a "traditional" resource portal, we also have social tagging-features implemented. This creates a situation where resources have both indexing terms that come from our multilingual Thesaurus, as well as teacher generated tags, that, btw, are also multilingual.

I was looking at the tags today with a specific question in mind: "How many of the user-generated tags are actually terms that exist in the Thesaurus?" If there is a tag that is added by a teacher, and if it exists in the Thesaurus, I will call it a "Thesaurus-tag".

Here are the figures:

Distinct tags: 4428
Tags applied: 5009
Distinct "Thesaurus-tags": 505
"Thesaurus-tags" applied: 714

I was actually really impressed: 11.4% of distinct tags are "Thesaurus-tags"! And if we look at tag application, "Thesaurus-tags" amount to 14.25% of all tags.

Moreover, 22.37% of distinct "Thesaurus-tags" are applied more than once. This amounts to 45.1% of all "Thesaurus-tag" applications! The top "Thesaurus-tag" were:
Europe (10), music (8), test (8), Vocabulary (8), Internet (7), art (6), biology (6), history (6), Australia (5), chemistry (5).

This is really quite interesting. On the one hand, we always ask ourselves how to make the Learning resource indexing better, and here we can totally "crowd source" part of the indexing, that is usually done by the experts", to end-users. If we think an end-user thought that a Thesaurus-tag was good enough to add for a resource, I can be pretty sure that it is also good enough for being an indexing term. The story can be VERY different for all tags, we do not think that ALL tags could become indexing terms, although we know they are good for other stuff.

On the other hand, being in the multi-lingual context, we have always a bit hard time with tags in different languages. At least with "Thesaurus-tags" we could easily show a translation of the tag, as we are certain it to be a good one.

There are lots of interesting things to see, for example, whether these resources previously had the same indexing term as the "Thesaurus-tag" was, i.e. was the tag redundant or does it really add some value to our system. It will also be interesting to see whether there was a trend; the resources that had poor/little indexing terms received more "Thesaurus-tags" from the end-users. Well, lots more, I guess, but that will be for another time.

Modelling the portal ecology: What goes around comes around

2009-01-13T13:12:00.009+01:00

The three main actions on the portal: discover resources, play them and annotate. The two main group of users: ones logged-in and the others not.

I've divided the resource discovery process in three slots (Millen et al.):

Explicit search
Community search
Personal search

Play is when the user clicks on the link. We also call this implicit interest indicator, however, we are not sure whether it was relevant to the user or not. Worth noting anyway. This is also called "hits" or "click-through" in some lingo.

Annotation is when the user makes an explicit interest marking (indicator) on the resource, this can currently be either a rating (usefulness, scale 1 to 5) and bookmark with tags. Both of these actions are public.

About users and logs

In general terms we record all kinds of clicks and actions on the portal (see here). I studied the logs from the last 2,5 months. We know that we have 340 users who have a user name, excluding staff, etc., we have 168 "real" users. Out of them 82 had clicked on a resource on the portal at least once, so these users are included in the logs. Additionally, there are users who do not log, but I do not have any idea currently how many they are (check Analytics). There were 13 604 actions recorded, 40% from the ones who logged in and 60% by others who did not log in.

In general, we can think that the relationship between these 3 actions is important on the portal and can indicate something about its efficiency for users to get what they want, as well as for the system to get what is needed to keep it going. In our case, we are in the process of looking at how Social information can help the discovery. So, a perquisite is to have SI available, thus the system needs ratings and bookmarks.

With the contributing users (=logged-in) on the MELT portal:

2 searches result to one play;
2.6 searches result to one annotation, this can be either rating or bookmarking;
1.3 plays result to one annotation.

For the comparison, in Calibrate the figures were the following:

0.5 searches result to one play;
5.7 searches result to one annotation, this can be either rating or bookmarking;
11.3 plays result to one annotation.

I will study this further too. A quick look would say that a system, which emphasises Social Information for users own benefits (Favourites) and for everyone's benefits (allows Community browsing) like is the case with MELT, the loop for getting annotations is more efficient than in the system which does not make use of such information (e.g. Calibrate). The ration of search-to-annotation is 2.6 to 1 in MELT, whereas the same in Calibrate is 5.7 to 1.

However, if we look at the ration of "hits", the Calibrate system has been four times more efficient: it took one search to play 2 resources, whereas in MELT it took 2 searches for one play. The MELT system search function has been under constant development for speed, which has been somewhat problematic due to huge amount of content. I will report later on the same ration after our last optimisation effort.

What goes around comes around

Graph 1 depicts what is going on on the portal. I will explain this later in details. For each action I have indicated the percentage of total, e.g. Explicit search 78% is from all Explicit searches executed by non-logged in users.

Graph 1

Discovering cross-boundary resources on the portal

2009-01-12T15:41:00.006+01:00

This is continuation for the previous post, the same data-

The question now is how and where do users discover cross-border resources? By this I mean that the user and the content come from different countries, and/or that the content is in a language other than the user’s mother tongue.

One challenge is discovering resources in general (previous post), another one is to discover resources that are in a different language and come from different countries. This is the case on our portal, so I'm interested in how to facilitate the discovery process that involves crossing those boundaries (language and national, which might have implications to the educational content of the resource), and hopefully make it more efficient.

My take is social information: making it readily available to all to leave cues from other users. I bet that this should facilitate the discovery process and thus also make it more efficient (more resources and faster).

So I looked at the previous data and the cross-boundary bookmarks from users. This is relative to the user, of course, so with every bookmark I compare whether the resource and the user come from the same country and/or is the users mother tongue different from the resource language.

41 out of 48 active users had bookmarked cross-boundary resources. They had added

299 distinct resources into their collections 350 times. Out of these resources
163 were cross-boundary resources, which had been added to their collections 198 times.
This means that 55% of distinct resources obtained during the period of 1.5 months were cross-boundary resources.

So it was interesting to see how many of these resources had Social information added to them?

Table 1

Table 1 shows that about a third of the bookmarked cross-boundary resources had Social Information on them, they were either bookmarked by previous users or existed in the "travel well" list. This is cool! Although we cannot say that the users only discovered these resources because of Social Information, it is important to know that it has had added value for users to discover them. I also found out that about 10% of bookmarks on resources with SI had been previously bookmarked by someone from the same country as the resource was. The fact that they were bookmarked, although still almost dismissible small (10%), is still good news for SI and social navigation based on it.

I then looked at where the resources are discovered: 62% of cross-boundary discoveries were done in Search Result List (SRL) whereas 37% took place in Community searches, most of them in the tag cloud (30%) and 7.5% on the Travel well and Most bookmarked lists (only one case in the latter :(

As comparison for the resource discoveries that did not cross any borders (e.g. German teacher found German resources), 90% of them took place in SRL. So it seems that for discovering cross-boundary resources the Social Information is important, as it allows users to do Community searches to discover these resources. We do see, though, that cross-boundary discovery is efficient also within the resources that cannot leverage the previous user experiences, as 63% of cross-boundary resources do not have any SI.

Interestingly, when we look at Table 1, we see that for the resource discovery that does not imply any cross-border action, users do not seem to care much for SI. Actually, more than 90% of these bookmarked resources had no SI. This is cool, as it seems that we need users to discover and annotate resources among their comfort zone (e.g. national and regional educational material in their own language) in order to make it more readily available to others.

Another thing that I've looked at are these measures for

usage coverage within the repository,
how many resources are shared among collections (e.g. Favourites) and
what I call the pick-up rate, this is how many of distinct used resources are reused. This could happen when someone discovers a resource that has SI related to it (e.g. in the travel well list or from other user's favourites, or just picks it up from SRL based on someone else's annotations). These were discussed in more details in the last paper.

Table 2

Table 2 shows these measure in LeMill, Calibrate and delicious, additionally, the gray column indicates this 1.5 month trial in MELT and the last column has all the data from MELT, which includes the pilot teachers, staff, partners, etc.

We can see that as the initial amount of resources is so big in MELT, we still cover only a minimal amount of resources, from 1 to 2 %. This does not even include the assets, which more than doubles the amount. Used here means that the resource is added to Favourites once (reuse more than once).

We can see, though, that even if the resources coverage is not that high, there is still quite a lot of sharing among used resources. This figure still remains low with 1.5month trial (about 15%, same as in Calibrate which did not make the SI available!), but if we look at all the usage so far, the sharing is at 43%. This is somewhat artificial, though, as there is a lot of staff use, but still I hope it indicates that making SI available on the long run helps sharing resources (or, I need to look better solutions on the portal for sharing, which is also planned but super delayed because of all the other dev programme).

We can already see that the pick-up rate is higher in MELT trail than in the 3 other platforms that I've looked previously. This is an indication that SI works, I hope.

Btw, I could not find any correlation between the act of putting the resources in Favourites and whether they have Social information related to them (I got some lousy 0.18 even if I removed 2 outliers who outperformed anyone else). Also, in some previous test I had hard time finding significant changes, so I might need to seddle with increased reuse rate, which has previously been shown to be abotu 20% across collections. I found that it was about half of this (or the same as general reuse) in my previous paper.

Where does a CC-licence take you?

2009-01-10T20:36:00.012+01:00

..or better, your photos? Some time ago one of my pics was "selected" for a travel guide from Prague. That was pretty cool and I really appreciated it. Tonight, I was checking my Flickr stats and noticed that one photo had been viewed many times recently. So I looked what's up with that.

I followed the referrals. Here are the top 3 sources, some other hits were through search engines for generic keywords like ski, etc.

1. It was the main picture on the Facebook group for Sankt-Anton fans.

2. It was on a Yahoo! travel website for best skiing in America. They had a Flickr badge with some random ski shots, et voila moi! (see the small image on the right corner)

3. The funniest of all, though, is that it is one some German photo website where the dude discussed how the picture should have been framed differently! Hmm..I think my boyfriend, who took the picture, did not appreciate this lesson..

It's just kinda weird to find yourself in odd places on the web, but hey, isn't that what the creative commons license is supposed to allow. So this is just one consequence of it, I guess.

Talking about odd places to be on the Web, the other night my Google alerts had picked up that my blog post appeared on a porno site. Sure enough, there after lots of photos of youknowwhat, were feeds from my latest blog post. Pretty hilarious... I don't recommend clicking on the link, but you can read the paper though!

Out Now: Special Issue on Social Information Retrieval for Technology Enhanced Learning

2009-01-07T17:42:00.006+01:00

I am glad to announce the Special Issue on Social Information Retrieval for Technology Enhanced Learning (SIRTEL) which just came out today in Journal of Digital Information (JoDI) Vol 10, No 2 (2009)!

I co-editored it with Erik Duval and Nikos Manouselis. The following stuff's in it, enjoy!

Special Issue on Social Information Retrieval for Technology Enhanced Learning	HTML
Erik Duval, Riina Vuorikari, Nikos Manouselis

Articles

Identifying the Goal, User model and Conditions of Recommender Systems for Formal and Informal Learning	Abstract PDF
Hendrik Drachsler, Hans G. K. Hummel, Rob Koper

The Pedagogical Value of Papers: a Collaborative-Filtering based Paper Recommender	Abstract PDF
Tiffany Y Tang, Gordon McCalla

Lost in social space: Information retrieval issues in Web 1.5	Abstract HTML
Jon Dron, Terry Anderson

Exploratory Analysis of the Main Characteristics of Tags and Tagging of Educational Resources in a Multi-lingual Context	Abstract HTML
Riina Vuorikari, Xavier Ochoa

Visualising Social Bookmarks	Abstract PDF
Joris Klerkx, Erik Duval

A Special thank to people who participated in the PC:

Alexander Felfernig, University of Klagenfurt, Germany
Brandon Muramatsu, Utah State University, USA
David Massar, European Schoolnet, Be
Hendrik Drachsler, Open University of the Netherlands, The Netherlands
Jon Dron, Athabasca University, Canada
Marc Spaniol, Max-Planck-Institute for Informatics, Germany
Martin Wolpers, Fraunhofer, Germany
Miguel-Angel Sicilia, University of Alcala, Spain
Nikos Manouselis, Greek Research & Technology Network, Greece
Rick D. Hangartner, MyStrands, USA
Salvador Sanchez, University of Alcala, Spain
Xavier Ochoa, Escuela Superior Politécnica del Litoral, Ecuador
Yiwei Cao, RWTH Aachen University, Germany

Users on the portal: consuming and contributing

2009-01-06T19:07:00.028+01:00

I continued the previous study with more data: this time I took all the logs from Oct 1 to December 18 2008. The idea, again, was to see how much the previous annotations (ratings and bookmarks) would "guide" the choice of new users.

The datasets:

1. All bookmarks and ratings in MELT until Oct 1 2008. This comprises of 565 distinct resources. We call these resources with Social Information (SI), as it is something that is shared and made public by users.

88 of these resources were on the list called "Travel well" resources. They are made available directly from the portal front page. They were recorded to the system by a user called "EUNRecommender" suggesting that these resources should be of interest and good quality. Additionally, annotations from other users were made available.
477 of these resources were annotated (ratings and bookmarks) by previous users. These were either the pilot teachers, project partners and staff in the office. The top bookmarked and rated once would appear on the "Most bookmarked list", also accessible from the front page. Similarly, annotations from other users were made available.

2. I took server-side logs from the period of Oct 1 to Dec 18 2008. At the end of this period the portal had 340 users registered, out of which some people never used the portal when logged in and a number was project related staff. We excluded those people from the logs and were left with 168 users, out of which 82 had clicked on a resource on the portal at least once. Additionally, users who do not log in are recorded, so our click-though includes many more users. This group had:

clicked ("played") on 1711 distinct resources (all users);
out of which 974 were clicked ("played") by logged-in users (82);
bookmarked 294 distinct resources 351 times;
rated 323 distinct resources 385 times;
= 394 distinct resources annotated.

The method for this study is a manual log-file analyses using my own defined logging scheme (see here).

Questions: How and where on the portal do users discover resources, do they rather discover new resources or re-discover once that have been annotated by others? Are the suggestion lists like "Travel well" resources or "Most boomarked" effective, i.e. are many resources discovered through them?

We split our understanding of "discover resources" meaning two different things and look at the separately. Discover meaning:

finding the resource on the portal, clicking on it (i.e. play). This is also called implicit interest indicator;
finding a resource and creating an explicit marker on the resource in terms of rating or bookmarking with tags (explicit interest indicator).

The first one is important to know how big coverage the resources that have been "touched upon" or "hit" cover from all resources in the repository (use). The second one, is important as it creates explicit maker which can be used for cues and social navigation purposes. Moreover, the second one can be used as a proxy for reuse. For reuse we use the following definition: " resource is integrated in a new context with other components, and when this occurred more than once, we consider the resource reused".

1. Who discovers resources?

Out of 82 active users, 63 had annotated a resource at least once, 75% had annotated more than once. Average was 11.6 annotations, median 4. The top two users annotated 120 and 108 times, the following users 50 times. There were 36 users who bookmarked and rated, and 14 users who only bookmarked and 13 who only rated.

2. What kind of "actions" take place?

By actions, in this case, I mean playing the resource (click on the link), bookmarking and/or rating it, but also viewing evaluations related to this resource or checking the "Favourites" from the user who had discovered an interesting resource.

Table 2 presents data on how the users (described in 2. above) interacted with resources. The left column explains where on the portal this action took place, whereas the top row indicates the action. There were total of 3542 actions related to this.

In general, this group annotated 75% resources in SRL, 15% in tag cloud, 4% on the Travel well list, 3% in their own favourites (ratings) and only 1% on Most bookmarked list.

Table 2

Let's first look at what happens in the Search Result List (SRL). Table 2 shows that most things happen when users are here: resources are played more than 2000 times (2/3 of all plays), most rating (70%) and bookmarking (73%) of "virgin" resources also take place here. We can also see that some small amount of resources with SI are rated and bookmarked from SRL (less than 10%). This indicates that some users paid attention to cues made available by previous SI and found them useful.

Mostly, though, it's "virgin" resources. This is a good news, in a way, as I was worried that users might be lazy and not discover resources that other users have not annotated before. On the other hand, most users click on resources and never annotate them, so we are left to guess whether they liked them or not... only 13% of these clicks lead to rating the resource (268), for example.On the average, 71% of virgin resources "hit" get annotated, and 29% of resources with SI get annotated.

Table 2 also shows that the tag cloud is a good catcher of ratings (15%) and taggings (15%), whereas the "Travel well" list is a real hit catcher. 13% of hits on resources with SI are played here. This list was comprised of 88 resources, out of which only 54 got played. The average was 6.8 plays/resource (median 3.5), but in reality, some got played a lot more than others. 30% received more than average hits. The highest were 30, 29 and 28 (great, just checked the top dog, and it's a dead link :(.

The story looks worse for actually bookmarking and rating resources from the Travel well list, only 18 of them got hitched (20%). 2 got bookmarked twise and two rated 5 times (no overlap). So, I suspect that we did not succeed that well in creating appealing "recommendations". I'll get back to that point at a later stage. A rather effective features of TW list was the use of "view evaluations" and "view other users", about 40% of both were generated here, so they helped users to social navigate the portal.

Compared to "Travel well" list, the "Most bookmarked" resources did not seem to have the same effect of chatching eyeballs. This is bizarre, as both these are available on the font page, however, "Most bookmarked" are a click away on the tab. As 84% of these hits come from users who are not logged in, I think it might be lots of clicks from our testing period, but also it's possible that this is only what users experience on the portal and then fly away (forever..?).

Lastly, table 2 shows that some ratings take place in Favourites. That's good, since we intended favourites to be the place for that. I imagined that teachers first want to use the resource, so they put it on thier favourites and then come back later to rate it. Well, users know better, they seem to take about a second to view the resource and bookmark it on the SRL. I will have to look if there is any qualitative difference between ratings of the ones in SRL and Favourites?

3. Where do these actions take place?

I was also interested in seeing what kind of search methods do users choose to use. Possibilities offered on the portal are the following:

Explicit search: using traditional search box with text or advanced search options. This results in resources on a Search Result List (SRL) where users can view the metadata about the resources as well as the annotations by others. Searches within results in SRL can also be refined, or ranked either by popularity or ratings. Also "Browse by category" results in this.
Community browsing: These include browsing the tag cloud, and examining bookmarks created by the community. In our case these are lists of most boomarked items; travel well; tags; other people Favourites.
Personal search: Looking for bookmarks from one's own personal collecction of bookmarks (Favourites)

Table 3.

Of all searches 76% were Explicit searches, where 21% Community searches and 2% personal searches.

What comes to spearing actions on other things on the portal, we can observe some differences between the logged-in and not logged-in users. Whereas not logged-in users spend most of their actions on searching (60% + 17%) and playing resources (23%), the logged in users have a more variety.

The logged-in users search differently, they use far less the Explicit search function (28%) and also spend less time on the Community search (only 8%), but additionally they have the Personal search function available (3,5%). The difference here is that these users can interact with the resources that they have found earlier ("keep found things found"). There is quite a huge difference between how much actions are spent on searching between these groups, 77% vs. 40%. Logged-in users also play less resources (18%), but still "outsmart" not logged-in users in terms of searches returning plays:

the first group spears 3.4 searches to play one resource, whereas
the logged-in users spear 1,98 searches to play one resource.

This apparent inefficiency is partly due to the fact that we are still testing the server and probably many tests are done when the user is not logged in. However, this is something to remark for now and keep the eye on (check: can I omit the searches from the office using the data from Analytics).

A major difference between the two groups is what I call contributing actions. We already have seen that 27% on resource with SI get played and that on the average 13.3% of searches take advantage of Community browsing. We also saw that some 6% of annotations on the SRL are on resources that have Social Information related to them. So where does this social information come from?

In Table 3 we see that contributing actions cover about 40% of all the actions by logged in users. This is about 16% of all actions during this period! I will come back to this point later trying to make a picture of what the input is by a group of logged-in users and how it can be taken advantage of by other users of the system.

4. What resources do get played?

Users can access 30116 learning resources through the portal, and many more assets. During the trial period, 1547 distinct resources got played 2828 times by all users. That make an average of 1.83 plays/resources, but in reality some get a few hits and a few gets many hits (median: 1). 27% get more than one hit, most hits were 35 ( strangely, this resource was not even bookmarked "123216875").

I was interested in what happened with resources that had Social Information vs. the ones that were "virgins", not yet annotated by previous users.

Of the total plays (2828 times) 73% were on resources without SI and 27% on resource with SI. If we look at it from all resources that were made available, out of 565 resources with SI 34,7% were played at least ones, whereas from all other resources (29551), 4.57% got played at least once. Also, some of the newly annotated resources got hits right away, we have 54 resources that got played 70 times, most likely thanks to their new annotations. Some of these newly annotated resources (32 cases) prompted 99 further annotations from the new users. These 99 annotations were made both in SRL (69%) and in social navigation areas like tagcloud and Favourites (31%). This shows that these new annotations became useful to other users right away, actually more so than the previsouly annotated resources, out of which only less than 10% were found of use by these users (31% vs. 10%, see Table 4).

It's quite interesting, though, that only about one third of resources with SI got played. I would have thought that it is more. This might be due that our search result list is not ranked to start with. It is possible for the user to re-organise the results by popularity or ratings, but actually we do not know whether this happens (currently have not found a way to log it). I guess the other side of the coin that surprises me is that users clicked on so many resources that did not have any annotations on them. I guess it's a good sign of curiosity :)

Interestingly, we find that users who are logged-in discovered less resources than the others. Out of all these resources (n=1547), 68% were played by users who were not logged in and 44% by logged-in users (last row in Table 1). The same can be observed for resources with SI and not.

Table 1.

5. How many annotations per resource?

Of the total amount of annotations received during this period, we can count that 394 distinct resources received 734 annotations, they were half and half ratings and bookmarkings.

322 resources received 383 ratings. 13% received more than one rating, average 1.18. Top amont of ratings were 5 ratings, which was only on 1 resources.
294 resources received 350 bookmarkings. 14% received more than one bookmark, average 1.19. Top amont of bookmarks were 5, which was only on 2 resources.

58.5% of annotations were on new resources, and 41.5% on old ones. They were distributed rather differently, the "old" resources received on the average more annotations than the new ones.

38 Travel well resources received annotations 99. 63 of them received more than one annotation, average is 2.6 each. The top one received 12 annotations.
97 previously annotated resources (this includes 32 resources that were discovered during this period and annotated later by other new users) received 209 annotations. Average is 2.15 annotations per resource.

6. Story of the Travel well list

Barely about 15% of the 88 resources on the list were re-discovered by the new users! Previously we saw that these resources received a good amount of "hits" and eyeballs, but not so many of them actually resulted in ratings, and when they did, they were not equally distributed among the resources (Table 4). Only 18 of the annotations took place on the TW-list, otherwise, an additional 20 resources from this list were annotated on the SRL and tagcloud. So the success-rate of these recommendations was less than 50%! Can barely call these recommendations. I will look later which ones were "thumbed up" and which ones downed. As with the previously bookmarked resources, ahem, maybe the taste differs between the two sets of users.

Table 4

The resources on the "Travel well" list received many more hits (average 7.12) than just "any previously annotated resource" (average: 0.68).

New users on the portal and resource discovery

2009-01-04T22:57:00.009+01:00

I looked at 18 new users on the portal, and studied resources that they bookmarked. I wondered how many of these resources had previous annotations by users? By annotations I mean that previous users had added ratings on them and Favourited these resources. If this is the case, it's clearly shown on the portal.

But are these annotations persuasive? Do they help users make their decision better or faster?

I had two sets of data:

last 3.5 months (Aug to Nov 17 2008)
Nov 18 to Dec 18. This are my 18 users who had bookmarked 114 resources.

Out of these results, it seems that 2/3 of the resources that these new users bookmarked had no previous annotations on them! I have to verify this finding, because currently I lack data from March to July to see what was bookmarked then.

Table 1

Anyhow, let's see what the current mini-study holds. Out of the third of resources that were discovered by this group, about 25% had previous annotations on them. They were mostly done by users before this group got initiated on the portal, however, some were also discovered thanks to the bookmarking by this group.

Only about 8% of resources were discovered through a special list called "Travel well" resources. These resources have been added there by "EUNRecommender" which currently is hand operated, but mainly based on picks by other users from at least two different countries. I find this figure rather surprising, as this "Travel well" list is the first thing that the user sees when they come to the portal.

Anyhow, I find it cool that 25% of bookmarks by this group were resources that had previous annotations. What we cannot say, though, is whether these users could have found these resources without these annotations displayed publicly on the search result list. However, I think that my measures like "pick-up rate" and "overlap" among Favourites will help me sort that out.

Here is a visualisation of this. The huge node is "sos-rec" which means that these resources were previously annotated (rate, bookmark). What is called "EUN Recommender" are resources from the "Travel well list". Other than that, the new users are the nodes which are connected by edges to resources that they have bookmarked.

Right from the bat, we can see that 5 users had taken their totally own trails and bookmarked resources that no one had bookmarked before - quite cool!

Additionally, there are few users on the lower right hand corner who are only connected to the whole graph through one resource (user: 192682) and (user: 217391).

Share early: Paper on Evidence of cross-boundary use and reuse of digital educational resources

2008-12-22T11:52:00.003+01:00

I finally sent off my paper to a journal. Exiting. The first comment was to cut it shorter by about 2500 words, even before they started reviewing it. Outch, I think I managed to do it, I have a copy of it here:

Vuorikari, R., Koper, R. (submitted). Evidence of cross-boundary use and reuse of digital educational resources. pdf

ABSTRACT: In this study we conducted an investigation on the server-end log-files of teachers’ Collections of educational resources in a number of content platforms. Our goal was to find empirical evidence from the field that teachers use and reuse learning resources that are in a language other than their mother tongue and originate from different countries than they do. We call these cross-boundary learning resources. We compared the cross-boundary reuse of educational resources to the general reuse figure of 20%, and find that it was either equal to or less than the general reuse. We further studied the coverage, the overlap and the pick-up rate of these resources, and propose steps that could improve the probability of discovery, use and reuse of cross-boundary resources.

I actually have a new academic homepage too, check it out http://elgg.ou.nl/rvu

How different is user behaviour on a portal from the ones who log-in to ones how do not?

2008-12-19T22:59:00.017+01:00

I've recently done quite a few studies on users of learning resources portals, I've looked for example how do they tag resources in a multilingual context or how much use and reuse is there across the borders. In all cases the studies have concentrated on the small amount of the (minority) users who actually log in and had created Collections of resources: in Calibrate that was about 30% and in LeMill about 10%.

Now in MELT we've revised the logging scheme to collect the click-stream from users who don't log-in. We also have Google analytics, but I don't have those at hand right now. I looked at the data from last 3 months, from Aug 18 to Dec 18, and then only from the last month (Table 1).

What do users do on the portal?

The most popular activity on the portal is search, 64.29% of all actions on the portal are different types of searches. They result in "playing" the resources in 18.31% of all actions on the portal. 13.09% of all actions are contributing actions on the portal, this means adding a tag, bookmarking or rating it. The figure of contributing actions is actually a bit distorted, we count each tag, rating and bookmarking there. As each bookmark has average of 4.3 tags attached to it, it brings up the figure. Actually, the number of actions that contribute to "acting with an individual resource" is around 4.3% of all actions (i.e. add rate and bookmark). Other includes activities like view evaluation, view other users who have bookmarked the resources, etc.

Table 1

The downside here is not having the stats from Google analytics, so I cannot exclude our internal usage, which I know has been quite a lot, since we've been testing the portal internally. So the figures might be somewhat distorted...

What about users who log-in and the ones who don't?

About a month ago I invited some 260 teachers on the portal, so I was intrigued to see what had happened. 2 weeks ago I checked that 11% of these teachers had started their own account. But, it seems like much more have come about and cruised around the portal.

Table 2 presents the data from the last 4.5 months (Aug-December) where I have divided it in two slots: first months include pilot teachers and lots of testing, in the table it's erroneously called "First 2 months". The second slot covers the time from Nov 18 to Dec 18 when we invited the new teachers (Nov 18/19 in 4 different patches of invites). It is called "the 3rd month" in the tables (again, my mistake). Moreover, the top half of the table has data regarding users who log-in and the bottom with users who did not log in.

I have mostly the same attributes for both, how did they search; advanced, browsing by category and by tag cloud and how many resources they clicked on (play). The table also contains the number of sessions and number of actions. A session is one consecutive event when the user does something, it's logged. If left idel, the user is logged off in some time. An action is anything, a search, a click on a resource, on a tag, etc. Additionally, we have the contribution by logged-in users, these are tags, bookmarks and ratings.

Table 2

As you can see, most of the sessions (above 86%, the second last row) in both slots take place when users are not logged in. Actually, the percentage of sessions stays pretty regular in both slots. Moreover, regarding the actions, we can see that during first months they mostly (70%) came from non-logged in users. However, when we invited the new teachers, we see 10% increase in actions by logged in users (from 30% to 40%). That's positive, as it shows that some of the invited teachers were motivated to contribute.

There is actually quite big differences in what do these two groups of users do when they are on the portal. Where logged-in users spend about 1/3 of their actions in searching, non logged-in users spent about 2/3 of their actions in searching. Chart 1 shows this clearly, however, I must say that most likely the disparity between the number of searches and plays by non-logged in users in the first months are due to our internal testing. If you compare that to non logged-in users in the 3rd month, you see that there is already less searches and more plays.

There has been a difference since the new comers (3rd month): within the logged in users, the number of searches executed has gone down (10%), whereas the number of plays has gone up (from 17% to 23%). Among non-logged in users there is the same 10% drop in searches, but plays have gone up by 10% (from 16% to 27%)! That shows that the new comers were interested in seeing what kind of resources were out there in the portal.

Chart 1 can maybe be used to illustrate

Chart 1

One difference can be observed in how differently these two groups seem to search: with logged-in users the advanced search seems to be the more popular way to search (more than 50% of searches are advanced), whereas with the users who are not logged-in browsing (both by category and tag cloud) is more popular. During first months 54% of searches were browsing, which went slightly up (to 56%) during the 3rd month. The tag cloud was the biggest winner in both groups (logged-in and not) at the cost of advanced search. I assume the difference is due to the fact that people who are not logged in are interested in seeing what is out there and browse around to discover learning material.

In any case, if we look at the figures of non logged-in users within the 3rd month, it's intriguing how equally the searches are distributed across these different ways of searching. We'll keep an eye on this in the future (e.g. when we know that most non-logged clicks come from us testing the portal).

Consumers and contributors

In Table 3, where I again have data for users logged-in and not, and by periods of first months and the 3rd month, we see that when users are logged in, they do things differently. First of all, the logged-in users spare much smaller percentage of their actions in searching (average 33% to 75%), however, bizarrely, they still seem to "play" about the same amount of resources (around 20%).

Table 3

Within the 3rd month we see the percentage of plays growing. We can assume that the logs from the first months period are most likely influenced due to our internal testing of the portal, which often times includes making searches. We see that the percentage of plays go up for the non logged-in users within the last month (from 16% to 27%), which, I assume shows a more normal user-behaviour than what could be observed before.

This still indicates that there is lots of inefficiency when non-logged in users search: on the average during the 3rd month, for those logged-in, one "play" was a result of 1.2 searches, whereas with those not logged-in, one "play" was a result of 2.6 searches - lot of time lost in searching. From Table 2 we can observe that there was more browsing (non logged-in users 1 month), I wonder if that was the reason? Have to keep on eye at that one!

Most interestingly, 40% of actions by logged in users contribute are the ones that contribute something to the portal, they rate, tag and bookmark. Folks who do not log-in are consumers: they only search, click and leave (- which is fine too).

So all in all, if we look at all the actions on the portal, the contributing actions by logged in users amount to about 17%. Too bad that this figure did not go up in the 3rd month like some others did. Anyhow, it seems to follow the power-law of distribution (20-80), where small amount of people contribute a lot so that other people can take advantage of this work, also know as participation inequality by J.Nilsen (2006).

J.Nilsen (2006) Participation inequality: Encouraging More Users to contribute

Learning resources landscape

2008-12-19T14:55:00.004+01:00

Learning resources come in all colours and shapes, that is for sure. They also come from all kinds of different places; repositories, portals, the web.... For a recent presentation and paper, I created this diagram to better depict the learning resources landscape. As I later had to remove this part from the paper to save place, I post it here.

Teachers use a plethora of ways to discover educational content online. Harvey et al. (2006) report on search strategies of 4500 US faculty members where Google-like searches are by far the most prominent (81%), second most important being own personal Collections of resources and also “portals” that provide links to disciplinary topics (55%). In our user group comprised of 45 language and science teachers in K-12 education, such diversity of strategies was also discovered: one third use national and regional educational repositories as their primary source of educational content, 28% use search engines, 21% said they create their own content, 7% use content from schoolbook publishers and 12% reported all of the above (Vuorikari, 2008a).

These search strategies also give an indication of the different types of resources that teachers use. Figure 1 illustrates a number of different sources of content that teachers use. First of all, on the horizontal axis we distinguish between platforms that have institutional support and the ones that are rather teachers’ community driven sources. On the vertical axis we distinguish between teacher-generated content and “other sources”. The latter encompasses a large number of providers from educational portals and repositories, schoolbook publishers to educational and non-educational sites created by a number of private and public stakeholders. This “other sources” category is essentially as large as a teacher’s pedagogical imagination is in taking advantage of the resources on the Internet.

This diagram allows us to draw a landscape for educational resources. In the upper left corner of the diagram, there are examples of institutional Learning Object Repositories (LOR), such as the ones managed by Educational Authorities (e.g. Learning Resource Exchange for schools and members of EdReNe) and other repositories that make educational content available. On the lower left corner we place initiatives like MIT OCW which is an institutional repository that makes available teacher-generated content. The lower right corner represents teacher-generated content in a community-driven environment (e.g., LeMill), whereas the upper right hand corner represents content that is found on the Internet from various sources and saved in community-driven environments like delicious.com. None of these boundaries are fixed and there are many in-between-models (e.g., LOR with both user-generated content and institutional ones). Our data sets for this study, which are presented in Table 1, cover a wide area of Figure 1. For learning resources we use Wiley’s (2002) definition of learning object as “any digital resources that can be reused to support learning”, as they vary greatly in granularity and other qualities.

Our evidence finding focuses on teachers in K-12 education in a European multilingual context. In the Europe Union area, where 497 million people (Eurostats) live from diverse ethnic, cultural and linguistic backgrounds, multilinguality has an important role (Council of Europe, 2007). There are 23 EU official languages, 3 alphabets, and some 60 other languages are part of the EU heritage and spoken in specific regions or by specific groups (COM, 2008). Multilinguality can be defined as a situation where several languages are spoken within a certain geographical area, as well as the ability of a person to master multiple languages. 56% of EU citizens say that they are able to hold a conversation in one language apart from their mother tongue, and 28% in at least two languages. English remains the most widely spoken foreign language throughout Europe (38%), second and third place is French (14%) and German (14%), whereas 6% have foreign language expertise in Spanish and Russian respectively. Over two-third say that they language lessons at school was the way they have learned foreign languages (COM, 2006).

..................

Harley, D., Henke, J., Lawrence, S., Miller, I., Perciali, I., and Nasatir, D. (2006). Use and Users of Digital Resources: A Focus on Undergraduate Education in the Humanities and Social Sciences. Available from
http://cshe.berkeley.edu/research/digitalresourcestudy/report/digitalresourcestudy_final_report.pdf

Vuorikari, R. (2008a). A case study on teachers' use of social tagging tools to create collections of resources - and how to consolidate them. In Wild, F., Kalz, M., Palmer, M (Eds) Proceedings of the First International Workshop on Mashup Personal Learning Environments. Available from http://sunsite.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-388/vuorikari.pdf

Wiley, D. (2002). The Instructional Use of Learning Objects. Online at: http://reusability.org/read

COM(2006). Europeans and their languages. Special Eurobarometer, European Commission.

COM(2008). 566 final. Multilingualism: an asset for Europe and a shared commitment, European Commission.

Council of Europe (2007). Un cardre Européen commun de référence pour les langues : apprendre, enseigner, évaluer. Division des Politiques Linguistiques, Strasbourg: France.

Mind those backups! - stolen laptops

2008-11-06T15:30:00.007+01:00

Oh boy, my digital misery does not seem to be over yet! After having my home broken into and my laptop stolen, I was dead-happy that the evil-minded robber left my external, one terabite back-up hard-drive right where it was, about 50 cm away from from the laptop that s/he stole.

Now about 10 days later I'm typing away using my new MacBook Pro. I must say that I was pretty happy to have restored my "life" using Time Machine, only about a week's worth of back-ups was missing. It felt great to log-in to my own desktop, have my applications restored just like that, and just start using my laptop like nothing happened. I even pledged to always do my backups.

Then I started noticing little things. Ops, my address book is missing. My bookmarks and passwords are missing in Firefox. Then, I wanted to upload a file, and I realised that ALL MY FILES ARE MISSING! WTF?

Those files are, for sure, on the back-up hard-drive, I can access them there. But for some reason, when importing "my life" from Time machine, it omitted to import my files. Hey, big deal, at least the file structure is there..dah.

Now I'm trying to discover an easy way to do that. I am finding all kinds of not so nice things on Time machine, like this.

Ok now, time to roll up my sleeves and start digging out those files. So far this post looks promising.

As a lesson learned: mind your back-ups!

Power of pursuation/example/recommendation

2008-11-05T16:18:00.006+01:00

We plan to invite about 130 teachers to the portal and I'd like to make a little experiment on this. The idea is to show two different examples of how to access and discover learning resources on the portal, and see whether that has an influence on

Uptake: users sign up
Retention: users come back
Different ways to discover resources (social cues vs. conventional search strategies)
Focus on the system (discovery e.g. clicking on resources vs. contributing, e.g. bookmark and tag, rate, etc)

The point would be to test the statement from the results reported in Harper et al (2007), where an email newsletter with manipulated social comparison made no difference to a member's interest in using the system, but it changed their focus within the system.

While subjects who received an email message with the comparison manipulation were no more likely to click on one of the links or log in to the system, they were more likely to rate movies.

There are differences, but in grosso modo the idea has similarities: In Harper et al (2007) the manipulated social information in the email concerned the subject him/herself (treatment group), whereas in my case the information would be about some other teachers that the subjects would be able to relate with (e.g. see favourites from a science or language teacher). The biggest difference would be that there is no comparison aspect of the subject's performance to the other users in the system, which was one of the central features of Harper et al study.

Study design

The randomly selected treatment group would receive an invitation with set goals of expectations on the use of the portal. Examples will be given on how to access and discover learning resources based on social navigation cues, show examples of how to browse other user's Favourites and how to access resources through a tag cloud.

The randomly selected control group will receive an invitation with set goals of expectations on the use of the portal. Examples will be given on how to access and discover learning resources would receive an invitation where examples would be given on how to access and discover learning resources based on conventional free text search or browsing keywords
(to be reworked, just initial ideas based on tracking methods that I could use).

Immediate reaction: number of people who access the portal through the direct links on the invitation as opposed to the number of people who access the portal though the main page. the time these people spend on the portal on the first time and how they search, how many searches they execute and how many resources they click on
Uptake: is there difference between the groups on signing up on the portal?
Retention: on the longer run, say, within a month, do they still come back
Different ways to discover resources (social cues vs. conventional search strategies)
Focus on the system (discovery vs. contributing in terms of ratings, tagging, etc) : do people who see examples of social navigation use these methods more than the control group? are there any differences how many resources the subjects in both groups tag and rate?

Hypothesis to test
(to be reworked, just basic ideas)

hypotheses would be that subjects in the treatment group will discover more resources than the control due to social navigation cues made readily available to them. By discovering I mean that they click on these resources on the portal to view them. I also would hypothesise that the retention rate is better among the treatment group, as they get a direct access to selected resources whereas the control group has to look for the interesting resources and might get diverted there.

Relevant studies in this direction, to be completed

A study towards this direction was reported by Harper et al (2007). They study the effect of email newsletters that told the community members whether their contribution was above or below average. They report that a) previous studies has shown that information about social norms can affect contributions, e.g. people recycled more material when they were provided with information about how much other people had recycled (Schultz, 1999).

Social Comparisons to Motivate Contributions to an Online Community., Harper, F.; Li, X.; Chen, Y.; Konstan, J. , Persuasive Technology, 26/04/2007, Palo Alto, CA, (2007)

Using Social Psychology to Motivate Contributions to Online Communities, Ling, K.; Beenen, G.; Ludford, P.; Wang, X.; Chang, K.; Li, X.; Cosley, D.; Frankowski, D.; Terveen, L.; Rashid, A.M.; Resnick, P.; Kraut, R. , Journal of Computer-Mediated Communication, Volume 10, Issue 4, (2005)

Changing Behavior With Normative Feedback Interventions: A Field Experiment on Curbside Recycling (1999)

by P Schultz

Web 2.0 = Learning 2.0?

2008-11-03T14:57:00.006+01:00

Last week I was part of an expert workshop on Learning 2.0. It was organised by the Institute for Prospective Technological Studies (IPTS), one of the European Commission's research institutes. They currently run a year-long study on the Impact of Web 2.0 Innovations on Education and Training in Europe. The objective is to assess the impact of Web 2.0 trends on the field of learning and education in Europe, and to propose avenues for further research and policy-making in Europe.

It was very cool to be part of this group, we were about 30 people with various backgrounds and our main job was to looked at the preliminary results of two studies. We first discussed the intermediate results of an exploratory study that seeks to identify and analyse the existing practices related to the Web 2.0 initiatives in the field of learning in Europe. For this reason, a Learning 2.0 database had been set-up where practitioners were able to report their cases. A presentation of this study is available.

The second part of the validation workshop focused on the cases studies: Case study on 'Good Practices for Learning 2.0: Innovation' and Case study on 'Good Practices for Learning 2.0: Inclusion'.

A lot of the workshop time was spent on brainstorming mode, which is something that I truly enjoyed. The point was to try to identify what would be the NEW in what was called Learning 2.0. That's pretty tough, as we hardly know what is the new thing in Learning 1.0! Here is one image of our brainstorming chart, thanks to Graham!

The most skepticism, if I could even call it that as many of us were very enthusiastic about the potential of Web 2.0 for education, was that how can Web 2.0 technologies and tool help the learning, or can they help it at all?

Lots of things could and have been listed by the proponents of Web 2.0, like personalisation, participation, collaboration, motivation, social skills, reflection and meta-cognition. Those, however, are not inherit to Web 2.0, but to any good learning!

So is there anything that makes learning with Web 2.0 so special? In contrary to encouraging reflective learning, Web 2.0 seem to promote sporadic grasshopper minds, like some current studies on multitasking suggest.

What the workshop could say, though, was that more well coordinated research on Learning 2.0 is needed to better understand its potential. One such study in this direction is the new Becta study, which is very impressive.

Some links to other literature that folks in the workshop pointed out: http://delicious.com/tag/iptsl20

Also, check out the literature reviews and other studies that have already come out of Learning 2.0 or are about to come out. There are interesting things going on!

From the Learning 2.0. site:

The rapid growth of social computing or web 2.0 applications and supporting technologies (E.g. blogs, podcasts, wikis, social networking sites, sharing of bookmarks, VoIP and P2P services), both in terms of number of users/subscribers and in terms of usage patterns leads to the fact that the phenomena are also increasingly being used in the educational field and for learning purposes. As it enables different types of learning and teaching settings (formal, non-formal and informal), it is an important driver of innovation in learning.

Description: The Learning 2.0 study will

1. Identify and analyse the existing practices and related success factors of major web 2.0 initiatives in the field of learning in Europe;
2. Look at the innovative dimension of using web 2.0 for learning;
3. Analyse the position of Europe vs. the rest of the world in terms of quantitative and qualitative use of innovative Learning 2.0 approaches;
4. Discuss the potential of social computing applications to (re)-connect groups at risk-of-exclusion;
5. Propose avenues for further research and policy-making.

Edupunk it, this is darn cool!

2008-09-26T18:28:00.002+02:00

Encourage lurking in conferences!

2008-09-19T11:41:00.005+02:00

I'm always amazed how old-school these conferences are. I'm currently in Ec-Tel 08, sitting outside of the session room on the floor and listening the speech through the half-open door. I'm a lurker.

I do not want to participate in the whole session, but I do want to lurk around because I'm half interested and there is one speaker that I want to see later. I just don't want to sit in the room and tap on my laptop.

It would be great to have a couch or a few seats outside of a session room with a screen, and hopefully also audio, to be able to properly lurk. We already know from online communities that it is totally OK to lurk, so why would we not make it also properly available in the conferences, too?

I've had really interesting conversations in these marginal places of conferences. Let's just make them happen more often! Let's provide places to officially lurk around.

Cross-boundary ranking of learning resources

2008-09-12T18:25:00.010+02:00

Based on the idea of Interest Indicators, like social bookmarks and ratings, I've looked at the data so that we can make the cross-boundary resources better available on the MELT portal.
The aim is that we can, based on previous users' behaviour :

a) make separate "travel well" lists of resources that have a potential to cross-borders better,

b) use this information to rank resources better in the normal search result list,

c) allow users search for resources that have a good "travel well" value (e.g. give me resources in math that can cross-borders)

This is the data that I'm using (table below) and this is how I've defined cross-boundary (e.g. cross-country and language) learning resources before. Using that definition I have manually verified the number of cross-country resources. In the dataset, about 82% of resources were cross-country.

Now, we have a problem, though. On our MELT portal we do not have information about the country where the resource originates from. Dah!

This is a big blunder (in my opinion) in our Application Profile, we have not defined the country where the resource originates. We do define the provider, and the country information could be inferred from the provider, but it does not always work.

For example, one of our providers has frequently metadata about resources that do not originate from the same country!

I've experimented with the data using the information that we have on the portal, which is LOM about the resource including the language of the resource. As we also know the mother tongue of the registered users, this gives us a kick.

In the table below we can see the coverage of cross-boundary actions that we can get on resources without using any manual labour or verification of the country or language. As a base-line, with manual verification I found that 82% of the actions concerned cross-border rating or bookmarking of a resource.

The first row represents the cross-language resources (i.e. user's mother tongue is different from the resource language). Just using this information, we get about 65% of resources right, as opposed using manual checking (82%). I think it's pretty good, I'd settle for that! (although I have to look what kind of material was left out!)

The two other comparisons in the table are based only using information about users' previous behaviour. These would be:

rating > 2
bookmark

Only using information regarding bookmarked and rated resources results in a lousy coverage of around 20%. The problem is that 25% of resources bookmarked or rated are on more than one resource, the data still is very sparse.

Anyway, I want to use that information to "cross-boundary rank" the resources. As we do not know the country where the resource comes from, my work-around is based on countries where these users come from.

Here is a visualisation about resources that have been bookmarked or rated by users (see also ManyEyes link below). We can see the orange node in the middle, a learning resource called "Five Days in New York..". We see 3 edges leading out to Finland, Belgium and Hungary. This means at least one user from each of these countries has bookmarked the resource!

So, even if we do not know the origin of the resource, we know that it has users from 3 different countries. I can infer that it is a cross-boundary resource.

As most likely one of these 3 users come from the same country than the resource comes from, I will minus one country out of the total of countries: (number of countries -1)

My cross-boundary rank will be the following:

Count the number of ratings grater than 2 and/or bookmarks for a resource (actions). Give each action one point
Count the number of these users and give each user one point
Count the number of user countries of origin. Give each country one point and then minus 1
Compare the mother tongue of each of these users to the language of resource. If they differ, give one point/mismatch.

Then, count the following:

number of users + number of actions + number of cross-language x (number of countries -1)

Let's take the above resource "Five Days in New York.." as an example

Count the number of ratings grater than 2 (3) and/or bookmarks for a resource (5). Give each action one point. (8)
Count the number of these users and give each user one point (5)
Count the number of user countries of origin (Hungary, Finland, Belgium). Give each country one point and then minus 1 ( 3-1=2)
Count the number of user mother tongue (hu, nl, fi). Compare the mother tongue of each of these users to the language of resource (en). If they differ, give one point/mismatch (3).
number of users (5) + number of actions (8) + number of cross-language (3) x (number of countries -1) (2) = 32 Travel well value

This way you can count a value of "travel well" for each resource that users have previously interacted with on the portal. The value will always be an integer, which is important from the technical implementation point of view (in Lucine index it apparently needs to be an integer).

The down side is that we'll have a huge cold-start problem. As I said, our data is very sparse. To seed the system, I actually still manually check the new resources that users have interacted with and make a fake bookmark on them so that it looks like it has at least two users from 2 different countries. This way the resource gets a "travel well" value counted and appears on the "travel well" list and is better ranked, etc.

Of course, at the end I will evaluate how this treatment affects on users, do we, for example, see a big amount of bookmarks on these resources that I have been able to count a travel well value?

You can see a visualisation here. This is based on on user's country of origin.

Cross-boundary use of learning resources in LeMill

2008-09-03T15:44:00.012+02:00

LeMill (http://lemill.net) is a web community for finding, authoring and sharing learning resources. It is divided to four sections: Content, Methods, Tools and Community. The main target audience is primary and secondary school teachers, but anyone can join. It is a wiki-like system where all the learning resources are published under open licence and can be edited by other members.

Registered users can publish learning content, and descriptions of educational methods and tools. Users can also create their own Collections of learning resources. About 10% of LeMill users have created collections (users total based on March 15 2008), this represents 188 users (data snapshot May 30 2008).

Collections are good for "Keeping found things found", the nice resources that you find in LeMill can be easily put in a collection. Another important thing is that Collections can be used to make content units or thematic lessons. Collections are actually folders that you give a name, you can make as many collections as you want.

For me the Collections tool is interesting. It's the whole thing about what content do users find interesting enough so that they want to keep it.

In Table 1 you can find the description of the data that I use for this study. I give quickly some descriptive statistics about it, and then drill into the Cross-Boundary use of learning resources. This this I mean that the user comes from a different country than the resource (cross-country) or the resource is in a different language than that of the user’s mother tongue (cross-language). If resource comes from a different country and is in a differnt language, then it's a cross-border resource. I'll give examples of this later to make it more clear.

What's in Collections?

Users had saved 1645 resources in 376 collections. There is 4.4 resources on the average in each collections. Some Collections are huge, the biggest has 82 resources in it, whereas the median is 2 resources.

When we look at how the users have used this feature, we find a wide variety of cases. Just by looking at numbers, the most active user had 94 resources in Collections, whereas the average is 8,75 resources. Median is 3, so as usual, we have a group of very active users (30% above average) and lots of not-so-active users of this functionality.

When we look at the resources in Collections, we find that there are 1387 different resources. 13% of the resources exist in more than one collection, but most of the resources (1205) are put to only one Collection. Not much of an overlap there, which is a bit surprising seen the fact that other users' Collections are public, so I can easily go and see the lessons created by others. There are nice pivotal navigational features that allow me to click on the other user's name and see their Collections. To my dismay, though, I noticed that it is not very easy to add resources from other users' Collection to mine, which might hinder the reuse a bit.

Who uses Collections?

The cool thing about LeMill is that their user-base is a total fruit bowl. There are users registered from all over, in this dataset we have 22 countries. The top number of Collections are Estonians, that's 45% of all resources in Collections. Others are Lithuania (14%), n/a (14%), Hungary (9%), and so on.

Where do resources come from in Collections?

It seems like most Estonians put resources made by another Estonian in their collections (48.5%). Then we have n/a "country" (13.2%), resources from Lithuania (11.4%), from Finland (7.5%), Hungary (6.6%), resources from Georgia (6.2%) and so on.

What about resources originating from different countries than the user? The case of crossing boundaries.

About 40% of Collection users are Cross-Boundary users (74). You can see this at the lower part of the 2nd Table above. I was able to calculate this by looking into the resources that they have put in their Collections. I found out that every fourth (419) resource in Collections crosses some boundaries, either language or country boundaries. Let's look at this in more details (Table here on the left).

1st Case are Cross-border resources: the resource comes from a different country than the user and is also in a different language than the mother tongue. 28% of the cases were like this.

Example: a German resource from a German teacher that a Finnish teacher has put in his Collection.

2nd Case is about crossing language borders. Most cases (76%) represent resources that are in a different language than the mother tongue of the user is. Many of these resources are in English (63%) even if they are not created by an English native speaker.

Example: Let's say an Estonian teacher has made a learning resource in English and put it in her own Collection.

3rd Case: Cross-country resource. These are resources that are in the user's mother tongue, but come from a different country. Much less of those, only less than 5%.

Example: A resource in English made by an American and put in a Collection by a Canadian. Or it could be a German resource added into a Collection by an Austrian teacher.

So are there any commonalities in this?

In LeMill, it looks like most cross-boundary actions (i.e. resources put in Collections) are within users' language skills (lower part of the Table on the left). You'd be surprised to learn the language skills these teachers have! Most of them have one additional language on top of their mother tongue, but many of them boast 3 or 4. This results that most of the 352/419 cross-boundary resources in collections are within Foreign language skills of these users. That's 84%.

Naturally, one is curious to know what those remaining 16% of resources are?
Actually, a bit disapointingly, there were little surprise (and very little of evidence on my last post on "Vygotsky's psychological tools". Many of these resources were in English, most likely the user had forgotten to mention that s/he masters this lingua franca. Remaining were either with no text or multimedia, some resources I was not able to locate in LeMill anymore. I put some of them in my Travel Well Collection.

Emerging search patterns on learning resources

2008-09-02T20:57:00.009+02:00

I am hugely inspired by the stuff from J.Feinberg and D.Millen, especially by the studies that they've done on doger, the IBM internal social bookmarking service. I must admit, though, that I had missed on it a bit, I cannot believe! Anyway..

This paper is really interesting, Social bookmarking and exploratory search (2007), not least for the reason that it offers a very interesting, almost similar study design that I am planning on my log files and search pattern analysis on the MELT portal. (Great minds think a like ;p yeah, right..)

The study design is a field study of a social bookmakring service in a large corporate (IBM) with quantitative data (click level analyses of log files and boomarking data) and qualitative data like interviews.

And, this is the data that I collect for my study (Table 1). Quite simlar!

They use a categorisation of search that I will adapt to my usage:

Community browsing (Examining bookmarks created by the community. In my case these could be lists of most boomarked items, travel well; tags; other people Favourites)
Personal search (Looking for bookmarks from one's own personal collecction of bookmarks)
Explicit search (Explicit search using traditional search box)

A few days ago I looked at the first logs from Melt. Here is the run-down. I have not used the same types of searches, but I will explore them for the later usage. But basically what you can see:

There were 512 search events:

41% Explicit searches (adv. search)
34% Community browsing (tagcloud)
25% browsing categories

What was called "click-through" in this paper is when the particular navigation path resulted in a page view. This is what I call view resources. We can see that there were 538 page views, which implies that most likely users have clicked on more than one resource as a result of a search.

74% of resources were viewed in search result list (srl), this means that they were results from advanced search and browsing categories
20% of resources were viewed as a result of community browsing (e.g. tagcloud, lists)
5% of of resources were viewed as a result of personal search (e.g. in Favourites)

How many searches resulted in viewing resources? I have to verify this

85% of explicit searches and browsing categories
65% of Community browsing (tag cloud and lists)

Millen et al. 2007 speculate that in their study, the a higher click-through rate indicates a more purposeful searching, whereas the community browsing was used more as an exploratrory search activity. I will need to keep my attention on this and whether I can make similar conclusions.

Most likely, anyway, I do not use click-through as an indication of intenet of using a learning resources, what I find most interesting in my study will be how many of the search activities result in a bookmark and/or rating. That is much cooler in my mind than viewing the page. Actually, I've noticed that in our system users view resources a lot, but they do not necessarily show any further interest on them.

That is why I use both implicit and explicit Interest Indicators. By Interest Indicators I mean Explicit Interest Indicators like ratings as a subjective relevance judgment on a given learning content, Marking Interest Indicators like bookmarks and tags on educational content, and Navigational Interest Indicators such as time spent on evaluating the metadata of educational resources, as well as Repetition Interest Indicators as categorised by Claypool, et al., 2001.

In this small study we can see that 19% of viewed resources ended up in users' Favourites. Again, ratings were much less, only about 6% of viewed resources ended up being rated. In the study from IBM system, they had 34-39% as high click-through. Will be keeping my eye on that.

My advisor asked me whether I could find search patterns in my logs. I think I could. In this paper (Millen et al. 2007) they do that :) and here is how: "Looking for Patterns using Cluster Analysis"

To better see the patterns of use, we performed a cluster analysis (K-means) for the different types of search activities. We first normalized the use data for each end-user by computing the percentage of each search type (i.e., community, person, topic, personal, and explicit search). The K- means cluster analysis is then performed, which finds related groups of users with
similar distributions of the five search activities. The resulting cluster solution, shown in Table 4, shows the average percentage of item type for each of four easily interpretable clusters.

Should not be too hard :)

Millen, D., Yang, M., Whittaker, S., Feinberg, J., Social bookmarking and exploratory search (2007). In L. Bannon, I. Wagner, C. Gutwin, R. Harper, and K. Schmidt (eds.).
ECSCW’07: Proceedings of the Tenth European Conference on Computer Supported Cooperative. Work, 24-28 September 2007, Limerick, Ireland

Activity Theory helping us explain folksonomies

2008-09-02T15:32:00.002+02:00

I've been lately reading Engeström's (1) stuff and about Activity Theory. In this paper (2) I found a cool reference that explains how using Activity Theory as a theoretical framework we can study learning resource repositories (LOR) and their communities as one single system rather than as a loose set of instruments, subject, objects and outcomes.

I think that is a very important point. I've been arguing for quite a while that tags and social bookmarking can be revolutionary for LOR because now we can make a connection between the user, the resource and its metadata.

Before, it was only the resource and metadata, and a scary looking form for searching the resources (this is time before Google's simple search box, right?). Now, if implemented correctly, social bookmarking and tagging not only helps individuals with their resources management (e.g. Favourites), but also helps other folks to find resources through other users and their digital traces such as tags, number of bookmarks, etc.

To understand Activity Theory it is important to get the bases: everything, well, everything within human activity, is based on the three dominant aspects which are production, distribution and exchange (or communication).

The model suggest the possibility of analyzing a multitude of relations within the triangular structure of activity. However, the essential task is always to grasp the systemic whole, not just separate connections. (no page number in my print, just below Figure 2.6).

This is also the base for the analysis in Margaryan & Littlejohn (2008) for the learning resource repositories as instrument, with rules, division of labour, outcomes, etc.

Moreover, Engström emphasises that there is no activity without the component of production.

The specificity of human activity is that it yields more than what goes into the immediate reproduction of the subjects of productions. One part of this "more" is the surplus product that leads to sharing and sociality, discussed by Leakey & Lewin and Ruben above...(found on the next page)

I was thinking of folksonomies and how the production of tags is first of all good for me. People often times tag and bookmark to "keep found things found", it's part of personal knowledge management activity. Similarly like above, when referred to, for example, production of food that leads to sharing and sociality, in tags, the fact that they are made available to all, leads to sharing and sociality.

I thought that was pretty neat.

Ideas for the design of the "Evidence" study on cross-border use of learning resources

2008-09-02T10:53:00.015+02:00

"Psychological tools" helping us explain "Travel well" resources

In the chapter 2, in the subsection called The Third Lineage: From Vygotsky to Leont'ev Engström (1) talks how Vygotsky distinguished between two interrelated types of mediating instruments in human activity: tools and signs. The latter belonged to the broader category of "psychological tools".

Psychological tools

..are directed towards the mastery or control of behavioral processes - someone else's or one's own - just as technical means are directed towards the control of processes of nature. (I guess this is from the same source as the quote above this, this is Vygotsky 1978, 55)

Examples of psychological tools:

various systems for counting; mnemonic techniques; algebraic symbol systems; works of art; writing; schemes, diagrams, maps, and technical drawings; all sorts of conventional signs, and so on. (Vygotsky, 1982:137, cited in Cole & Wertsch)

...and folksonomies :)

I was thinking that the concept of psychological tools is a pretty cool way to start looking into the cross-border use of digital learning resources. My definition is: By cross-border use of digital resources we mean that the user comes from a different country than the resource (cross-country) and/or the resource is in a different language than that of the user’s mother tongue (cross-language).

Many people are baffled about the cross-border use, they always ask me "..but how would teachers be able to make any use of a resource that is in a language that they don't understand".

Let me first elaborate on different types of use of cross-border resources, and then I give my theory on it. The plan is to make a study to see whether this holds or not.

We had a workshop with 35 teachers in science and language learning from different European countries. We asked them to bring along a learning resource that they though would be useful for other teachers in other countries. The observations were the following:

1. Resources that contain psychological tools: examples of this type of resources were in science, biology and math. Here are some examples of the characteristics of these resources

How chromosomes define characteristics (e.g. eye color, color of rabbit) or how the human heart works (we actually had examples of this in 2 different languages!).
If you know the concept (as this would be part of the acquired knowledge of a teacher) you can explain it using this type of examples. It's not important that the manipulations are not in their own language, as the user interface is pretty symbolic and self-explanatory. Also, the little texts in other language did not seem to bother teachers.
DNA and how it works, another one on chemistry. The intersting thing is that the text is in Estonian and the resource was intended for pupils, but the group of teachers agreed that they would find it useful as a tool to demonstrate the concept by themselves (note: different intended user group). They explained that they would manipulate the resource for demonstrational purposes, not let the pupils to use it.
GeoGebra was one of the examples of a math application. They all loved it! Most importantly, it can be translated easily and it has user communities in different languages.
Besides those points, they said that as math symbols are commonly shared, it is easy even in other languages. Even this type of applet would be useful for a non-Spanish speaker. BTW, they hated when some resources did not use the proper symbols, but wrote out "tiempo"

2. Resources with more text in a foreign language. One could also observe that some teachers were not minding too much about the text in foreign languages, but they used their pedagogical skills to work that into a challenge to learn.

One example from another workshop was a history resource about Greco-Persian war. Although this was already harder to navigate in Polish, a Belgian teacher started coming up with ideas where learners have to solve the language as a challenge. An example was given about a "match the words with an image"-type an exercise about the war equipment of a Greek solder.
Another innovative usage was this Japanese virtual reality game, where users have to find their way out of the virtual room with a help of a team. This Hungarian teacher had given it as an English exercise for his students to solve as a group and write down the instructions in English. He said that students were completing the exercise on Friday evening working online with their buddies!

3. Resources that are in the languages that the user has competencies in. This is of course the most used case. If you have studies Russian, for example, you can use resources in that language.

4. Language teachers. This is a group a bit apart too. They, of course, find the whole Internet as their resource for learning! But also the language resources that are created, say, in Finland to study Way finding in French, can be useful in any other language teacher somewhere else. Here the important thing is to make the instructions also in the language that is being taught, so that teachers understand (of course best is making interfaces easy enough without any instructions needed)

So, my theory is that the use of cross-border resources plays on a continuum that has two quite distinct extremes: On the one end we have psychological tools (example 1 and 2) and on the other Foreign language as a tool (example 3 and 4).

The acceptance or willingness of using this kind of material is related to the teacher's previous knowledge and understanding on the topic on the one hand, and on the other, it can be the knowledge or previous experience on coping with foreign languages. Additionally, the pedagogical skills set and pedagogical concepts that are preferred by that teacher drive the final decision on using such material.

A study design

In my study of "Finding evidence" I will only focus on the continuum of psychological tools and foreign language skills. I have a huge dataset from at least 3 or 4 different learning resource environments where users (teachers) have bookmarked or made collections of learning resources that exist in multiple languages.

The dataset currently has 440 users who have selected at least one learning resource to bookmark or add into their collection.

Calibrate (176 users, number of posts=1742)
LeMill (189 users, number of posts 1645, out of which 238 cross-border actions)
del.icio.us (16 users, number of posts 1176).
MELT

When I look at the titles of these learning resources, there are 3700 of them. 2992 of these resources have been bookmarked only once. The idea is to sort out cross-border resources (using my definition above), and see whether I can classify them on my continuum.

The good thing is that I know the user languages and country of origin in all the cases, the bad this is that I do not know the country of origin or language of all the resources :( That seems like lots of resources starring.

1 Engeström, Y.: Learning by expanding: An activity theoretical approach to developmental research. Helsinki: Orienta-Konsultit Oy (1987) Retrieved August 25, 2008, from http://lchc.ucsd.edu/MCA/Paper/Engestrom/expanding/toc.htm

2 Margaryan, A., & Littlejohn, A.: Repositories and communities at cross-
purposes: Issues in sharing and reuse of digital learning resources. Journal of
Computer Assisted Learning (JCAL), 24(4), 333-347 (2008)

Notes on Margaryan, Littlejohn and Activity Theory as a framework

2008-08-25T18:08:00.007+02:00

Margaryan and Littlejohn (2007, 2008) analysed the mismatches in the perception of repository curators and users. One of the issues really hit home for me:

The curators focus on repository centric factors, while users spotlight a wide range of contextual factors. They explain this as following: Repositories are frequently introduced to users as sandalone tools. Users, however, see them only as one component within an entire activity system. They recommend that curators and users have to think through the ways in which individual components inter-relate.

This is what I actually realised this summer when we were at the summer school with MELT teachers. At the point where our system failed to work, teachers did not loose too much time but started checking their delicious accounts and bookmarking some interesting learning resources there that had been introduced earlier during the day. That moment, somehow, was an awakening moment for me. I realised that what I've been hassling about for so long, our dear repository, the one and only, is not really one and only source of information for them. Just one among many others that we are not even interested about.

Hence the little idea of integrating users delicous tags and bookmarks on the MELT portal. A logical place for them would be at the Favourites' section: here, on the first tap, are my bookmarks from MELT, and over here on the second tap, are my bookmarks from delicious too. Cool,ugh, inter-relating the services that teachers use. Also, since by default all my Favourites in MELT are publicly available to other users, so would my delicious bookmarks be.

The whole idea goes much further to integrating these using APML to create a profiling tag cloud from my tags from both places. The workshop paper is found here, I still need to work on it a bit.

Other interesting things about the papers:

The study was build using the Activity Theory from Engström 1987 as a theoretical framework. It also might be interesting for me, as I am missing one. Margaryan and Littlejohn (2007, 2008) claim that it offers a holistic framework that allows to study LORs and communities as a single system, rather than as a loose set of instruments, subject, objects and outcomes. It provides an analytic lens to understand the complex relationships wihin each system.

Activity Theory as such belongs to the family of socio-cultural approaches to learning (e.g. Vygotsky), situated learning theoris (Lave) and communities of practice approaches to learning (Wenger, there he is again..). The paper explains that the common denominator for socio-cultural theories is the importance of social and cultural contexts in learning.

From that perspective Activity Theory might make a nice match. One thing why I first was skeptical about it was that Margaryan and Littlejohn in (2007) say this theory offers a method of analysing the development of LORs as participatory environment where knowledge is co-constructed rather than "exchanged" or "consumed". I am not sure whether LORs really were developed in thinking of co-construction of knowledge, at least not before we mixed in the social tagging stuff. From that point, then, it becomes interesting, maybe.

In Margaryan and Littlejohn (2008) authors also talk about how social co-creation of knowledge is facilitated through the use of tools, either concepetual or physical. A dialogue can be such a conceptual tool, but so can email or blogs. Also tags, I guess, can subscribe to that.
Another thought that came out from reading the 2008 journal paper was that it also talked about Leontiev (1981) and analysing an activity from 3 different levels. The first level related to the overall motive for engaging with an activity. The second level relates to the actions that constitute an activity that are governed by (short-term) goals. The third level of activity related to the operations necessary for carrying out the actions. This made me think of "levels of participation" like in this ladder (or the long tail one). What they also try to depict is that there are different levels of participation, they are differently motivated, and maybe when talking about learning, we can also observe similar levels as pointed out by Leontiev (1981).

A few ideas for the evidence finding paper:

The dimensions of repositories and communities can be used to describe the datasets that I will use

Start for the evidence paper: Assume that repositories and learning resources get rid of technical, socio-cultural and pedagogical barriers for usage (references from the JISC report on Learning Communities and Repositories from CD-LOR project), does the re-use across the national and linguistic borders happen? If evidence is found, how much and where?

Engestroem, Y. (1987). Learning by expanding: An activity theoretical approach to developmental rsearch. Helsinki: Orienta-Konsultit Oy. Retrieved August 25, 2008, from
http://lchc.ucsd.edu/MCA/Paper/Engestrom/expanding/toc.htm

Margaryan, A., & Littlejohn, A. (2008). Repositories and communities at cross-purposes: Issues in sharing and reuse of digital learning resources. Journal of Computer Assisted Learning (JCAL), 24(4), 333-347.

Margaryan, A., Littlejohn, A. (2007) Communities at cross-purposes: Contradictions in the views of stakeholders of learning object repository systems. Proceedings ascilite, Singapore 2007.

How do tags connect to the Thesaurus terms?

2008-08-24T00:16:00.004+02:00

Our social tagging system in MELT is special in two ways:

one, we support multi-linguality
two, we have not only tags, but all resources that users tag also have Thesaurus terms

In that context it becomes very interesting to know how do tags relate to the Thesaurus terms that have been used to index the resources that users tag.

I took a sample of tagged resources (n=185) that have 1013 tags associated with them. Out of those tags, there are 595 distinct tags. There are 44 users.

I made a network diagram visualisation that displays the Thesaurus terms as nodes that are connected by edges to tags. You'll find it here to play around with it. Unfortunately, I found out that 24 resources did not have Thesaurus terms related to them(that's about 13%, hmmm), thus a big plumb node in the middle without a Thesaurus term.

There is another visualisation here, it's more explorative about the data.

It's rather interesting that 595 distinct tags from users can be comprised to 34 thesaurus terms. That is 17,5 tags per Thesaurus term on average. Of course it does not go like that, it's more like rich-get-richer-type of a story. In the visualisation above you can see that most tags are related to language learning, for example.

If you look at the distribution of tags you'll find that many of the top tags are also about languages. Interestingly, many of them repeat the topic of the resource, but some of them (clearly less) state something about the nature of the resource (e.g. interactive) or the type (e.g. exercise).

The problem with creating this kind of visualisation of tags on the system level will be that the resources seem to have too many Thesaurus term. If there are 5 or so indexing terms, everything becomes related to everything else. It might be interesting to either to ask limit the Thesaurus terms to three (as should be the case anyway) or ask the indexer to give one term priority over others.

The same also goes for content-based recommendations, btw. If there are too many terms, you recommend everything for everyone.

On the memory lane of the Internet - Paris 8

2008-08-20T22:00:00.010+02:00

It's great to be getting older. It turns the Internet into a memory lane, something like cleaning your old cupboards in the place where you grew up, finding old pictures, mails, etc.

I just received a mail from someone asking me if this was something that I have written. It was "mon memoire du DEA" from 1999 in the department of Hypermedia in Paris 8! I have not even put my name on it, but somehow this person was able to find it and associate it to me. Best of all is that she still found it useful for her studies, she wanted to cite it in her own dissertation on language teaching and new technologies, but since the text did not have the author nor the publication date, she found me. You never know, do you now..

So here it:
Riina Vuorikari
ENSEIGNEMENT ET APPRENTISSAGE
LE CAS DES LANGUES ÉTRANGÈRES
EFFETS DES TECHNOLOGIES DE L’INFORMATION ET DE LA COMMUNICATION
Date de parution: septembre 1999
Lieu de publication: L'Universite de Paris 8, Saint-Denis, France.

Notice the way that the title was written, no commas but different lines. Pretty arty, ha? It was mostly influenced by, hm, my really eccentric pormotor, J.Feat. So, I went back to my Yahoo! mail that I used already back then to check the mails between us when studying. Man, he was somewhat strange, but who would not be in Paris 8!

I always joked that the hardest task in it all was to get out of there with a diploma, what a mess. But Fun. A good place to hang out. Check out what the French version of wikipedia says about it. The English version is lame, it's hard to capture that feeling of "papa cools", all the old hippies from the late sixties who had installed themselves there ever since the Youth revolution of 1968. Every day there was (I bet still is) a student "manif", a little protest or signing a petition on this or that. When I read parts of my dissertation I noticed how that radicalism had snuck in..

D’un côté Internet offre " un accès libre au monde ", il est " international, pluriculturel et multilingue ", mais ceci est une image idéalisée d’Internet. D’un autre côté Internet est vu comme un média conditionné par McWorldet par les concepts d’américanisme à l’échelle mondiale. L’homogénéisation culturelle et le commerce électronique comptent sur l’idée que la consommation devient l’unique activité humaine qui uniforme le goût des consommateurs.

..and at the end about the future perspectives:

Une autre piste de recherche sera l’industrialisation de l’enseignement. Les questions soulevées par ces tendances sont multidimensionnelles : Est-ce que l’interaction humaine pourra être remplacée ? Est-ce que les enseignants qualifiés seront remplacés par les moins qualifiés une fois que le contenu du cours est mis en place sur Internet ou sur le cédérom ? Qui aura la propriété des contenus de ces cours qui sont devenus des produits à exploiter, le professeur ou l’administration de la faculté ? Qui aura l’intérêt à vendre ces contenus, qui aura les droits d’auteur et qui va gagner l’argent ?

Outch. Then again, there is lots of good stuff too. I love the translation of knowledge network in to "le tissu de savoir" or calling the whole Internet as " tissu social virtuel"! What a foresight! I remember sitting with my supervisor in the Montmartre graveyard and he was explaining that instead of talking about the Internet, I could use the term " tissu social virtuel", like a web woven in a tissue where the threads are all intervened, to illustrate the use of the Internet. pretty funny in its own way..

This is my opening line:

Internet est souvent associé au concept d’interactivité. Est-il possible d’exploiter cette propriété pour mettre en œuvre des techniques spécifiques pour l’enseignement de langues vivantes étrangères ? Au contraire de l’enfant qui apprend sa langue maternelle dans un environnement naturellement interactif - et de façon permanente - l’étudiant suit régulièrement un cours où l’immersion linguistique est artificielle et de courte durée. Jusqu’à quel point est-il possible de reconstituer, à partir d’un tissu social virtuel (=Internet) les conditions idéales d’apprentissage, particulièrement l’apprentissage des langues étrangères ?

I remember when I finally was writing my dissertation, my promotor was merciless. He really cracked the whip on me. But he knew how far to push, and at the end also I was really pleased with the result. After all, it was mention bien. When I thanked him for this, he said " Please, don't you give no "thank you" -- after all, it's my job...". That's a true educator! But hey, he could have taken some credit for it.

A funny thing was that he's English was perfect, but I only learned about that after we were done with all the writing. He was harsh on my French too. When I had already handed in the first version of my dissertation, he congratulated me on it. But half way down on his mail he says that in its current version no one can read it without lots of difficulties :

Toutefois, tu ne recevras ton diplôme que si tu déposes plus tard un nouvel exemplaire, corrigé de toutes les fautes de français -- il faut comprendre que cet exemplaire est destiné à la bibliothèque, et que, tel qu'il est, personne ne peux le lire sans une grande fatigue...

The final discussions that we had before my defense took place in Montmartre. We once met in that same café where they shot scenes for Amelie, or at the graveyard. I was quite surprised to see them in the movie, I remember.

It was also fun now to check out on some professors from Paris 8. I found Jean-Pierre Balpe who at the time was the head of the department of Hypermedia. Jean Clement who taught us a course on hypermedia. Imad Saleh, one of my "rapporteurs", is now the head of Laboratoire Paragraphe, that's the nickname of the deparment (so French, I love it!). Or Jean-Lois Weissberg - I never understood anything about his lectures, he talked about "telepercence" and such. I only remember that soulful intellectual Greek philosophy student who engaged in discussions with him about things that I could not follow. After all, they all carried the legacy of Pierre Levy who had just left the department year earlier.

I tried to dig out on my Lycos account where this stuff now resides, but I could not find my password and the system does not recognise my username :( But I found some other old stuff, like the index page of the site that I made at that time for Finnish students in Paris, Suomalainen Osakunta, my cool looking CV and the side of akseli that Pia, "Puumatyttö", teki suomalaisille taiteilijoille ja jota autoin jossain vaiheessa. Ihanaa!

Well there, that was a nice moment of memories from Paris and end of the Millennium.

Can Social Information save teachers' time when choosing interesting learning resources?

2008-08-07T16:59:00.026+02:00

One of my research questions is aimed at understanding what so called Social Information can do to help teachers to choose the right learning resources from a seemingly overwhelming collection. By Social Information I mean information about previous users' interactions with the resource. I am mainly interested in explicit annotations like ratings and tags, and more implicit ones like bookmarks.

As I'm interested in the use of resources that come from different countries than users do, I think Social Information (SI) should display not only annotations, but also information from where the user comes from.

One thing that I hypothesise is that among other things, Social Information, when associated with conventional metadata about learning resources, can make the decision making process faster for teachers when, for example, looking at the search result list. As a multilingual context in a repository can result in metadata that is in different languages, it could be speculated that Social Information indicating the origin of the users who have previously annotated the resource, could help the other users to make up their mind (see the image for an example).

We were interested in two different aspects:

Does the appearance of Social Information make the decision making process any faster?
Does the appearance of Social Information make the users choose more resources?

Method

We had 25 users from five different European countries. These teachers are primary and secondary teachers in science, language learning and ICTs in Finland, Estonia, Hungary, Belgium and Italy. xx of them are females and xxmales. xx participant is under 30 years old, xx are under 40 years, xx under 50 years, xx under 60 years old.

They have been part of the MELT project since Summer 2007. In March 2008 they were invited to create a profile on the MELT portal, where they are able to access multilingual learning resources for different topical areas.

We designed an experiment where teachers were shown two different imitations of search results list with learning resources and their associated metadata. One of the lists showed what we call the conventional metadata, such as title, url, language of the resource, a short description, subject area, type of content and its target audience. Here is an example.

The other list had the same metadata, but we also added the Social Information from the previous users. This could be the tags in their original language, the number of times bookmarked (favourites) and the ratings. Also, for bookmarks we would mention from which country the users come from. As example of this was shown above, the first image in this post.

We had 48 learning resources that came from different countries and were in different languages. About half of them were in English and other half in other languages, this also seems to reflect the division of the resources that users have bookmarked on the portal. The resources were about language learning, primary education, ICTs and science material, those were the areas of the teachers. I'll prepare better information about this later.

We had 12 learning resources on a page imitating a list of search results that user could get on a repository. In total, there were 4 such pages for each user, we call them sets. Every second set had conventional matadata, and every other had additionally also Social Information as indicated above.

At the beginning of the each set the participants were asked to write their names and the time when they started with the set of 12 resources. At the end, when they submitted their results, the system recorded a time. To answer to our first question we were interested in how much time do teachers spent to evaluate the appropriateness of 12 resources for them.

The teachers were asked to look at the metadata of the resource and the resource itself if interesting, and were asked one single question: "Would you use this resources, or parts of it, in your teaching in next Fall?" They answered on a scale 1 to 5, 1 being "I don't teach the topic", 2= No, 3= Maybe not, 4= Maybe and 5= Yes. Looking at the number of resources that users choose in their topical areas would give us indication of whether resources that have more Social Information related to them were more often chosen than the onces without.

Because of the low number of participants (n=25) we decided upon a within-subject design for this experiment. This is the one where the same group of subjects served in both treatments, i.e. they received both the material with conventional metadata and with social information.

Moreover, we had the participants in two different "groups". Group 1 had 12 participants and Group 2 had 13. Group 1 started first with a set with conventional metadata and Group2 with a set of resources that had Social Information added to it. When analysing the results, we found that one user in Group 2 had consistently added incorrect times. We excluded these times from the counts for time spent per set, leaving 12 participants in each. Moreover, in both set there were a few cases where the start time was forgotten.

Results

Descriptive statistics

We had 1129 responses to our questions, which means 71 responses were left blank. In 53% of the cases the users had answered that they do not teach the topic, which means that they deemed the resources not suitable for the topical area that they were teaching. 25.6% of the users found resources that they said that they would use (yes or maybe yes), whereas 21.6% of the resources were not found of use in the upcoming school year (not, maybe not). The mean for the responses was 2.23 (Min=1, Max=4), standard deviation was 1.138.

Q1: Does the appearance of Social Information make the decision making process any faster? Time spent on 12 resources (i.e. set)

On the average, users spent a bit more time on the sets that did not contain Social Information. The average to review a set of 12 resources with conventional metadata was 9 minutes and 8 minutes with Social Information.

I do not know yet whether this is a significant difference (my SPSS license ran out), but one could assume it is at least a sign of good news for Social Information. We can imagine that users go through a lot of resources when browsing a learning resources repository (I currently do not have the logs about the number of resources that users review per session, but I will produce them). So if you think of small cycles and multiply that number with, say 1o times, you could come up to some significant time savings when Social Information is made available to speed the decision making process.

Individual differences

Still looking at the average times spent, we can see that there were many individual differences. In the chart below the blue lines show the amount of time that participants spent with conventional metadata and the red one with Social Information added to it. You can see that for some users one metadata setting seems like a faster way, but anyhow, the lines follow one another pretty closely, apart from some odd-balls (like user 23). You can also see that there seem to be a wide variety of personal ways, some users scrutinise resources with a great care (user 7 and 8), whereas some go through them very fast (user 16 and 17).

I'd like to mention that here it does not matter that some of the resources are not in the competence area of the participants. We focus purely on the time that they spent going through pages and making decisions whether some of the resources are useful for them in the upcoming school year or not. However, this becomes crucial to answer to our second question:

Q2: Does the appearance of Social Information make the users choose more resources?

Table below presents the results when I looked at the amount of resources chosen per set. There was 4 different sets and each contained 12 learning resources in different languages. The two different treatments meant that teachers reviewed 2 sets with Social Information available, and two sets without. As teachers were from different tpical backgrounds, I excluded the responses from users who said that they do not teach the topic of the given resource. In table below you can see the percentile of positive responses (maybe use, use).

It appears that consistently teachers chose more resources when the Social Information was not available. This is contrary to what I expected. I have not calculated the significance of these results, but the differences do look big. In some cases, like in the 2nd set, even about 15% in favour of no Social Information available.

In a way, maybe the appearance of SI makes the teachers more careful or critical to choose the resources?

What is needed now is a follow up study at the end of this school term to check whether these teachers actually used the resources in their teaching. Or, I could check if they have bookmarked these resources on the MELT portal. They know the resources are available there. MORE to follow...

Asume nothing. Plan for everything.

2008-08-06T20:12:00.012+02:00

I noticed this ad at the train station when I was returning from my weekend sailing trip in Friesland, Nl. It kinda captured the mood of the first half of this year, or better, it kind of captured the lesson I wish I have learned during that time. Assume nothing. Plan for everything. And then plan some more.

I think I sometimes assume too much and take things for granted. I think that everyone else is "with me" in the same thing or on a same (mental) trip, and I do not bother to explain the plot, set out my major expectations and go through all the details, etc. Then at one point, usually when it is already too late, I realise that this is not what I assumed it was.

Last weekend this dangerous thinking left me in the water when we accidentally capsized out small sail boat (seen in the pic)...and, there was no plan for what to do then. (one can be found here)

Clearly once again I had been trapped with my not so productive way of thinking. We had been sailing on a 6-meter open Falk boat already for a day with a crew of 4. My pal was skippering and we were 3 others with some/good sailing experience. I got in some really good sailing :) I managed to get us through a pretty rough channel with a strong head wind. We had to tack at least ten times to get to the other side, but I managed it all, we did not loose too much speed in turns and I made the guys jumping from side to another to give the weight to keep the boat from flipping. I realised that Falks are darn sensitive to your weight, it really makes a difference on which side you lean on.

So comes along the next day and we are sort of returning. We have one reef down because of the heavy winds. The skipper was steering, I was taking pics in the head, and the two others were somewhere in the middle. The next thing I hear is that one of us slips on the floor and tumbles on the side. Mind you, these are small boats, so by the time I turn to watch, he's trying to grab onto something and I'm sure he's going to fall off the boat.

With the sudden shift of his bodyweight and the heavy gust from the side, the next thing I observe is that he is not going to fall off the boat, but the boat is going to fall with him. I see our chart flying, the bottles and the gas jar landing in the water, and soon everyone else follows. Since I had yanked myself firmly in the head to take pictures, I found myself standing on the low side wall of the boat that now was horizontal in the water and slowly sinking down with more and more water getting in.

I was not sure what to do. Would it be better to stay in the boat or leave it. After a thought of the boat turning turtle, and seeing myself being tangled in the ropes under the boat, I decent in the water and start collecting our belonging. Soon enough I realise it's heavy to swim with cloths on and I thought "heck with our empty bottles of water and my fake crocks that I tried to save", and swam to the others who were on the other side of the boat. The skipper lifted me on the keel. I was like "so what now", but I did not get any instructions. I had not prepared for this and was not sure what was the next thing to do. Our weight on the keel did not have any effect on righting the boat (which was the goal that the skipper pursued).

Then, that's like seconds later, there was the rescue troop behind us. I could not figure how they were there so soon, but later they told me it was because of the regatta that was going on and they saw us capsize. They put a rope on our mast, and with a help of a second boat, we got it back right up. The mast was all muddy, the wind was pushing the boat so much that instead of turning all the way around, it got stuck in the low waters. No wonder our weight did not do a thing to right the boat.

We were eventually pulled back to the harbor by these Dutch gentlemen of the sea and left sorting out our dripping wet packages, cursing about wet mobile phones and the bill to pay for some lost gear from the boat.

In retrospect it feels just like what happened with the PhD where I got "snowflake*d" (i.e. my adviser at the time kicked me out of the programme) . What happened was just like with sailing, one day you feel like you can do all the good moves, and then the next is that you find yourself swimming in the water and wondering "this is not how I assumed it would end". The worst being that I was not prepared for it, I had not thought of what would be the plan B or C for that matter.

Reflecting on all this makes me see a reoccurring pattern that I'd better avoid in the future. It's clear that accidents will always happen and mistakes are made, but there are also precautions that can be taken to prevent them. Those are done by careful planning and someone taking clear leadership on things. Then the other thing is that when accidents happen or mistakes are made, one needs to know what to do next. Be prepared for them. And then some more.

Being comfortable having other people taking the lead is good, good leaders need good followers, like Dan is known to have said once. Perhaps it would be time for me to think more about taking leadership on things and try to influence, at least on my own life, that I'm prepared for everything. That is one characteristics that I expect from a skipper of a boat, and for that matter, it probably should go for any other areas too.

Measures for cross-border actions with Tags and Resources

2008-07-28T09:22:00.010+02:00

If I break down the triple of {user, tag(s), resource} I can study the three things separately

User - resource
User - tags

Resource - users
Resource - tags

Tag - resource
Tag - users

And as I am especially interested in the cross-border actions, I would study the cases where:

User _country ≠ Resource _country

In this case I am interested in studying users collections of bookmarked resources, especially establishing the facts based on which country the resources are originated from. Using the cross-border metrics I can take a snapshot of the resources and calculate a cross-border resources value for the use.

E.g. User _Finland has bookmarked Resource1 _Poland, Resource2 _Spain and Resource3_Finland
This would make a User _Finland to have a resource profile Poland 33%, Spain 33% and Finland 33%
In this case, as the user is from Finland, the cross-border profile would be 66% which would most likely have a value of .66, if we imagine that the cross-border value is between 0 and 1.

So what, you say. It makes a difference, I say.

This allows me to categorise this user into cross-border user of resources. I assume
that users have differences in their inclination of using resources that come from different countries, some use them a lot others do not want to bother with them.
So this metric allows me to study who does what and thus better understand our user-base.
On the long run this of course will make it easier to recommend resources to users, as we
already know that in their profile it shows that they are inclined to use cross-border resources.

Resource _country≠ User _country
This allows to me to look at the thing from a different point of view. Here, I am interested in establishing a profile for a resource. It appears that some resources are used a lot by people from different countries, whereas others are used predominantly by users from the same country than the resource itself is from.

E.g. Resource _Finland has been bookmarked by User1 _Poland, User2 _Spain and User3Finland

This makes the Resource_Finland to have a profile Poland 33%, Spain 33% and Finland 33%.
In this case, as the resources is from Finland, the cross-border profile would be 66% of users, which would most likely have a value of .66, if we imagine that the cross-border value is between 0 and 1.

So what, you ask again. I think that it's cool, because then I can quickly and in an automated way calculate which of my resources have a high potent to cross borders easily. First of all, this will help me study whether there are some characteristics that make these resources to cross-borders.

Second, we can use this information to make filter out the resources that we think cross borders easily. This could be cool for example on our portal, we could flag out these resources for users, and furthermore, we could give these resources a priority when other repositories are harvesting or searching us in a federated manner.

Resource _country Tag_language
It'll also be interesting to create profiles for resources based on tags in different languages. For tag, we do not trace the country of origin, rather just the language. So in this case I'm interested in looking at resource profile on tags.

E.g. Resource _Finland has been added a Tag1 _Polish, Tag2 _Spanish and Tag3 _Finnish

This makes the Resource_Finland to have a tag profile Polish 33%, Spanish 33% and Finnish 33%.
In this case, as the resources is from Finland, the cross-border tag profile would be 66% of users, which would most likely have a value of .66, as above.

This is also an indication that the resource has a potent to cross borders. Tags in different language might yield some interesting information on how this learning resource could be used in a new context. In this example the resource was created in Finland, so one could assume that it has some underlying ingredients that make it suitable for Finnish curriculum. On the other hand, the fact that users have added tags in Polish and Spanish too might indicate that this resource is also useful for teachers in those countries.

Here an interesting case seem to emerge for topics like Language learning, say, English as Second Language (ESL). Language learning and teaching resources seem to be easily reusable in another language context. Interestingly, though, we've seen that in these cases teachers tend to tag them in the language in question.
E.g. User _Finland has added a Tag _English for ESL Resource _Poland

Tag _language Resource _countryWe can also look at the things from tags perspective.

E.g. Tag _Finnish has been added to Resource1_Poland, Resource2 _Spain and Resource3 _Finland

This makes the Tag_Finnish to have a resource profile Polish 33%, Spanish 33% and Finnish 33%.
In this case, as the resources is from Finland, the cross-border tag profile would be 66% of users, which would most likely have a value of .66, as above

This allows us to observe cases where a tag is related to learning resources that most likely share some thematic resemblance. It could be for example Science resources from different countries that Finnish teachers have collected. In this case we also can find evidence that these resources were adaptable to Finnish curriculum despite the fact that they come from other countries.

Tag _language ≠ User _countryOn the other hand, we also find tags that have been used by users from different countries. These are the tags that we have previously identified as "travel well" tags. They have some interesting properties that make them easily understandable without translations, e.g. names (people, country, place), acronyms, common terms (web2.0).

By looking at the connection between Tag _language and User _country we can possibly identify such tags. The other common case for this seems to be that these people have tagged the resource in English. In any case, if many people have done that, we can identify these terms and manually analyse them. The hypothesis is that they either are "travel well" tags or then they are some super popular tags that could also count high on tag non-obviousness metric by Farooq et l (2007).

User _country - Tag _languageLastly, just to enumerate the cases, we also have the relation User _country and Tag _language. This can be used to study user's personal tagging behaviour. In the previous study in Calibrate we found that on average users tag in their mother tongue and in English (75% to 25%). It seems though that things look different in MELT, where teachers are tagging more in English.

We are not sure whether these are personal preferences or the influence of social awareness, as in MELT tags are made readily available to others through a tag cloud, whereas in Calibrate they were only used for personal knowledge management reasons.

In any case, this relation allows us to measure individual differences between users and thus understand our user-base and possible user scenarios better.
_{What next? I will make a case study to apply these measures to MELT tags that we've got in the system so far}Dataset:

Learning resources: 199
Users: 40 (From Fi, Hu, Et, Be, At, It)
Tags:

572 distinct,
969 applied tags

75% of tags were used only once
25% of tags were used more than once

Tags and SNA measures

2008-07-28T07:08:00.005+02:00

Studies on tags commonly have the triple of {user, tag(s), item} as a unit of study. That's also what I'm interested in, especially in those underlying structures that build relationships between users, tags and items. Some apply Social Network Analysis to study, for example the centrality measures of the network.

A run-down of SNA measures from Wikipedia

Betweenness

Degree an individual lies between other individuals in the network; the extent to which a node is directly connected only to those other nodes that are not directly connected to each other; an intermediary; liaisons; bridges. Therefore, it's the number of people who a person is connecting indirectly through their direct links.

(Somewhere else:The betweenness measurement indicates a node or nodes that connect clusters of nodes. Nodes that have hight betweenness have high influence over what information flows in the network.)

Closeness: The degree an individual is near all other individuals in a network (directly or indirectly). It reflects the ability to access information through the "grapevine" of network members. Thus, closeness is the inverse of the sum of the shortest distances between each individual and every other person in the network.

(Degree) centrality: The count of the number of ties to other actors in the network. See also degree (graph theory).

Flow betweenness centrality: The degree that a node contributes to sum of maximum flow between all pairs of nodes (not that node).

Eigenvector centrality: a measure of the importance of a node in a network. It assigns relative scores to all nodes in the network based on the principle that connections to nodes having a high score contribute more to the score of the node in question.

Centralization: The difference between the n of links for each node divided by maximum possible sum of differences. A centralized network will have many of its links dispersed around one or a few nodes, while a decentralized network is one in which there is little variation between the n of links each node possesses

Clustering coefficient: A measure of the likelihood that two associates of a node are associates themselves. A higher clustering coefficient indicates a greater 'cliquishness'.

Cohesion: The degree to which actors are connected directly to each other by cohesive bonds. Groups are identified as ‘cliques’ if every actor is directly tied to every other actor, ‘social circles’ if there is less stringency of direct contact, which is imprecise, or as structurally cohesive blocks if precision is wanted.

(Individual-level) density: the degree a respondent's ties know one another/ proportion of ties among an individual's nominees. Network or global-level density is the proportion of ties in a network relative to the total number possible (sparse versus dense networks).

Path Length: The distances between pairs of nodes in the network. Average path-length is the average of these distances between all pairs of nodes.

Radiality: Degree an individual’s network reaches out into the network and provides novel information and influence

Reach: The degree any member of a network can reach other members of the network.

Structural cohesion: The minimum number of members who, if removed from a group, would disconnect the group.^[15]

Structural equivalence: Refers to the extent to which actors have a common set of linkages to other actors in the system. The actors don’t need to have any ties to each other to be structurally equivalent.

Structural hole: Static holes that can be strategically filled by connecting one or more links to link together other points. Linked to ideas of social capital: if you link to two people who are not linked you can control their communication.

What made me think of this now was that I read this mini study Using Social Network Analysis to Highlight an Emerging Online Community of Practice. Anthony Cocciolo, Hui Soo Chae, Gary Natriello, Teachers College, Columbia University

The method used made me tick. They used

..System Theory to define the uploading and downloading of materials as "communicative acts", the users of the system were the "actors" and the cululative communicative exchanges as "interactions" (Buckley, 1967). .. this particular systems arrangement is useful because it provides a readily available metric for assessing actors' interactions within a network.

I think it might be interesting to think how this could be used to study the underlying networks with tags.

The most comprehensive reference is: Wasserman, Stanley, & Faust, Katherine. (1994). Social Networks Analysis: Methods and Applications. Cambridge: Cambridge University Press. A short, clear basic summary is in Krebs, Valdis. (2000). "The Social Life of Routers." Internet Protocol Journal, 3 (December): 14-25.

Notes: Tagging tagging. Analysing user keywords in scientific bibliography management systems

2008-07-10T14:47:00.007+02:00

An interesting paper on JoDI about tagging in bibliography management system.
Tagging tagging. Analysing user keywords in scientific bibliography management systems
Christian Wolff, Markus Heckner, Susanne Mühlbacher
Journal of Digital Information, Vol 9, No 27 (2008)

Some outcomes:

a category model for tags in a scientific bibliography management scenario. This model covers linguistic features, the relation between tags and the text of the tagged resources, as well as functional and semantic aspects of social tags.

Here is an image of the model that I copied from the paper:

This is actually a really cool model for tags. I've been so far using three categories from MovieLens and Golder (2006)/Huberman (2005) studies; Factual, subjective and personal. I've noticed, though, that I've added many sub-categories for the Factual ones.

Like in this model, I've discovered very similar types in tags. Especially the "Functional Category Model" is interesting : it has 2 sub-classes:

subject related (e.g. resource related and content related) and
non-subject related, personal tags (e.g. affective, time and task related, tag avoidance=no tags).

Other things:

The ”typical tag” is a single-word noun, taken from the title of the respective article
(identical or variation), thus directly related to the respective subject.

Yep, we have many of these too! When I talk about these I refer to the non-obviousness metric from Farooq et al. (2007).

In contrast to previous studies the number of non-subject related tags remains rather low in the scientific data we observed and the full potential of tagging systems to describe qualities or aspects of resources does not seem to be used. But the absence of tags like cool, interesting, to_read does not mean that users who tagged the resource do not think it is cool, of interest or worthy of reading, but simply that the users did not express their ideas they may have or may not have about the resource.

This is interesting too. I think each audience tags differently. Our target audience are teachers, about 35-55 years old. They do not seem to go around tagging learning resources with tags like cool, etc.

Compared to author keywords, social tags tend to introduce less and simpler con-
cepts: Altogether, only one third of the social tags matched with (the far more numerous) authors’ keywords. Moreover, tags tend to be more general and users tag their articles more general and with less words than authors.

This is also interesting. There are some studies that have compared the tags and expert indexer keywords and have found even less overlap, if I remember right.

I love this one, it is so much the case:

Additionally, it shows that the respective system environment, e.g. tag suggestions, has a major influence on the tagging behaviour in terms of spelling errors, tag usage and creation of a specific tagging languages. This extends the number of the main influential factors on tagging behaviour being personal tendency and community influence through the additional component system influence.

They also flag out as an interesting study area the comparative studies across tagging platforms. I've looked at different tagging systems for educational resources a bit. This version is an old one, but I post it anyway:

Vuorikari, R., Poldoja, H. (submitted). Comparing tagging and its purposes across learning resource repositories. pdf

Teachers as Netpromotors of digital content

2008-07-09T22:17:00.011+02:00

I made a survey with 28 teachers from different European countries on multilingual learning resources. You can find those 28 resources from this list. Our portal has a lot of multilingual resources that come from a variety of Ministries of Education in Europe.

But - we do not know for sure whether teachers find resources useful that come from different countries than they do, and that are in different languages than they speak. Hence my little survey. You can read more details here.

We only considered responses from teachers who came from different countries than the 18 resources did that we had in our survey. Quick round of results:

43% of respondents found resources, which came from a different country than they did, of use for preparation purposes.
41% of respondents found resources, which came from a different country than they did, of use for teaching purposes.
65% of respondents said that they would share these resources, or parts of them, with their colleagues and friends.
Even 35% of respondents, who said they did not have expertise in the given subject area, thought that they would share the resource with their colleagues

These were the results on a scale 1-5 (n=254)

It made me think that:

a) If teachers use multilingual or foreign language resources, they most likely use them both for preparatory purposes and for teaching purposes. We do not know, though, whether they would use the resource in their teaching themselves or let pupils interact with this resource.

b) Teachers are good filters. More teachers said that they would be willing to share resources with their colleagues than actually use them themselves. It might be that this happens with a resource, which they think is interesting, but does not match to their curriculum goals for the year. They might say, "Hey, my colleague would love this, I'll send it to her!" This is the basic mechanism of viral marketing, how can we leverage this on a learning portal?

c) "Would you like to share it with your colleagues" is one of the key questions when studying customer satisfaction and loyalty, topic that we in learning repositories often neglect. If teachers are happy users, or if teachers find good material on the portal, they can become promoters of those resources. This might be very important especially when we deal with resources that are in multiple languages, because sometimes it is hard to discovery those resources.

If we take the teachers in the survey, we could calculate the Net Promoter Score by subtracting the % Detractors (e.g. the ones in my survey who rated this 1 or 2 on the scale 1-5) from the % Promoters (e.g. the ones in my survey who rated this 4-5).

Take the case for sharing: it would be 65% -22% =43%. That is a pretty good net promoter score, most companies have it around 5 to 10%, and it is very unusual to have it above 50%.

This can indicate that teachers are willing to put their credibility on the line by recommending a resource that comes from a different country than they do to a friend!

Now, I just have to think of the best way to do this ;)

A draft idea for a paper: A case study on teachers' use of social tagging tools to create collections of resources - and how to consolidate them?

2008-07-09T19:08:00.012+02:00

UPDATE: the submitted paper, comments welcome!

This paper explores how a group of pilot teachers (16) create collections of digital learning resources using tagging tools. We study two different tools: an educational portal (MELT) and del.icio.us. We first look at the characteristics of these collections (number of resources, languages of resources, number of tags used, etc), and then propose a way to display the resources and tags from del.icio.us on the learning portal (MELT) using Attention Profing Markup Language (APML). This allows a higher level of integration between a learning portal and an external social tagging service like del.icio.us, and thus enhances the wider variety of digital learning resources to be discovered.

Method

We selected 16 pilot teachers to be subjects of this study from the MELT project. These teachers have both an account on the MELT portal and on the delicious bookmarking service. These teachers are primary and secondary teachers in science, language learning and ICTs in Finland, Estonia, Hungary and Belgium. 7 of them are females and 10 males. One participant is under 30 years old, 8 are under 40 years, 5 under 50 years, 3 under 60 years old.

They have been part of the MELT project since Summer 2007, when they were first introduced to delicious during a summer school. In March 2008 they were also invited to create a profile on the MELT portal, where they were able to access multilingual learning resources for different topical areas.

From the MELT portal we know the detailed profiles of these teachers: their names, topics they teach, country where they teach and languages they speak. Moreover, we have information regarding the learning resources that they have bookmarked using the portal. This includes the information about the resource itself and the tags applied. We additionally have asked for their delicious username to be part of this small study.

From delicious, using the html service, we were able to download the 100 last bookmarks and tags that these teachers had posted on delicious. We also took all the data regarding the tags and people these users had in their network. Lastly, we recorded the number of posts each teacher had on their account.

We collected the following data for our selected 16 users:

Additionally, the delicious data contained the following information regarding the networks. Two people had chosen to keep their networks private:

Number of distinct people in the networks: 104
Number of people in the networks: 270

Results
Discussion

References

del.icio.us API and other not so successful trials

2008-07-09T17:51:00.004+02:00

I am getting somewhat disappointed in some of these web 2.0 "things". Take, for example, the delicious API.

I wanted to download the posts by a number of ppl in my network to study what the hell are they doing. The API allows you to download all your posts in a neat xml format. That's cool, I thought, let me just do this to 20 of my buddies, and I can study better how teachers are bookmarking - especially how are they bookmarking websites that are not from their own countries or in their own languages (e.g. cross-border use).

The delicious API only allows you to get 30 latests posts from people that you do not know the password of. wtf? The same if you try to get them through RSS feeds, you only get 30. Then, there is the html code that you can use, but it also allows you to get only 100 posts. What about the rest, those 999 posts that I want? That stuff is so badly documented on the site that it's very annoying. Why not just be frank about it and say this is how things are?

I do not understand why to limit the API, RSS or html code when all that stuff is freely viewable anyway. So I tried using wget to suck that stuff out, but there is also something fishy and I can never get past 100 posts. So, I guess that just makes me to limit my study to a sample of 100 posts per user. Easy.

The other thing that I've been sightly disappointed with lately is APML and a number of tool that they make available for you to track your online profile, like engagd.com or tagurself.com.

You know what, the idea is great, but those tools/widgets suck, and they are so badly documented that it makes you just wanna cry. I've tried like 3 times in engagd to make my APML profile of 2 different feeds, and it never works. The tagurself cannot even load the example from the url that they have themselves posted as an example. wtf?

Moreover, the Yahoo! pipes are also somewhat strange, they never actually seem to post what they should. I put this example in one of my lasts post and it hardly never loads. Not so fun.

Hmm...I guess if more people used all these 2.0 tools, and not only talked about their potentially revolutionary usage by non savvy web-users, we could face the fact that the user-created web is far from being so revolutionary and does not empower users like me. Instead, I'd like to see those folks walk that talk, sit down on their asses and finally get past the BETA versions of their tools to actually make them work properly. Dude, cannot wait to get rid of all the BETA versions on the web.

Checking tagurself

2008-07-08T15:29:00.001+02:00

Pipe trial for delicious

2008-06-13T12:24:00.000+02:00

This is it! Resources that cross boundaries

2008-06-02T20:19:00.003+02:00

Ok, I think this graph is the coolest kid in the blog!!

What you can see here are the communities of users by mother tongue (nodes) and the edges are the resources that these users have added to their collections.

This is a great visualisation of communities of practice. What you can see here at a glimpse is that the learning resources that these users have added to their collections, are very much community oriented, in this divided by languages.

I sometimes frame my research question as the following:

Does a multi-lingual and multi-cultural learning resources portal rather act as one system divided into different language or country groups, or is it more like one monolingual system with its own sub-groups and communities of practice (think of a system like delicious) that cross the language and cultural borders?

This visualisation seems to point more to the first one (this REALLY needs to be further investigated!!), it seems that users are divided into groups by mother tongue. Why I say so is that you cannot see many resources that are shared among the groups.

To play around with this by yourself, make sure that you click on the arrow head down at the menu bar. This allows you to see in which directions the links go. They often time just go to one direction.

There are some resource that indicate communities of interests between countries. For example, in this image, we can see that there are some resources that are shared by both Estonian and Lithuanians. One of them is highlighted in orange.

These are the interesting resources as they cross between boundaries. The more I think of it, the more I'm convinced that you cannot call these call boundary objects (see my previous post). If I got the boundary object right, they are the objects that help these two groups to talk to one another, because they do not share the same language or jargon. But in this case, I think it's the contrary, these people share so much the same, that they can even share resources in Russian (of course being ex-Soviet countries, Russian is a common knowledge).

Anyway, even if the rather disappointing news were that users on an international portal seem to stick to one another based on their mother tongue rather than common educational interests, the good news is that I believe that through making more social cues and traces available to them, they would actually start exploring the resources in other languages and other areas.

And besides, who says that my data here really actually displays this community correctly!? This is based only on the common resources that users have put to their collections. Actually, LeMill is more of an authoring environment, so maybe a better way to study this community would be through collaborative authoring of learning resources? Or something else, like common search terms or tags that are used.

So, take this exploratory description of this data set with a little bit of skepticism!

In what languages are the resources that end-up in collections?

2008-06-02T19:57:00.006+02:00

Well then, I guess that will be a no-brainer...

In this visualisation, you can see the languages of resources (e.g. English) as nodes and the languages of users as edges (e.g. en, de..).

If you click, for example, on English, lot of edges are highlighted. Those are the mother tongues of users who have bookmarked these resources. After little bit of playing, you'll find that English resources, and the ones with no languages, seem to be most popular with users.

However, it is cool to see that resources in other languages also end up in users' collections. Here, for example, you can see that Czech (sorry for misspelling) are used also by users with Polish and Lithuanian as mother tongue.

More analyses are needed to give you any numbers, but this already is an interesting insight.

Resources country of origin and user mother tongue

2008-06-02T19:24:00.003+02:00

This visualisation shows the links between the country, where the resources in the collections were created in, and the mother tongue of the users who had added them in their collections. You can explore the diagram by yourself.

This image here shows how, for example, resources created in Finland (the orange node in the network) have ended up in collections of users who speak Hungarian, Estonian, Lithuanian, etc. as their mother tongue.

Note that this graph does not make any assumption of the language in which these resources are in! If I'm right in my guess, most of these resources were in English, not in Finnish..

But anyhow, I find that as a demonstration that these resources can cross borders of some kind. In this case, a Finn has created the resource. It can be just a very little hint available in the design of the resource that it was a Finn, but still some of the underlying pedagogical assumptions or some hints of Finnish curriculum might be embedded in these resources. Nevertheless, or thanks to that, the resources created in Finland seem like a hit (they are in 8 different language groups).

Ok, to me more truthfully, I think this is because LeMill was create in Finland that many of the Finnish resources are shared.

About networks of resources and users

2008-06-02T19:07:00.002+02:00

This visualisation is to explore the networks of users that form between resources that are shared in collections. I think this is one of the most interesting visualisations of the dataset, and the one that inspires me the most.

Same as before, click to interact within the image, or if you click on the title on top of the image, you can get the network in a bigger window.

What's there? It's a network diagram where the nodes represent users (user id number) and the edges are the names of learning resources that these users have saved in their collections.

You can zoom into the diagram and explore it. Same as with the previous post, we can see that lots of the resources that users have put in their collections are not shared with other users. These are the singletons that are not part of the common network here.

Then, there are some star like structures that can be found. Like this one. Here the resource highlighted is something that both users (user 59 and 155) had added into their collection.

What I think, I would almost bet on, is that if these users were made aware that they share this resource in their collections, they would be interested in looking at what other resources are in the other person's collection. In this case the user 59 could be interested in looking at the collection of the user 155 has put in her collection.

This basically would be the idea of making underlying social networks visible in a repository to allow social navigation of like-minded users collections. Or, if you wish, a recommender could take advantage of these underlying connections as well. For the recommender, though, the data is very sparse, as can be seen from the visualisation. For that reason, I think we first should explore social navigation possibilities, and then launch for recommenders, when we get more data.

These resources that connect users, or in some cases (hopefully one day) even communities together, are valuable stuff. I have previously referred to this as one way to identify learning resources that cross borders easily. In this case, the two communities could be speaking different languages or be from different countries.

Some suggested that these objects could be also boundary objects. I cannot get my hands on the original article now (frustration of working from home!), so I am referencing some others that reference it:

Star (1989) and Star and Griesemer (1989), on the other hand, are concerned with the distribution of artefacts across communities. Boundary objects are artefacts used by communities: they cross the boundaries between communities and retain their structure, but are interpreted differently by them. The notion of boundary objects was developed by Star (1989) and Star and Griesemer (1989) as a way to explain co-ordination work between communities.

In a larger sense, maybe some of them could be boundary objects. I will need to think about this more..

Anyway, here is another little visulaisation that is actually an overview of the resources that users have saved in their collections. You can visualise it in many ways, you the ordering function on the top.

Star, S. L. 1989. The structure of ill-structured solutions: boundary objects and heterogeneous distributed problem solving. In Distributed Artificial intelligence (Vol. 2), M. Huhns, Ed. Morgan Kaufmann Publishers, San Francisco, CA, 37-54.