Notes on Margaryan, Littlejohn and Activity Theory as a framework

Margaryan and Littlejohn (2007, 2008) analysed the mismatches in the perception of repository curators and users. One of the issues really hit home for me:

The curators focus on repository centric factors, while users spotlight a wide range of contextual factors. They explain this as following: Repositories are frequently introduced to users as sandalone tools. Users, however, see them only as one component within an entire activity system. They recommend that curators and users have to think through the ways in which individual components inter-relate.

This is what I actually realised this summer when we were at the summer school with MELT teachers. At the point where our system failed to work, teachers did not loose too much time but started checking their delicious accounts and bookmarking some interesting learning resources there that had been introduced earlier during the day. That moment, somehow, was an awakening moment for me. I realised that what I've been hassling about for so long, our dear repository, the one and only, is not really one and only source of information for them. Just one among many others that we are not even interested about.

Hence the little idea of integrating users delicous tags and bookmarks on the MELT portal. A logical place for them would be at the Favourites' section: here, on the first tap, are my bookmarks from MELT, and over here on the second tap, are my bookmarks from delicious too. Cool,ugh, inter-relating the services that teachers use. Also, since by default all my Favourites in MELT are publicly available to other users, so would my delicious bookmarks be.

The whole idea goes much further to integrating these using APML to create a profiling tag cloud from my tags from both places. The workshop paper is found here, I still need to work on it a bit.

Other interesting things about the papers:

The study was build using the Activity Theory from Engström 1987 as a theoretical framework. It also might be interesting for me, as I am missing one. Margaryan and Littlejohn (2007, 2008) claim that it offers a holistic framework that allows to study LORs and communities as a single system, rather than as a loose set of instruments, subject, objects and outcomes. It provides an analytic lens to understand the complex relationships wihin each system.

Activity Theory as such belongs to the family of socio-cultural approaches to learning (e.g. Vygotsky), situated learning theoris (Lave) and communities of practice approaches to learning (Wenger, there he is again..). The paper explains that the common denominator for socio-cultural theories is the importance of social and cultural contexts in learning.

From that perspective Activity Theory might make a nice match. One thing why I first was skeptical about it was that Margaryan and Littlejohn in (2007) say this theory offers a method of analysing the development of LORs as participatory environment where knowledge is co-constructed rather than "exchanged" or "consumed". I am not sure whether LORs really were developed in thinking of co-construction of knowledge, at least not before we mixed in the social tagging stuff. From that point, then, it becomes interesting, maybe.

In Margaryan and Littlejohn (2008) authors also talk about how social co-creation of knowledge is facilitated through the use of tools, either concepetual or physical. A dialogue can be such a conceptual tool, but so can email or blogs. Also tags, I guess, can subscribe to that.
Another thought that came out from reading the 2008 journal paper was that it also talked about Leontiev (1981) and analysing an activity from 3 different levels. The first level related to the overall motive for engaging with an activity. The second level relates to the actions that constitute an activity that are governed by (short-term) goals. The third level of activity related to the operations necessary for carrying out the actions. This made me think of "levels of participation" like in this ladder (or the long tail one). What they also try to depict is that there are different levels of participation, they are differently motivated, and maybe when talking about learning, we can also observe similar levels as pointed out by Leontiev (1981).

A few ideas for the evidence finding paper:

• The dimensions of repositories and communities can be used to describe the datasets that I will use
  

Start for the evidence paper: Assume that repositories and learning resources get rid of technical, socio-cultural and pedagogical barriers for usage (references from the JISC report on Learning Communities and Repositories from CD-LOR project), does the re-use across the national and linguistic borders happen? If evidence is found, how much and where?

Engestroem, Y. (1987). Learning by expanding: An activity theoretical approach to developmental rsearch. Helsinki: Orienta-Konsultit Oy. Retrieved August 25, 2008, from
http://lchc.ucsd.edu/MCA/Paper/Engestrom/expanding/toc.htm

Margaryan, A., & Littlejohn, A. (2008). Repositories and communities at cross-purposes: Issues in sharing and reuse of digital learning resources. Journal of Computer Assisted Learning (JCAL), 24(4), 333-347.

Margaryan, A., Littlejohn, A. (2007) Communities at cross-purposes: Contradictions in the views of stakeholders of learning object repository systems. Proceedings ascilite, Singapore 2007.

How do tags connect to the Thesaurus terms?

Our social tagging system in MELT is special in two ways:

• one, we support multi-linguality
• two, we have not only tags, but all resources that users tag also have Thesaurus terms

In that context it becomes very interesting to know how do tags relate to the Thesaurus terms that have been used to index the resources that users tag.

I took a sample of tagged resources (n=185) that have 1013 tags associated with them. Out of those tags, there are 595 distinct tags. There are 44 users.



I made a network diagram visualisation that displays the Thesaurus terms as nodes that are connected by edges to tags. You'll find it here to play around with it. Unfortunately, I found out that 24 resources did not have Thesaurus terms related to them(that's about 13%, hmmm), thus a big plumb node in the middle without a Thesaurus term.

There is another visualisation here, it's more explorative about the data.


It's rather interesting that 595 distinct tags from users can be comprised to 34 thesaurus terms. That is 17,5 tags per Thesaurus term on average. Of course it does not go like that, it's more like rich-get-richer-type of a story. In the visualisation above you can see that most tags are related to language learning, for example.

If you look at the distribution of tags you'll find that many of the top tags are also about languages. Interestingly, many of them repeat the topic of the resource, but some of them (clearly less) state something about the nature of the resource (e.g. interactive) or the type (e.g. exercise).

The problem with creating this kind of visualisation of tags on the system level will be that the resources seem to have too many Thesaurus term. If there are 5 or so indexing terms, everything becomes related to everything else. It might be interesting to either to ask limit the Thesaurus terms to three (as should be the case anyway) or ask the indexer to give one term priority over others.

The same also goes for content-based recommendations, btw. If there are too many terms, you recommend everything for everyone.

On the memory lane of the Internet - Paris 8

It's great to be getting older. It turns the Internet into a memory lane, something like cleaning your old cupboards in the place where you grew up, finding old pictures, mails, etc.

I just received a mail from someone asking me if this (updated link to Internet Archive: https://web.archive.org/web/20090501062208/http://membres.lycos.fr/riina/dea.html) was something that I have written. It was "mon memoire du DEA" from 1999 in the department of Hypermedia in Paris 8! I have not even put my name on it, but somehow this person was able to find it and associate it to me. Best of all is that she still found it useful for her studies, she wanted to cite it in her own dissertation on language teaching and new technologies, but since the text did not have the author nor the publication date, she found me. You never know, do you now..

So here it:
Riina Vuorikari
ENSEIGNEMENT ET APPRENTISSAGE
LE CAS DES LANGUES ÉTRANGÈRES
EFFETS DES TECHNOLOGIES DE L’INFORMATION ET DE LA COMMUNICATION
Date de parution: septembre 1999
Lieu de publication: L'Universite de Paris 8, Saint-Denis, France.

Notice the way that the title was written, no commas but different lines. Pretty arty, ha? It was mostly influenced by, hm, my really eccentric pormotor, J.Feat. So, I went back to my Yahoo! mail that I used already back then to check the mails between us when studying. Man, he was somewhat strange, but who would not be in Paris 8!

I always joked that the hardest task in it all was to get out of there with a diploma, what a mess. But Fun. A good place to hang out. Check out what the French version of wikipedia says about it. The English version is lame, it's hard to capture that feeling of "papa cools", all the old hippies from the late sixties who had installed themselves there ever since the Youth revolution of 1968. Every day there was (I bet still is) a student "manif", a little protest or signing a petition on this or that. When I read parts of my dissertation I noticed how that radicalism had snuck in..

D’un côté Internet offre " un accès libre au monde ", il est " international, pluriculturel et multilingue ", mais ceci est une image idéalisée d’Internet. D’un autre côté Internet est vu comme un média conditionné par McWorldet par les concepts d’américanisme à l’échelle mondiale. L’homogénéisation culturelle et le commerce électronique comptent sur l’idée que la consommation devient l’unique activité humaine qui uniforme le goût des consommateurs.

..and at the end about the future perspectives:

Une autre piste de recherche sera l’industrialisation de l’enseignement. Les questions soulevées par ces tendances sont multidimensionnelles : Est-ce que l’interaction humaine pourra être remplacée ? Est-ce que les enseignants qualifiés seront remplacés par les moins qualifiés une fois que le contenu du cours est mis en place sur Internet ou sur le cédérom ? Qui aura la propriété des contenus de ces cours qui sont devenus des produits à exploiter, le professeur ou l’administration de la faculté ? Qui aura l’intérêt à vendre ces contenus, qui aura les droits d’auteur et qui va gagner l’argent ?

Outch. Then again, there is lots of good stuff too. I love the translation of knowledge network in to "le tissu de savoir" or calling the whole Internet as " tissu social virtuel"! What a foresight! I remember sitting with my supervisor in the Montmartre graveyard and he was explaining that instead of talking about the Internet, I could use the term " tissu social virtuel", like a web woven in a tissue where the threads are all intervened, to illustrate the use of the Internet. pretty funny in its own way..

This is my opening line:

Internet est souvent associé au concept d’interactivité. Est-il possible d’exploiter cette propriété pour mettre en œuvre des techniques spécifiques pour l’enseignement de langues vivantes étrangères ? Au contraire de l’enfant qui apprend sa langue maternelle dans un environnement naturellement interactif - et de façon permanente - l’étudiant suit régulièrement un cours où l’immersion linguistique est artificielle et de courte durée. Jusqu’à quel point est-il possible de reconstituer, à partir d’un tissu social virtuel (=Internet) les conditions idéales d’apprentissage, particulièrement l’apprentissage des langues étrangères ?

I remember when I finally was writing my dissertation, my promotor was merciless. He really cracked the whip on me. But he knew how far to push, and at the end also I was really pleased with the result. After all, it was mention bien. When I thanked him for this, he said " Please, don't you give no "thank you" -- after all, it's my job...". That's a true educator! But hey, he could have taken some credit for it.

A funny thing was that he's English was perfect, but I only learned about that after we were done with all the writing. He was harsh on my French too. When I had already handed in the first version of my dissertation, he congratulated me on it. But half way down on his mail he says that in its current version no one can read it without lots of difficulties :

Toutefois, tu ne recevras ton diplôme que si tu déposes plus tard un nouvel exemplaire, corrigé de toutes les fautes de français -- il faut comprendre que cet exemplaire est destiné à la bibliothèque, et que, tel qu'il est, personne ne peux le lire sans une grande fatigue...

The final discussions that we had before my defense took place in Montmartre. We once met in that same café where they shot scenes for Amelie, or at the graveyard. I was quite surprised to see them in the movie, I remember.

It was also fun now to check out on some professors from Paris 8. I found Jean-Pierre Balpe who at the time was the head of the department of Hypermedia. Jean Clement who taught us a course on hypermedia. Imad Saleh, one of my "rapporteurs", is now the head of Laboratoire Paragraphe, that's the nickname of the deparment (so French, I love it!). Or Jean-Lois Weissberg - I never understood anything about his lectures, he talked about "telepercence" and such. I only remember that soulful intellectual Greek philosophy student who engaged in discussions with him about things that I could not follow. After all, they all carried the legacy of Pierre Levy who had just left the department year earlier.

I tried to dig out on my Lycos account where this stuff now resides, but I could not find my password and the system does not recognise my username :( But I found some other old stuff, like the index page of the site that I made at that time for Finnish students in Paris, Suomalainen Osakunta, my cool looking CV and the side of akseli that Pia, "Puumatyttö", teki suomalaisille taiteilijoille ja jota autoin jossain vaiheessa. Ihanaa!


Well there, that was a nice moment of memories from Paris and end of the Millennium.

Can Social Information save teachers' time when choosing interesting learning resources?

One of my research questions is aimed at understanding what so called Social Information can do to help teachers to choose the right learning resources from a seemingly overwhelming collection. By Social Information I mean information about previous users' interactions with the resource. I am mainly interested in explicit annotations like ratings and tags, and more implicit ones like bookmarks.

As I'm interested in the use of resources that come from different countries than users do, I think Social Information (SI) should display not only annotations, but also information from where the user comes from.

One thing that I hypothesise is that among other things, Social Information, when associated with conventional metadata about learning resources, can make the decision making process faster for teachers when, for example, looking at the search result list. As a multilingual context in a repository can result in metadata that is in different languages, it could be speculated that Social Information indicating the origin of the users who have previously annotated the resource, could help the other users to make up their mind (see the image for an example).

We were interested in two different aspects:

1. Does the appearance of Social Information make the decision making process any faster?
2. Does the appearance of Social Information make the users choose more resources?
   

Method

We had 25 users from five different European countries. These teachers are primary and secondary teachers in science, language learning and ICTs in Finland, Estonia, Hungary, Belgium and Italy. xx of them are females and xxmales. xx participant is under 30 years old, xx are under 40 years, xx under 50 years, xx under 60 years old.

They have been part of the MELT project since Summer 2007. In March 2008 they were invited to create a profile on the MELT portal, where they are able to access multilingual learning resources for different topical areas.

We designed an experiment where teachers were shown two different imitations of search results list with learning resources and their associated metadata. One of the lists showed what we call the conventional metadata, such as title, url, language of the resource, a short description, subject area, type of content and its target audience. Here is an example.

The other list had the same metadata, but we also added the Social Information from the previous users. This could be the tags in their original language, the number of times bookmarked (favourites) and the ratings. Also, for bookmarks we would mention from which country the users come from. As example of this was shown above, the first image in this post.

We had 48 learning resources that came from different countries and were in different languages. About half of them were in English and other half in other languages, this also seems to reflect the division of the resources that users have bookmarked on the portal. The resources were about language learning, primary education, ICTs and science material, those were the areas of the teachers. I'll prepare better information about this later.

We had 12 learning resources on a page imitating a list of search results that user could get on a repository. In total, there were 4 such pages for each user, we call them sets. Every second set had conventional matadata, and every other had additionally also Social Information as indicated above.

At the beginning of the each set the participants were asked to write their names and the time when they started with the set of 12 resources. At the end, when they submitted their results, the system recorded a time. To answer to our first question we were interested in how much time do teachers spent to evaluate the appropriateness of 12 resources for them.

The teachers were asked to look at the metadata of the resource and the resource itself if interesting, and were asked one single question: "Would you use this resources, or parts of it, in your teaching in next Fall?" They answered on a scale 1 to 5, 1 being "I don't teach the topic", 2= No, 3= Maybe not, 4= Maybe and 5= Yes. Looking at the number of resources that users choose in their topical areas would give us indication of whether resources that have more Social Information related to them were more often chosen than the onces without.

Because of the low number of participants (n=25) we decided upon a within-subject design for this experiment. This is the one where the same group of subjects served in both treatments, i.e. they received both the material with conventional metadata and with social information.

Moreover, we had the participants in two different "groups". Group 1 had 12 participants and Group 2 had 13. Group 1 started first with a set with conventional metadata and Group2 with a set of resources that had Social Information added to it. When analysing the results, we found that one user in Group 2 had consistently added incorrect times. We excluded these times from the counts for time spent per set, leaving 12 participants in each. Moreover, in both set there were a few cases where the start time was forgotten.

Results

Descriptive statistics

We had 1129 responses to our questions, which means 71 responses were left blank. In 53% of the cases the users had answered that they do not teach the topic, which means that they deemed the resources not suitable for the topical area that they were teaching. 25.6% of the users found resources that they said that they would use (yes or maybe yes), whereas 21.6% of the resources were not found of use in the upcoming school year (not, maybe not). The mean for the responses was 2.23 (Min=1, Max=4), standard deviation was 1.138.

Q1: Does the appearance of Social Information make the decision making process any faster? Time spent on 12 resources (i.e. set)

On the average, users spent a bit more time on the sets that did not contain Social Information. The average to review a set of 12 resources with conventional metadata was 9 minutes and 8 minutes with Social Information.

I do not know yet whether this is a significant difference (my SPSS license ran out), but one could assume it is at least a sign of good news for Social Information. We can imagine that users go through a lot of resources when browsing a learning resources repository (I currently do not have the logs about the number of resources that users review per session, but I will produce them). So if you think of small cycles and multiply that number with, say 1o times, you could come up to some significant time savings when Social Information is made available to speed the decision making process.

Individual differences

Still looking at the average times spent, we can see that there were many individual differences. In the chart below the blue lines show the amount of time that participants spent with conventional metadata and the red one with Social Information added to it. You can see that for some users one metadata setting seems like a faster way, but anyhow, the lines follow one another pretty closely, apart from some odd-balls (like user 23). You can also see that there seem to be a wide variety of personal ways, some users scrutinise resources with a great care (user 7 and 8), whereas some go through them very fast (user 16 and 17).

I'd like to mention that here it does not matter that some of the resources are not in the competence area of the participants. We focus purely on the time that they spent going through pages and making decisions whether some of the resources are useful for them in the upcoming school year or not. However, this becomes crucial to answer to our second question:

Q2: Does the appearance of Social Information make the users choose more resources?

Table below presents the results when I looked at the amount of resources chosen per set. There was 4 different sets and each contained 12 learning resources in different languages. The two different treatments meant that teachers reviewed 2 sets with Social Information available, and two sets without. As teachers were from different tpical backgrounds, I excluded the responses from users who said that they do not teach the topic of the given resource. In table below you can see the percentile of positive responses (maybe use, use).

It appears that consistently teachers chose more resources when the Social Information was not available. This is contrary to what I expected. I have not calculated the significance of these results, but the differences do look big. In some cases, like in the 2nd set, even about 15% in favour of no Social Information available.

In a way, maybe the appearance of SI makes the teachers more careful or critical to choose the resources?

What is needed now is a follow up study at the end of this school term to check whether these teachers actually used the resources in their teaching. Or, I could check if they have bookmarked these resources on the MELT portal. They know the resources are available there. MORE to follow...

Asume nothing. Plan for everything.

I noticed this ad at the train station when I was returning from my weekend sailing trip in Friesland, Nl. It kinda captured the mood of the first half of this year, or better, it kind of captured the lesson I wish I have learned during that time. Assume nothing. Plan for everything. And then plan some more.

I think I sometimes assume too much and take things for granted. I think that everyone else is "with me" in the same thing or on a same (mental) trip, and I do not bother to explain the plot, set out my major expectations and go through all the details, etc. Then at one point, usually when it is already too late, I realise that this is not what I assumed it was.

Last weekend this dangerous thinking left me in the water when we accidentally capsized out small sail boat (seen in the pic)...and, there was no plan for what to do then. (one can be found here)

Clearly once again I had been trapped with my not so productive way of thinking. We had been sailing on a 6-meter open Falk boat already for a day with a crew of 4. My pal was skippering and we were 3 others with some/good sailing experience. I got in some really good sailing :) I managed to get us through a pretty rough channel with a strong head wind. We had to tack at least ten times to get to the other side, but I managed it all, we did not loose too much speed in turns and I made the guys jumping from side to another to give the weight to keep the boat from flipping. I realised that Falks are darn sensitive to your weight, it really makes a difference on which side you lean on.


So comes along the next day and we are sort of returning. We have one reef down because of the heavy winds. The skipper was steering, I was taking pics in the head, and the two others were somewhere in the middle. The next thing I hear is that one of us slips on the floor and tumbles on the side. Mind you, these are small boats, so by the time I turn to watch, he's trying to grab onto something and I'm sure he's going to fall off the boat.

With the sudden shift of his bodyweight and the heavy gust from the side, the next thing I observe is that he is not going to fall off the boat, but the boat is going to fall with him. I see our chart flying, the bottles and the gas jar landing in the water, and soon everyone else follows. Since I had yanked myself firmly in the head to take pictures, I found myself standing on the low side wall of the boat that now was horizontal in the water and slowly sinking down with more and more water getting in.

I was not sure what to do. Would it be better to stay in the boat or leave it. After a thought of the boat turning turtle, and seeing myself being tangled in the ropes under the boat, I decent in the water and start collecting our belonging. Soon enough I realise it's heavy to swim with cloths on and I thought "heck with our empty bottles of water and my fake crocks that I tried to save", and swam to the others who were on the other side of the boat. The skipper lifted me on the keel. I was like "so what now", but I did not get any instructions. I had not prepared for this and was not sure what was the next thing to do. Our weight on the keel did not have any effect on righting the boat (which was the goal that the skipper pursued).

Then, that's like seconds later, there was the rescue troop behind us. I could not figure how they were there so soon, but later they told me it was because of the regatta that was going on and they saw us capsize. They put a rope on our mast, and with a help of a second boat, we got it back right up. The mast was all muddy, the wind was pushing the boat so much that instead of turning all the way around, it got stuck in the low waters. No wonder our weight did not do a thing to right the boat.

We were eventually pulled back to the harbor by these Dutch gentlemen of the sea and left sorting out our dripping wet packages, cursing about wet mobile phones and the bill to pay for some lost gear from the boat.

In retrospect it feels just like what happened with the PhD where I got "snowflake*d" (i.e. my adviser at the time kicked me out of the programme) . What happened was just like with sailing, one day you feel like you can do all the good moves, and then the next is that you find yourself swimming in the water and wondering "this is not how I assumed it would end". The worst being that I was not prepared for it, I had not thought of what would be the plan B or C for that matter.

Reflecting on all this makes me see a reoccurring pattern that I'd better avoid in the future. It's clear that accidents will always happen and mistakes are made, but there are also precautions that can be taken to prevent them. Those are done by careful planning and someone taking clear leadership on things. Then the other thing is that when accidents happen or mistakes are made, one needs to know what to do next. Be prepared for them. And then some more.

Being comfortable having other people taking the lead is good, good leaders need good followers, like Dan is known to have said once. Perhaps it would be time for me to think more about taking leadership on things and try to influence, at least on my own life, that I'm prepared for everything. That is one characteristics that I expect from a skipper of a boat, and for that matter, it probably should go for any other areas too.