Arguing the case for qualitative research on learner experiences

 

Exploring at the beach
Exploring at the beach   by julochka CC BY-NC 2.0

Reading Simon Ensor’s article in Hybrid Pedagogy about the paper Jenny Mackness and I wrote about Rhizo14 reminds me that I made a promise to Simon and others, a promise that I haven’t kept. I said back in March that I would respond to some of the criticisms that Simon and others made of our paper. I put that on hold because of personal issues but it’s time to put that right.

Simon expresses powerfully his personal reaction to reading our paper and I respect that. I can benefit from reading his article, even though, unsurprisingly, I take a very different view from him in many respects. One aspect that Simon and I share as an interest is ‘community is the curriculum’ and Jenny and I are currently working on more research on how that seemed to play out in Rhizo14.

Simon and anyone else has a perfect right to tell their story but I do wonder if in telling his story, he also tells stories about others. That seems inevitable. Several times over the last year, Simon has told me previously that our research is scientific, and implied that we crave objectivity, presumably at the expense of subjectivity. We explained it differently in our paper. Shortly after our paper was published, someone who left Rhizo14 after a disagreement in the first few weeks contacted me to thank us for publishing the paper, as they put it, “for saying what needed to be said”. I don’t know if this person completed our survey, I suspect not, but I was fascinated by their perspective, and it was different from Dave’s take in this video conversation.

Simon and some of the other commenters on our paper from Rhizo14 have criticised the paper for a lack of balance. I think our paper does acknowledge that for many Rhizo14 was a wholly positive experience and we indicate that the negative experiences were in a minority.  My view is that minority experiences can be important and revealing – offering us an opportunity to learn more about something. If more learning is taking place online away from traditional class rooms, then finding out more about how to maximise inclusion, minimise problems/misunderstandings and recover from the ones that occur seems like a worthwhile endeavour to me.  The number of participants and nature of participation is impossible to tie down and we haven’t claimed to do that. We know that we had 47 respondents and that more than 500 people participated in some way in Rhizo14. We couldn’t and wouldn’t claim to say there was an x% satisfaction rate in Rhizo14 – that would be fairly meaningless. What I don’t understand is why we are expected to achieve balance by word count within our article. I am looking forward to reading the auto-ethnography publications when they come out  and I don’t expect them to achieve some sort of arithmetic balance.

We had to develop our research approach on the hoof and we worked hard to consult Rhizo14 participants as we went along. Speaking personally, I am proud of what we achieved and pleased that it has since been reused by others on Connected Courses.  Since some of our respondents elected to be anonymous, I think we can say that they were, in some cases, saying things they wouldn’t have said in public or in Dave Cormier’s published survey. I have been extremely puzzled by some reactions from Rhizo14 participants that seem to suggest that it’s somehow unfair for people to share bad/mixed experiences anonymously – wasn’t confidential sharing the foundation of a long history of qualitative research? Why not wonder about why they didn’t feel able to raise their concerns at the time? or be interested to find that others’ experiences differed from your own?

Simon alluded to what community might or might not mean in Rhizo14
I am beginning to think that ‘The community as curriculum’ is a hopeless simplification of rhizomatic learning.

It is a mess with lions, hyenas, bacteria, and all and sundry running around in an open ecosystem.

and Keith Hamon used the analogy of a rule-based game played on a geographically located pitch.

If a group of people wants to play futbol except for one who wants to play baseball, then that one should disengage or decide to embrace the futbol game, and the group should not feel compelled to quit playing futbol to accommodate the one. Fortunately, MOOCs can be large enough to accommodate both futbol and baseball games, if the players will organize themselves that way. What isn’t acceptable is for the one baseball player to stay and poison the futbol game. It would have been wrong of me, for instance, to insist that Rhizo14 focus its discussion on Deleuze and Guattari’s rhizome metaphor just because that was the game I wanted to play.

Leaving aside the question of who might be the baseball player who stayed to poison the futbol game (answers on a postcard please), I was left wondering where was the pitch at Rhizo14?  Was the pitch the Facebook group, the G+ group, the Twitter hashtag, the baseball player’s blog, a futbol player’s blog comment stream?

Or do all of these form an open eco-system as Simon suggests and how will the question of which game (or games) will be played be decided? These seem to be important issues for a community (possibly of sub-communities) that is getting together and forming curriculum.

The possibility of new games in learning online excites me – and I want to play those games and sometimes engage in research in them.

Veletsianos(2013) has identified that emerging technologies may not be new, are always becoming, and may be hyped even though they haven’t achieved their potential. His final point that they are neither fully understood nor fully researched has been taken up by Jen Ross and Amy Collier  as ‘notyetness’ and they have identified Rhizo14 as an example of the ‘’notyetness of practice’ .
I would agree with Veletsianos in seeing research as a potential antidote to hype and would argue that our research is complementary to the notyetness of Rhizo14, uncovering hidden and different perspectives that can contribute to the becoming of courses like Rhizo14 and to the becoming practice of participants.

What does surprise me in some of what Simon says and what I read elsewhere is an attitude that seems to reject (rather than critique) research based on qualitative data. I am beginning to think I am missing something – why would research not be needed?

Veletsianos, G. (2013). Open practices and identity: Evidence from researchers and educators’ social media participation. British Journal of Educational Technology, 44(4), 639–651. doi:10.1111/bjet.12052

Are we there yet? the notyetness of emerging technologies practice and research in online learning

Looking rear view mirror
Looking backwards and forwards CC-BY-NC

Any parent will recognise the plaintive question “Are we there yet?” asked by children impatient for the journey to be over, the holiday to start. In using technology in learning and education, hopefully we will never really get there but experience fun in our experiments and learn from our experiences. We may find that there are better and worse journeys, decisions that opened up opportunities and others that closed them down.

Amy Collier and Jen Ross have come up with the intriguing concept of notyetness, based on George Veletsianos’ assertion that emerging technologies are ‘not yet fully researched” and “not yet fully understood” in online learning. Amy flags up the enabling characteristics of emergence that can flow from notyetness

creating space for emergence to take us to new and unpredictable places, to help us better understand the problems we are trying to solve.

She gives three examples of projects with notyetness that have touched me personally in the last year: Domain of One’s Own , Fedwiki Happening, and Rhizo14.  I am currently reinventing my web site with the wonderful help of Tim Owens at Reclaim Hosting, I have two fedwiki happenings under my belt and I was a participant and researched (with Jenny Mackness) Rhizo14. This morning, I watched the livestream of Sheila MacNeill’s keynote to #OER15, where one of the themes was the different shades of open that might be tried in the drive to mainstream Open Educational Practice. Sheila seemed to be recommending a pragmatic approach to mainstreaming of OEP, and this chimed with Lisa Chamberlain’s identification of Facebook Groups as a kind of not-yetness

The ideas of not-yetness at an MIT or Stanford are so far beyond the realm of my little community college that they would intimidate or even shut down emerging technology discussion for all but a few of the most technologically-edgy of faculty at my school.

But Facebook, good ol’ Facebook, almost the grandpa of social media now, is a kind of “not-yetness” on my campus. (Not to mention it has a nearly flat-line learning curve which is important for a 10 week quarter). The idea of opening a class to social media of any kind is not-yetness here. The use of Facebook groups is not-yetness here. The connectedness of letting outsiders participate with students in a class via Facebook is very not-yetness here.

I should declare that I have a love-hate relationship with Facebook: it’s great to keep in touch with my far-flung family but I have concerns about Facebook privacy aspects and its use of data. I taught a first year undergraduate module 2009-2013 where most of the students were Facebook users, making it possible for them to conduct small group investigations into privacy and data use (meaning Facebook membership was not compulsory and there was no Facebook interaction with tutors). So I feel sympathy with Lisa’s pragmatic approach but also a little uneasy.  There are quite a few reasons not to use Facebook in formal education: the creepy treehouse effect, Selwyn(2009)’s recommendation to leave Facebook as a backstage space for students, and the implications of mandating students of a space that is managed externally to the college or university. 

In thinking further, I also recognised an interesting link between the notyetness of both emerging technologies online learning practice and research into that practice. Ferguson(2012) identifies three different interest groups for learning analytics – governments, educational institutions and teachers/learners. When learners interact via Facebook, the major interest groups for the data are Facebook and the customers to whom they sell advertising.  Although, Facebook makes some anonymised interaction data available through apps like Netvizz, their track record for research is questionable. Selwyn’s 2009 study found that only 4% of student interactions related to their studies and so students who elected not to participate in an optional Facebook social space were not too disadvantaged but moving significant  elements of learner interaction to Facebook seems different somehow.

There is already a trend of outsourcing some educational services to private companies, and it’s easy to see that ‘free’ services can be attractive but Facebook isn’t Reclaim Hosting whose strapline is “Take Control of your Digital Identity”.  What will we see in the rearview mirror if there is a substantial move of study-related interaction to Facebook groups?

Ferguson, R. (2012). Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5/6), 304–317. Retrieved from http://oro.open.ac.uk/36374/1/IJTEL40501_Ferguson Jan 2013.pdf

Selwyn, N. (2009). Faceworking: exploring students’ education-related use of Facebook, 34(2), 157–174. doi:10.1080/17439880902923622 also currently available at http://blogs.ubc.ca/hoglund/files/2011/05/facebook.pdf

Four women on a train – one of many possible stories

This story is dedicated to Heli Nurmi whom I have known since we worked together on the CCK08 MOOC in 2008. Heli blogs about open learning, and regularly participates in MOOCs. She has extensive experience of research and practice in education, brings much insight to discussions at blogs and on forums.

I have visited Finland twice- the first time in 1999 was to meet up with educators using digital technologies and the second time was to attend a conference in Turku in 2004.

View of lake at Repovesi
Repovesi by https://www.flickr.com/photos/anroir/ CC BY-NC 2.0)

On the first trip, with a colleague I travelled by train from Helsinki to a college in Kouvola, then on to the University of Tampere and back to Helsinki. Train journeys are a great way to see a country and I remember the landscape of lakes and forest we saw, though checking the map reveals how little of Finland I saw.

On one of our train journeys, we sat in a pair of seats facing two women, one old, one young. The older woman was very keen to speak to us though she spoke no English and we spoke no Finnish. She soon established that the younger Finnish woman spoke English and so she persuaded her to translate. We struck up a conversation about where we were going and where we came from.  The translator seemed to become increasingly bored and uncomfortable until the older lady launched into an animated story that lasted about 5 minutes. Our translator turned to us, shrugged her shoulders, and said “Shit happens”. We smiled and got off the train at the station where we needed to change trains.

In writing this story, I struggled to remember details (apart from the memorable ending) and it occurs to me if either of the other women remembered the meeting they would very likely tell the story differently, as would my colleague.

Cycling between private and public in researching Rhizo14

Howling at the moon
Howling at the moon, Sculpture Park, Aalborg

Our first paper Rhizo14: A Rhizomatic Learning cMOOC in Sunlight and in Shade from the research we conducted at Rhizo14 was published last week at Open Praxis.  We would love you to read it and respond.

One of the themes that has engaged us in the research process is the delicate dance between the private and the public.  Public and private can relate to strategies for engagement behaviours rather than being properties of spaces (Lievrouw 1998). We found that offering privacy in data collection was a good strategy in that we are able to reveal some things that were not apparent on the surface of Rhizo14, adding to our partial, provisional understanding.

On the other hand, we are pledged to publish only in open access journals, and shared our data collection approach with Rhizo14 participants who helped us to shape it. Dave Cormier, the convener of the MOOC kindly agreed to a private conversation reflecting on Rhizo14.  We have spent a long time (a year) reading and analysing the data, reading other writers, and writing, alone or together.

We have also presented interim findings at a conference at University College London, blogged over several posts and we have blogged our ideas before, during and after Rhizo14.  It was great to get feedback at the conference and on our blogs.

And now we have published an article that was private while it was being written, reviewed and edited, and we look forward to getting your feedback on what we have said. Of course this apparently fixed article is only a snapshot of ideas. Our ideas have moved on even in the relatively short period since we completed this article (November 2014), and it would be different if we wrote it today.  We have, this month, submitted a second article on the rhizome as metaphor and concept, and we are currently working on an article about community formation.  But the cycling between private and public – reflecting, reading, thinking, writing – that’s so important. If our paper motivates you to respond, we would be thrilled to hear your ideas.  We don’t want to howl at the moon.

Lievrouw, L.A., 1998. Our Own Devices: Heterotopic Communication, Discourse and Culture in the Information Society. The Information Society, 14, pp.83–96. Available at: http://classes.design.ucla.edu/Spring06/259M/readings/OwnDevices.pdf

Jenny Mackness @jennymackness jennymackness.wordpress.com/

Frances Bell @francesbell https://francesbell.wordpress.com

ALT Annual Conference 2015: Shaping the future of learning together – submission deadline approaching

Snapshots of ALT-C
Snapshots of ALT-C

The ALT-C 2015 Conference is at the University of Manchester 8-10 September, and will be hugely enjoyable if my experience of previous conferences is anything to go by.  Two excellent keynote speakers have been announced: Laura Czerniewycz who does great work in Open Education and Steve Wheeler , a prolific and popular blogger and tweeter.

The programme will reflect the theme of shaping and sharing learning through breaking down the traditional divisions between stakeholders and between their roles, with a focus on:

Harnessing the power of the crowd – collaboration and connectivist learning;

Social media in learning and teaching;

Open educational practice;

Learners as agents of change;

Participatory approaches to the development of learning technologies.

This has encouraged me to prepare a submission – we have until Friday 13 March.  But what has really excited me is a change to the organisation of the sessions, and to what they might lead to.  Instead of categorising them as workshops, presentations or research papers they are just sessions with clear criteria for audience engagement.  I think that can offer opportunities for creativity for presenters and audience.

There is also an open and flexible concept of education that is dear to my own heart

Here education is considered broadly and includes formal and informal learning settings in schools, colleges, universities, the workplace, homes and communities, at any stage in learners’ lives.

I recommend you to visit the web site and read the Call for Papers. I am a former editor of Research in Learning Technology, and I really welcome the shift of focus from inviting Research Papers for the Conference to supporting authors to develop the work that they present at the conference into a paper for Research in Learning Technology.  As the Call says,

This is a new opportunity we are offering this year with the intention of increasing the work showcased at the conference being published.

This is a wonderful opportunity to increase the relevance of the conference and the journal to each other. I am imagining more conference attenders writing and reading papers from RiLT and elsewhere, and exciting topics, articles and multimedia making their way from the conference to the journal.

One of the aspects that really excites me is the potential for collaboration all through the process – co-authoring the abstract, getting feedback from reviewers, presenting the session and interacting with participants, getting feedback there and via social media, co-writing a research paper, getting feedback from editor and reviewers, rewriting and finally publication, then getting feedback via social media, since RiLT is open access.  I could even recommending blogging as part of the development process – see the rich discussion around this blog post

So go on – submit a session abstract and hopefully I will see you in Manchester in September.

A table of cMOOCs – can you improve this?

During the course of a research study that Jenny Mackness and I are doing, we have been thinking about cMOOCs.  What have been the subjects and purposes of the various courses that might style themselves as cMOOCs?  Who does cMOOCs and why?

It’s difficult to say what a MOOC is – apart from it being a Massive Open Online Course..  Not all examples of MOOCs are Massive, they can see Open as being about open to join, open to the web, they are usually Online though participants may have local meetups, and they usually have some affinity with the idea of a Course, in that people may learn over a more less fixed period and a teacher might show up from time to time.  Dominik Lukes has come up with an undefinition of MOOC as family resemblance.

So if that’s what a MOOC isn’t, then what is a cMOOC?  cMOOC is now used to refer to connectivist MOOC family relations of CCK08, the original MOOC Connectivism and Connective Knowledge, run by George Siemens and Stephen Downes.

So if you are or have been a convenor or participant in a cMOOC, can you help by checking, amending, extending or commenting on the information we have gathered so far? This table can’t be completed or fully representative but we would like to get a good enough picture of the short history of cMOOCs – particularly who does cMOOCs and why they do them?

Link to Google Doc http://bit.ly/cmooctable – it’s probably easier to edit the table through this link than the embedded document below.

Getting another perspective

This post is a progress report on the research that we (Frances Bell and Jenny Mackness) have been conducting following our participation in the open course Rhizomatic Learning: The community is the curriculum in January 2014.  This research to date has included an online survey, email interviews and a conversation with the convenor of the course Dave Cormier. It has also resulted in a conference presentation (see http://altmoocsig.learningtechnologist.co.uk/category/altmoocsig/

Research approach

If your interest lies solely in the brief report of our conversation with Dave Cormier, go straight to the summary report below.

Jenny and Frances met through the first Massive Open Online Course (MOOC) CCK08, run by George Siemens and Stephen Downes in 2008. In contrast with the huge MOOCs that followed it, CCK08 has been characterised as a connectivist MOOC or cMOOC.  Separately, we undertook research from that MOOC , and though we have come across each other from time to time, we did not work together until January 2014 when we met on a MOOC convened by Dave Cormier, Rhizomatic Learning – The community is the curriculum, now commonly called #rhizo14.

Rhizo14 was a diverse and busy MOOC: light on readings and loosely structured around weekly ‘provocative’ questions posed by Dave.  Interaction between participants was lively, spread across a variety of platforms and spaces: the P2PU space, Dave’s and participants’ blogs, #rhizo14 Twitter hashtag, a G+ community, a very active Facebook group and of course the less visible but equally valuable backchannels.  There was evidence of joy, creativity and a few clashes of expectations where participants seemed to expect different things from #rhizo14 and were sometimes disappointed by the actions and behaviours of other participants. That is a scant description of what happened.

We were struck by the contrasts between #rhizo14 and other cMOOCS: there were plenty of learning moments but we also experienced and observed some rather painful interactions. Our curiosity was piqued: we wanted to know what was going on beneath the surface and how a range of participants were experiencing #rhizo14.  This seemed significant, given that MOOCs are an opportunity for learning for people who may not otherwise have access to formal education. This inspired us to undertake some research, starting by recording our participant observations, saving links to significant interactions, and then planning how to get participant views first hand.

Given that our research was, to some extent, precipitated by concerns, we were both conscious of the dangers of finding what we were looking for in our research, as outlined by Stephen Downes in his presentation on MOOC Research.  It seems to be a good thing to strive for objectivity, whilst acknowledging that it is an unachievable goal. In outsider research where the researchers see themselves as outside the researched situation, objectivity is a key element of the traditional science-based approach normally adopted.  However, the emphasis on the rigour of the research method can compromise the relevance of the research.  We were engaged in insider research, we became participant observers in #rhizo14.  We could acknowledge the danger of reduced objectivity (whilst taking measures to counteract it) but we could also benefit from the subjectivity of researchers and other participants who shared their views with us. Subjectivity can bring a significant contribution to research in complex situations involving people and their relations with material things and each other.

Our first steps were to declare that we were conducting research, and to engage with other participants (via a Google doc) on what would be ethical ways of using data in our research.  Having consulted and planned our data collection, we shared this as widely as possible in all the spaces in which rhizo14 was evident https://francesbell.wordpress.com/research/rhizo14-research/

Our research process developed organically – the current (but probably not final) representation of this is shown in the following diagram.

Research processv2
Organic Research Process

 

As well as being mindful of and explicit about our roles as researchers, we are very conscious of the partial nature of the data we have collected and are trying to analyse.  The distributed nature of the spaces, the mix of public / private, and the number of survey respondents (47) combine to remind us that we must be missing some important perspectives. What does encourage us is that despite this partial view, our decision to allow for confidential and electively anonymous responses to our surveys, has enabled a light to be cast on what people are thnking, and not saying in public and semi-public forums.. We will make a contribution to the hidden MOOC experience.

Having conducted our survey and email interviews,and recorded our observations, we were conscious that we were missing another important perspective, that of Dave Cormier the course convenor. We approached him to engage in a conversation, rather than an interview, loosely based on the issues  and ambiguities that had emerged to date in our research.  Whilst Dave’s natural inclination was towards openness, he graciously conceded the benefits of having a private conversation where we could discuss issues freely.  The outcome was that although we recorded the Skype conversation between Dave and Frances, we agreed that the recording would be shared between Dave, Frances and Jenny, to be used confidentially for research purposes.  Although this blog post is written by Frances and Jenny, it has been agreed by Dave prior to publication.

Summary of conversation

We started by establishing the privacy/ confidentiality arrangements for our conversation and how we would publish it on blogs. We explored the actual ethics of research as compared with formal ethical approval. It is interesting that some of the most interesting parts of the conversation don’t make their way into the following summary. In a way, this vindicates its private nature. The following summary is not in strict chronological order as the conversation took itsown course, returning to topics at different times.

Community – concepts and issues

Dave explained that his conceptions of community owed more to Nancy White than to Etienne Wenger’s Community of Practice. He feels that being able to participate in a community of knowing is the goal of the learning process, and for him community is people caring about each other.

Dave discussed his experiences in EdTechTalk community , that he started in 2005 with Jeff Lebow and the benefits of being part of EdTechTalk. Dave highlighted the impact involvement in EdTechTalk had made on his work at UPEI.

We discussed that while both teachers and learners may form community, teachers may particularly value parallel community experiences to those they are facilitating with their students.

Sustainability

Sustainability of communities and MOOCS proved to be an interesting topic. We discussed examples of more or less sustainable examples of each.

Compatibility of community with rhizomatic thinking

This was an interesting topic.  There seems to be inevitable tension between the two concepts, and we discussed the advantages and disadvantages that can emerge from this tension. Community is the curriculum is a concept that Dave explored in the lead up to his 2008 article. Dave has found the rhizome useful in the cause of getting rid of the ‘content’ in education as he sees that as in opposition to how people, doctors for example, really learn their practice.

De-centring the Leader

Several times we returned to the issue of the centring or otherwise of Dave’s role in the course. We explored the benefits that his leadership brought and also some of the problems in which it played a part. Is it possible to achieve benefits such as seen in EdTechTalks without a leader? We also discussed the effectiveness of some of the tactics that communities employ to take the focus away from a leader’s decisions and actions, including community guidelines, FAQs, distributed moderation.  This was a rich discussion drawing on real incidents from rhizo14, and made more possible in the context of a private conversation.

Inevitably, we ran out of time to discuss everything on our agenda.  It was a challenging, rich and warm discussion that will inform Jenny and Frances’ research and Dave’s planning for rhizo15.