Surveys

From interviews to Instagram, how did we engage students in the evaluation of Clement House?

This article is one of three blog posts on the newly refurbished learning spaces in Clement House. It is written by Emma Wilson, Graduate Intern for LTI. You can find her on Twitter (@MindfulEm). For more information about the Clement House evaluation, please take a look at our final report.


Working with students as partners in the development of their university experience should form an integral part of any institution’s set of policies. However, securing a sufficient level of student engagement, which is also meaningful, poses a challenge across the sector.

Within the evaluation process for Clement House, we have been keen to utilise a wide array of communication channels – including some innovative new approaches which have involved social media. By complimenting the old and new, our mixed method approach to data collection has secured the involvement of 196 students. In addition, we carried out 67 non-participant observation; as such, the Clement House evaluation benefited from 263 pieces of data for analysis.

How did we publicise the work and recruit volunteers?

Put simply: targeted and personalised communications. Which departments are the most active users of Clement House? Where are students most likely to pay attention to posters on the wall? What incentives would attract students to participate? If students want to get involved, how would they like to do so? With the never-ending stream of emails, how do we know which will be paid most attention by students, and what are the alternative channels of communication?

By taking the time to consider the above, it is far more probable that students will show a willingness to engage themselves in a project evaluation.

The use of visual communications has been a core component of this project evaluation. Posters were visible in strategic locations throughout the project, whereby a QR code and bespoke hashtag was used (where applicable). These posters were displayed across all floors of the Student Union’s building, and electronic versions were broadcast in the library and Clement House (including the International Relations Department which is based there).

Poster One: Seeking student engagement in an online survey
Poster Two: Seeking student engagement in a social media competition

   

Findings based on method of engagement

We created an online and paper version of a survey. The questions were identical although the online survey provided space to make any additional comments. We received 55 responses to the survey in paper format, and 45 via the online survey. The social media campaign ran outside of term time, for a shorter period of time (2.5 weeks), and received 12 responses. This data was supplemented by 74 structured interviews of 1-3 minutes that were carried out during the non-participant observations (of which 67 were carried out across 4 weeks).

Key findings from the evaluation can be found in our report and in our other blog posts (see links). We have also drawn together a selection of Tweets and Instagram responses and displayed them as a collection on StorifyA sample of Tweets and Instagram posts can also be viewed in the slideshow below.

Sample of Tweets and Instagram posts 

What lessons have we learned?

A mixed approach to data collection enabled us to find a balance between a purely qualitative or quantitative approach. Whilst interviews provide an opportunity to understand how and why a student feels a certain way, the use of close-ended survey questions ensures a certain amount of objectivity in particular instances. For example, in the survey it was useful to provide students with four options when asked about the purpose of their visit to the learning space. This allowed comparability across floors. However, it was the richness of data collected from the subsequent open-ended questions (whether in the interview or survey) that enabled us to fully understand the reason why a student feels a certain way.

With a mixed method approach, it is important to ensure consistency of methodology across data collection methods. Do you have the same questions for the paper and online versions of the survey? If not, why not? How can any differences be taken into account?

Looking ahead, I would be keen to encourage the future use of a mixed methods approach to data collection. If carrying out a social media campaign, it is important to consider the time of year in which the campaign in launched; if it’s outside of academic teaching, many students will not be on campus, and you will have to place a greater reliance on online promotion. It is also useful to check whether the university is conducting any other surveys – such as the NSS or end-of-year departmental feedback questionnaires – to ensure that students are not overwhelmed by the number of surveys they are being asked to complete.

There is no one-size-fits-all solution to successful student engagement and it is important to consider the following:

  1. Know your audience
    • Who are you trying to secure engagement from? (Students? If so, are you seeking feedback from those in a particular department or academic year?)
    • When might they be most willing to get involved? (Whilst waiting for their next class? As a break or distraction from revision? During a particular event?)
    • What are the incentives for them to get involved? (Focus on your language – emphasise the power of the student voice in contributing towards policy change; offer students the chance to win a voucher; if running a workshop, say that it’s an opportunity to network with peers and even make new friends)
  2. Think about how the ways in which they can get involved
    • Will canvassing a busy student before class necessarily be more effective than a survey that can be filled out in their own time?
    • Is the university keen to promote engagement through Instagram or Snapchat? Can your project also utilise these platforms?
  3. Connect with colleagues across departments and student groups or societies
    • Partnerships and collaborative working are great ways to contact groups of students who might be harder to reach.
    • Think about your audience – who are they likely to be in contact with? If students, do they have a student representative for their academic course?
    • Make contact with the university’s Student Union (SU); for example, their student engagement and communications officer. Getting some publicity on their website, social media feeds and newsletters is great for exposure. Asking to place posters around the SU building is a good way to reach more students.

Ultimately, this project unveiled a positive message: students are keen to get involved in sharing their views on the teaching and learning experience at LSE. 

Don’t be scared to pilot a new approach to student engagement. Understand your audience, think about how they interact in the university community, and take advantage of the new channels of communication. Over the next few years, we are likely to witness a changing landscape in higher education as Generation Z bring to their university a whole set of new expectations, skills and approaches to life in an ever-evolving digital environment. It is an exciting time for universities to engage with students and discuss the potential and opportunities for the future of higher education.  By approaching engagement in a creative way, we are more likely to kickstart a widespread conversation across the entire learning community.

 

Links

Other blogs in the LSE 2020 series: (see here and here)

July 25th, 2017|Clement House, Learning Spaces, Social Media, Student projects, Surveys|Comments Off on From interviews to Instagram, how did we engage students in the evaluation of Clement House?|

Technology and the student journey: introducing phase two of LSE 2020

Written by Emma Wilson (@MindfulEm), Research and Evaluation Graduate Intern for LTI.

This blog post is one in a series of articles that will catalogue the process of, and findings from, phase two of our student-centred project, LSE 2020.

About LSE 2020

Launched in 2016, the LSE 2020 project seeks to discuss, debate and engage with students about what teaching and learning with technology could look like at LSE in 2020. Phase one used a multi-methodological approach that began to uncover the views, experiences and expectations of students. Building upon this work, phase two aims to look more closely at how students use specific pieces of technology in their personal, educational and working lives. From smartphones to iPads, phase two aims to fully investigate the relationship of technology as it integrates itself into all aspects of a student’s journey. By better understanding how students currently view and use technology, future policy can be guided by the voice of the student.

As a recent MSc graduate at LSE, I have experienced first-hand the important role of technology in teaching and learning. Having undertaken an undergraduate degree in 2010-2013, I have already seen how much has changed in a relatively short period of time. This project provided an exciting opportunity to gain a better understanding into how today’s students navigate our technology-rich world, and the beliefs and values in which they hold around its use.

What have we done so far? 

LSE 2020 places the student voice at its heart and several interesting findings have been uncovered so far. Data collection has involved interviews around campus with 88 studentsan interactive workshop and an online survey which has collected the views of over 350 students. So far, three short videos have been developed and a report is due for release later this year. We have presented early findings to delegates at the Change Agents’ Network (CAN) Conference 2017; the PowerPoint presentation can be found here.

Finally, we are going to work with two filmmakers in designing a creative interpretation of the findings that will truly document what it is like to be a student living in the digital age.

What are students telling us about their use of technology..?

We have divided our research findings into three categories:

  1. Digital Ownership, Collaboration, Communication and Usage
  2. Digital Wellbeing and Identity
  3. Digital Literacy

The following videos provide an initial insight into some of the views held by those students who took part in the 88 interviews around campus.

Ultimately, technology is not going to disappear any time soon. Rather than resisting the changing landscape in which we live, study and work, it is an exciting opportunity to embrace the many opportunities that technology affords us in teaching and learning. It is hoped that LSE 2020 will bring to light an insight into students of today, and we can continue to have these conversations in years to come.

Further analysis, theories and conclusions will be presented in future blog posts. Stay tuned for our next blog post, where we will begin to look at the emerging findings in more detail and discuss what this will mean for Higher Education.

May 12th, 2017|innovation, LSE 2020, Projects, Social Media, Surveys|Comments Off on Technology and the student journey: introducing phase two of LSE 2020|

Edtech: The student view on educational technology

Given the limited amount of innovative tools used in their studies, it is hard for students to actually know how technology could better their education.

Having reviewed all the interviews from our Student Voice project, we created a video highlighting a few of our key findings.

As the video suggests, a majority of students stated that PowerPoints are the main “technology” used in the classroom. Many added that, given the limited amount of innovative tools used in their studies, it was hard for them to actually know how technology could better their education.

That being said, students believed that technology – if used correctly – could challenge the current “one to many [educative] system”. The expression “one-to-many” refers to lectures where teachers talk and students listen, often giving the impression of a unidirectional information flow. Students stated that technology could be implemented to make lectures and classes more interactive, to foster teacher-students and student-student collaboration.

The video also suggests that students expect an increase in online pedagogical content. This includes more online courses and online exercises but also online exams. Students suggested that, to prepare them for the use of technology in their future career, more tasks should be carried out on line.

All findings are currently being written up and the full report will be available shortly!

The previous post can be found here

Students’ Expectations for the Future of Technology in Education

Last term, Learning Technology and Innovation (LTI) started a project involving three days of interviewing all over campus. We asked 100 students questions designed to gather their insight about what teaching, learning and technology could look like at LSE in 2020. The three-minute interviews, whether filmed or just audio recorded, have helped us start a conversation from the grassroots up about the future of innovation and education at the school.

We are currently reviewing the hours of footage gathered to create a short video and a report relaying the students’ voices about the future of technology in education. In the meantime, we have designed the following teaser to give you some insight into the project. This teaser is a compilation of the answers given to a single question: if you could describe, in one word, what you would expect from technology in the future what would it be?

I would like to end this post by thanking all the students that accepted to be interviewed, your feedback is tremendously helpful. Stay tuned for more updates and videos!

LTI Staff Survey 2014

In July this year we conducted our annual staff survey about our services, i.e. the support and guidance we provide and the technologies we promote/ support. The survey tells us what academics think of our workshops, what else they need from us, what they think about the benefits of using educational technologies as well as what obstacles to using them exist, and more. You can find the fairly short report on our website.
Two things stood out for me that I would like to raise here. 1. “lack of time”, an issue that has cropped up in our surveys since at least 2010. This isn’t unique to our staff, nor is it only related to learning technology, and it isn’t a modern phenomenon either: we are all time-poor, “there are only so many hours in a day”. Hence, one must prioritise, and it seems that to engage with the benefits of educational technologies isn’t high on everyone’s list. We can only hope to raise the profile of our work even more to tip the scales in our favour. 2. A final comment explains that “I get the impression you are trying to push a solution for a problem that doesn’t exist, i.e. too much tech does not necessarily get students to study better, or for learning to improve”. That second clause is of course trivially true: we would never say otherwise. But that is exactly why we do not push technologies for their own sakes only. Rather, our role as learning technologists is to explore the benefits of technological solutions to problems that we know do exist. For example, we know that students feel that they don’t get consistent feedback, so we offer staff a variety of ways of providing better feedback, more conveniently, e.g. through moodle (including use of voice tools for audio feedback). Or we do know that students like to revisit aspects of lectures that they didn’t understand at the time. If staff agree to be recorded, we offer a lecture capture system that allows students to review those lecture sections. And we allow staff to take control of how they release these lectures, too. We know that very large lectures can be alienating and aren’t particularly good learning opportunities. So we offer ways in which lectures can be changed to make them more engaging, by introducing video elements, or using instant voting systems to allow students to think through their learning together.
But at no moment do we push any of these and certainly not if we haven’t at first identified a problem to which these technologies might be a solution. If you think that we are too pushy (perhaps we have been too enthusiastic in conversations?), then do always feel free to come and speak to any of us, engage us in the debate, explain to us what we can improve about how we go about embedding educational technologies. That is the main purpose of our annual survey, but if we get to speak with you in person, that’s even better!

October 10th, 2014|Surveys|Comments Off on LTI Staff Survey 2014|

Survey 2013 results: Device ownership, ‘BYOD’ & social media for learning

The results of our IMT student survey 2013 are in. We asked about student ownership of, habits with and attitudes towards mobile devices, and about their use of social media in a teaching and learning context.

Ownership of mobile devices amongst students at the LSE is very high – and practically all devices are used in some way to support their learning on campus, from accessing materials and writing notes and assignments on tablets and laptops, to using smartphones for communication and finding rooms or other campus information. We were keen to know if they would mind teachers asking them to use their devices in lectures, e.g. to participate in live online polls and about two thirds said that they would be fine with it (more than a third agreed to using mobiles and tablets). On average, students describe wifi provision good to fair, complaining mostly about frequent drops in connection.

On the social media side, LSE students are fairly strong users of social media, and use them in their learning for communication and collaboration, and to create and share files and documents.  The most frequently used one was, unsurprisingly, facebook, but the most frequently used ones in a learning context were document creation tools such as google docs and dropbox. We asked if students would mind using facebook with students and teachers, and while 62% said they would mind with teachers, only 23% said they’d mind using it with fellow students. One main reason for this is that they do not want to mix the personal with the professional, and another that students quite strongly believe that social media are not conducive to supporting learning.

We will be analysing these results further to see what implications they have for future projects in CLT, i.e. for Learning Technology and Innovation in particular and IMT in general.

A full report of the survey results can be accessed on LSE Research online http://eprints.lse.ac.uk/51652/.

August 13th, 2013|Announcements, Social Media, Surveys|Comments Off on Survey 2013 results: Device ownership, ‘BYOD’ & social media for learning|

CLT LSE Staff Survey 2013 is open

Please help the Centre for Learning Technology to develop and improve its services in line with your needs and interests by participating in our staff survey. It should take no more than ten minutes to complete and your feedback will help to shape the team’s continuing practice.

This year’s survey seeks to identify current practice with learning technologies, staff perceptions of the value of technology-enhanced teaching, learning and assessment, and some of the learning and development needs of LSE staff.

The survey is open until midnight on 30th August to allow you to find a convenient time for completion. Your opinions and feedback are extremely valuable to us and any information you can give us will help to improve the services that we provide to you and your students.

July 29th, 2013|Announcements, Surveys|Comments Off on CLT LSE Staff Survey 2013 is open|

CLT Survey 2011

We had 165 responses to our call to take the annual CLT Staff Survey and while we always appreciate the the praise, the gratitude, the pure love, we usually concentrate on the constructive criticism to make sure we improve accordingly! Staff continue to use Moodle and are starting to make it change the ways they teach. That’s a positive result. However, the big lesson we take away from this year’s survey is that need to communicate more effectively with our academic staff and students, especially with regards to who we are and what we do. We revel in the comment that said “you guys are fantastic! Best part of IT at LSE.” The sentiment is great, but CLT aren’t actually part of IT Services, though we work closely with them. CLT support staff in the use of technologies to enhance teaching and learning. We aim to promote the integration and use of technology in teaching through staff development, advice and guidance, research, collaboration and networking. As several more respondents complained about technical hiccups in the past academic year, we realise we must make clear who we are in our next survey. Of course we will pass on these comments to the relevant departments.  What does fall into our remit is Moodle – technical support, training to use it, and advising on best practice. Since one respondent suggested they would like to be able to have blog feeds appear on Moodle, we have great news! Moodle is indeed flexible enough to do this and if you follow this link it tells you how to: http://docs.moodle.org/20/en/RSS_feeds_block! For more help with Moodle or how to use technologies to support your teaching, get in touch with us – clt-support@lse.ac.uk. To find out a bit more about what we do, who we are and which technologies we support and advise on, why not visit our webpages: http://clt.lse.ac.uk/ . We’re here over the summer and the doors are always open, so why don’t you send us an email or just pop in if you have any queries we can help you with.

plasticine models of CLT team on elephant

We are CLT

August 10th, 2011|Surveys|Comments Off on CLT Survey 2011|