Dr Pamela Abbott, University of Sheffield
Video
Introducing the module
The module INF6110, Information Systems Modelling, is core to three interrelated Masters programmes delivered in the Information School, University of Sheffield, UK. It regularly recruits large student cohorts of between 100 to 200 students per year. The aim of the module is “to consider the role of information modelling within the organisation and to provide an appreciation of the rigorous methods that are needed to analyse, develop and maintain computer based information systems”. The learning outcomes stress the development of practical skills in problem solving and systems modelling techniques.
Introducing the team
Team-based teaching is the accepted norm for the delivery of learning and teaching in the Information School. The teaching team for large modules usually consists of core teaching staff members supported by several graduate teaching assistants (GTAs). In 2020-21, the teaching team consisted of four core teaching staff and two GTAs as outlined in Table 1.
Table 1. INF6110 Teaching Team 2020-21
| Teaching Team Member | Teaching Team Roles | Team Member’s Position |
| Dr Pamela Abbott / Module Coordinator | Module design, coordination, delivery | Senior Lecturer in Information Systems |
| Dr Salihu Dasuki / Deputy Module Coordinator | Deputising as module coordinator when necessary | Lecturer in Information Systems |
| Dr Caitlin Bentley / Group Tsar | Developing and implementing group work support strategies | Lecturer in AI-enabled Information Systems |
| Dr Efpraxia Zamani / PLA Lead | Developing and implementing PLA strategies at the programme level | Senior Lecturer in Information Systems |
| Mr Haoyu Xie / Group Work Support Assistant | Overseeing and reporting on group work activities | PhD Student/GTA |
| Dr Jun Zhang / Tutor | Running practical sessions and designing coursework solutions | Former PhD Student/GTA |
Since the inception of the module, peer learning has been a strong pedagogical element. Graduates of the MSc Information Systems programme would be expected to demonstrate group working skills since so much of the work done in professional IS settings is based around team working. The design of the assessment therefore typically involves teams of students working together on modelling tasks based on a bespoke case study developed particularly for the learning outcomes of this module. The tasks enable students to practice skills in all of the modelling techniques and paradigms introduced in the module, e.g. soft systems analysis or business process modelling.
Introducing the pedagogy
Several group-based components are thus traditionally part of the assessment, e.g. collaborative work on producing one or more aspects of information systems modelling and a reflective report on the team process. In the pre-Covid face-to-face setting, groups would meet in the labs, perform their practicals, bond in doing this work collectively at various points of the module, and then work on a large group report at the end of the module. Various methods of assessing group and individual performance within teams were also employed, e.g. peer assessment techniques, group meeting logs and monitored attendance of group tutorials and practicals. Non-contribution of team members was also assessed as part of the marking process.
As a core module on the Information Systems programme, INF6110 provides foundational knowledge to IS students who progress through the rest of the programme. The module does enrol mixed cohorts, however, some of whom take INF6110 as a core elective. Prior to 2020-21, it was noted that some students enrolling into another core IS module INF6001, Information Systems Project Management, in Semester 2, were underperforming compared to those who had taken INF6110 in Semester 1. There were thus ensuing consequences for the cohesion of the cohort and maintaining a consistent learning and teaching approach across the programme in Semester 2.
The Learning and Teaching Innovations
In many universities in the UK, the Covid-19 pandemic has created the conditions for workarounds and innovations in the delivery of learning and teaching. In 2020-2021, the University of Sheffield, in line with other Russell Group universities, adopted a blended online/on-campus approach towards ensuring the continued delivery of quality learning and teaching. Modules recruiting large cohorts, such as INF6110, were selected for special support. Module leaders were encouraged to use digital technologies to innovate and offer an equivalent on-campus learning experience in an online environment. The module recruited 148 students, many of whom were Chinese nationals who were unable to attend campus-based tutoring due to travel and government restrictions both in the UK and abroad.
The immediate challenge to the module team was to ensure an equivalent student experience as compared to a pre-Covid environment and to enable on-campus and off- campus students to participate equally, using the resources available to the module. The University’s enabling strategy around learning and teaching ensured that innovative thinking could be easily accommodated, thus three innovations were introduced into the module, all enabled through digital technologies to create a unique digital pedagogy for the delivery of the Information Systems Modelling module. The objectives of the innovations were as follows:
- To enable continuity and consistency of delivery in the IS programme between core Semester 1 and 2 modules by adopting a programme level assessment (PLA) approach;
- To provide alternative online-only group-based collaborative processes to ensure group cohesion and effectiveness;
- To devise learning and teaching methods to support the delivery of online only team-based approaches to learning and teaching equivalent to in-presence models.
We describe the three innovations below. Common to all of the innovations was the overarching use of digital technologies. We thus propose the innovations as a digital pedagogy for the delivery of the module. Given that student engagement would be our main issue in an online-only environment, we discuss the innovations within that context.
Driving student engagement at the curriculum level: a joint cross-module assessment approach
With Covid restrictions in place, it was even more important to provide cohesion and continuity across the IS programme in 2020-2021. The blended approach meant that group ‘bubbles’ could potentially be implemented as a Covid social-distancing strategy should students return to on-campus teaching. Hence, the continuity of groups formed in Semester 1 and continuing into Semester 2 was of vital importance to a safe and successful student experience.
The PLA lead on the team, who is also the IS programme coordinator, therefore proposed new programme regulations to make INF6110 a prerequisite for INF6001. This enabled the team to develop a shared assessment and to plan that groups already created and working together in Semester 1 would continue into Semester 2. This approach is in line with the University’s PLA strategy and with IS curriculum capstone courses such as those promoted by the MSIS 2016 task force[1]. Such pedagogical innovations create the space for core content delivery from which targeted student-specific tailoring can successfully proceed, e.g. through elective modules.
The module coordinator and PLA lead set up objectives for a shared joint case study, which was based on the ongoing real-world failing NHS test and trace system operating in the UK. The team created a portfolio of documentation which mimicked the outcomes of a real-world systems analysis to support the case study. All the documents, together with the coursework brief, module outline and pre-recorded content explaining the coursework were made available to students on the module’s virtual learning environment (VLE), Blackboard. Once we had set up the technical and procedural infrastructure for cross-module cohesion and continuity, the other two innovations were intended to reinforce this to ensure the more long-term advantages that we expected.
Driving student engagement at the module level: an online-only group process
It was clear that with no reliable or consistent method of ensuring in-presence opportunities for group members to meet face-to-face, the majority of their collaborations would take place online. Thus, a key concern for the module coordinator was determining which aspects of the group process should be retained in an online-only learning and teaching environment. Her first step was to consult with the School’s Teaching Committee on best practices for online group work and out of this came the idea to appoint a Group Tsar to develop group work support material for online and on-campus delivery. The newly appointed Group Tsar also devised with the module coordinator a group process retaining aspects of group assessment and group dynamics normally expected in the pre-Covid situation. The innovations thus introduced were: (1) group contracts to replace group logs, minutes and peer assessment ratings and (2) peer assessment to be revamped to include active listening and engagement using one of the assigned modelling tasks.
In addition, a specially appointed GTA would support and monitor all group-related activities including:
- Sending reminders of milestones and deadlines and following up on missing information
- Monitoring of group activities in the specially created VLE areas for group work
- Providing feedback on group contracts and peer assessments
- Keeping a log of each group’s progress through their activities and reporting to the Group Tsar and Module Coordinator.
Driving student engagement through pedagogy: digital innovations in teaching team-based modes of delivery
The University had issued general guidelines to module teams regarding how learning and teaching delivery could be achieved in 2020-21 using synchronous and asynchronous methods. For example, for large-cohort lectures, both online synchronous lectures and asynchronous pre-recorded sessions were advised. On-campus sessions with smaller cohorts and groups were also recommended where possible for reinforcing aspects of online delivery. In INF6110, we made use of all three modes: asynchronous for flipped learning, synchronous for seminars and Q&A on-campus/online sessions.
The challenge in INF6110, however, was in reproducing practical and tutorial sessions that best deployed the teaching team’s capabilities and ensured an equivalent or superior student experience online. For this, a mix of synchronous and asynchronous modes was needed and hence the “sandwich” innovation was devised. Online practical sessions were designed as “closed” sandwiches, while online tutorial sessions were designed as “open” sandwiches, where a sandwich refers to the online learning design of the session.
Essentially the “bread” of the sandwich was the synchronous part of a session and the “filling” was the asynchronous group work that the groups could arrange to do in a mode that worked for them. For example, many held their own meetings, worked on collaborative documents, or communicated via text messages before re-convening for a synchronous session where they could share and learn from each other and the tutor. These sandwich elements would be staggered throughout a teaching week to allow the students adequate time to bond and do their work in their groups.
A Digital Pedagogy for Online-Only Group-based Assessment and Team Teaching
The innovative techniques described above can be conceived as a digital pedagogy consisting of (a) supportive but not constraining digital technologies (b) a collaborative learning and teaching ethos for both the learner and the teacher and (c) hybridised pedagogical elements. These three elements of the pedagogy are further explained below.
Digital technologies
All synchronous and asynchronous elements and group activities were supported through the Blackboard VLE. The VLE platform was used for distributing all assessment materials, pre-recorded videos, module documentation and short explanatory videos of module content. Blackboard Collaborate was used for large-cohort synchronous sessions. The VLE platform provided group work support for file exchange, email, discussion boards, blogs etc. However, groups were encouraged to establish their own patterns of working online together through their group contract and could use whatever platforms they preferred for their group and course work. Hence, there was widespread use of collaborative software, e.g. those provided by Google Suite, and collaborative social media, e.g. WeChat, together with traditional office suite software such as MS PowerPoint and Visio.
Collaborative learning and teaching
The module team also leveraged digital collaboration to create its own learning and teaching ethos, as shown above. We heavily relied on online collaborative tools for module coordination. In addition, other digitally enhanced innovations were emerging because of this ethos. For example, our approach to marking also included innovative elements enhanced by the online-only environment. An annotated marking guide was developed and distributed through collaborative software. We had several online sessions to compare our marking, moderate in between marking and reflect on our feedback to ensure consistency, fairness and transparency, all of which contributed to exceptional student feedback.
Hybridised pedagogy
The pedagogy for the module, which includes peer learning and assessment through constructive alignment, was not fundamentally changed, but hybridised with the introduction of the online elements. Traditional face-to-face activities meant to achieve pedagogical outcomes were re-crafted through the intervention of digital technologies. This meant re-imagining how these elements would work, attempting creative solutions through trial and error and finding ways to evaluate staff and student impacts. We report on these in the following section.
Outcomes
In this section, we summarise both our observations of the impact of this pedagogy on the learners and our reflections on our learning and teaching practice.
Impact on Learners – Observable Outcomes
The module benefited from high student engagement throughout the semester, with most group members attending and engaging in the very busy discussion boards set up to capture their ongoing communications about their work. The first full-cohort synchronous lecture in week 1 of the semester attracted 89% of the cohort while the final one, in week 10 attracted 87%, a drop-off of only 2%. Attendance at the practical sessions were either at full capacity or just below (the lowest recorded attendance is at 96% of the cohort).
The module received exceptional student feedback both from the informal mid-semester review, which the team undertook and from the formal end-of-semester university-led module feedback. In the informal mid-semester feedback, students voted the module the best in the semester, and commended some of the group-based activities we designed for them, e.g. the group contract. At the same time, they did admit that the pace of the weekly turnaround of activities could be quite stressful. In the formal feedback, the module scored highly in all quantitative categories of evaluation and outperformed the department, school and faculty averages in all of these categories. The module consistently scored at least 4.5 or 4.6 out of 5.0 for these categories.
In addition, student performance on the module was on par with previous years; preliminary results show a similar class average, with 27% in the higher percentiles. This is especially significant given the lack of face-to-face teaching opportunities.
Impact on Learners – Reflections on Practice
Returning to the three objectives that underpinned these learning and teaching innovations, our reflections are:
- The PLA approach enabled reduced cognitive load on learning case study materials across the two modules for greater focus and deeper engagement with the programme level learning outcomes, especially in the project management module. Consistency and connectivity were achieved since it was easier to connect what they were learning to employability concerns, since they gained experience in both modelling and project management aspects of information systems development.
- The online collaborative group process helped to engender more consistent student engagement. Online delivery makes it difficult to establish a culture of student engagement, however, the group work component enabled students to participate in ways that they were accustomed to in an offline world. Moreover, the necessity of co-producing outputs that they had to share with peers and tutors, i.e. to take responsibility for, pushed them a little out of their comfort zones.
- The team-based approach to online delivery provided flexibility for both the learners and the teaching staff. The learning design (‘open’ and ‘closed’ sandwiches) allowed learners to independently establish modes of group work and study patterns in different time zones that worked for them and their own contexts. Teaching team members could sharpen their expertise in one or two topic areas, and lead those areas, while sharing in open, collaborative ways their solutions and guidance to other teaching staff and students. It was a learning experience for all.
Conclusion
We strongly believe in the efficacy of the digital pedagogy we introduced to deliver blended learning in this core IS module in 2020-21. In future work, we would like to integrate our lessons learnt into the growing literature on digital pedagogy.
[1] See: https://www.acm.org/binaries/content/assets/education/msis2016.pdf