Sense-making – moving from quality assurance to quality growth

HE is seemingly being ever more exposed to the use of metrics. This is most obviously the case in the current development of the TEF (Teaching Excellence Framework) where often vague, tangential datasets are to be used as a measure of the quality of teaching. One of the stated purposes of such frameworks is to offer institutions insights into how to improve their processes. However, one of the main problems with the use of any type of summative evaluation is that it might offer insights into patterns, and hence ‘what’s’, but has little to offer in terms of ‘how’ or ‘why’. Evaluations can also begin to pervert the processes they are intended to improve as they become part of the ‘accountability-complex’,

The paradox is that the accountability fervor meant to assure performance can have direct and indirect consequences that undermine it.’

(Halachmi, 2014)

A number of different approaches to programme and module evaluation have started to emerge, including the inclusion of student perspectives. Here, I outline a view which takes this as a starting point and considers the potential of a theoretical framework called Normalization Process Theory (which was developed within the health and social care area) to help develop holiploigic practice. The process I am currently developing starts from the definition of sense-making of Klein, given by Snowden,

‘Sensemaking is the ability or attempt to make sense of an ambiguous situation. More exactly, sensemaking is the process of creating situational awareness and understanding in situations of high complexity or uncertainty in order to make decisions. It is “a motivated, continuous effort to understand connections (which can be among people, places, and events) in order to anticipate their trajectories and act effectively.’

This stresses the ongoing nature of sense-making in an attempt to understand the evolving complexity of a context. In the case of a masters module, sense-making becomes a process of understanding experiences and perceptions of students as their work develops within a module, rather than waiting until the end of the module to gain retrospective perspectives.

To develop a framework for sense-making, I suggest here the use of Normalization Process Theory (NPT). This theory might not always work for sense-making activities, but where the focus is on embedding a change or process, it is ideal. NPT was developed by May and Finch (2009) as a way of understanding and assessing innovational change in health and social care contexts. It distinguishes between implementation (a relatively straight forward process), and normalization such that the innovation or change becomes embedded (a very difficult shift to achieve). It is in the gap between implementation and normalization that ‘zombie innovation’ (Wood, 2017) occurs, senior leaders believing that organizational proclamation leads to embedded day to day practice. But often, such proclamations merely lead to initial implementation, followed by ‘compliance under surveillance’ – i.e. the change will be present in official documents and assurance of practice, but not in day to day work. NPT is structured into four elements:

1.      Coherence –  this is the element of using a new practice which involves understanding how the new practice is different to what is currently done, and also being able to clearly understand and operationalize the aims and objectives of the new practice.

2.      Cognitive participation – this is the work individuals do to develop a collaborative approach to the change which is being undertaken. Are they able to create a successful community of practice?   

3.      Collective action – this relates to the resourcing and collective work done by a community to embed practice. It includes the development of new knowledge, understanding how the facets of a change can be brought together and generation of new practices.

4.      Reflexive monitoring – this is the appraisal work a group and individuals do to understand the processes and outputs of a change, as well as considering how localized changes might be developed further to ensure successful embedding of new practice.

By using these elements to sense how learning across a module is developed, it might be possible to understand how learning and skills are becoming embedded as the module is being experienced. This leads to the potential for changes and development in real time.

The example here is a module in an MA International Education programme. Early in the course all students complete a core research methods module. This module allows students, many of whom have never encountered research methods in their prior university experiences, to gain a foundation across approaches in education. An assignment concludes the module, based on asking students to create a research project plan, before piloting a single data collection technique and evaluating it.

Whilst the research methods module offers a positive initial experience of research methods, it is a large jump from this to a dissertation study of 20,000 words. As a consequence, those students undertaking an optional pathway in innovation and reform in education are asked to complete a research module (30 credits). They work in pairs to develop and complete a small-scale research project based on an issue relating to innovation and/or reform. Sessions are led as group tutorials, covering and developing issues the students feel they need further help with, as well as reporting back to the group on a regular basis to discussion ideas, plans and execution.

Given the challenging nature of the project for many, whilst it would be possible to evaluate the module at the end of the process, it would be far better to sense-make throughout the module. Therefore, given that the nature of the project is to help students embed new practices as they move towards their dissertation, I am currently beginning to think about the potential for NPT to act as a positive framework. The intention is to use four short questionnaires at points over the course of the module, followed by a focus group on each occasion as a way of understanding the nature of student learning and practice development. The first questionnaire, focused on issues of coherence, is given below as an example,      

The intention of this phase is to ensure that the students understand what the purpose and aims of the research project are. If students do not understand this then we are building on a poor foundation from the very start of the process. By investigating this early on I can work with the students to sense the level of confidence, knowledge and conceptual understanding on which they can baser their work in the coming weeks.

Once we have started this process, in a couple of weeks’ time, we will move on to consider and develop a sense of participants’ emerging work together, and the degree to which the taught sessions are helping them become part of a wider research community.

References

Halachmi A (2014) Accountability Overloads in M Bovens, R.E. Goodin & T. Schillemans, The Oxford Handbook of Public Accountability, Oxford University Press.

May C and Finch T (2009) Implementing, embedding, and integrating practices: An outline of Normalization Process Theory. Sociology 43(3): 535–554.

Wood P (2017) Overcoming the problem of embedding change in educational organizations: A perspective from Normalization Process Theory. Management in Education 31(1): 33-38.

 

Innovating teaching and learning in HE: developing an HE studio

Teaching and learning are increasingly seen as central to the work of universities, particularly with the introduction of the TEF. One of the unfortunate aspects of this emerging emphasis is the over simplification (or complexity reduction) of the processes involved, as league tables, metrics and quality assurance systems kick in. This is a shift which occurred in the schools system a couple of decades ago, and has ultimately led to overly simplistic perspectives concerning the work of teachers, driven by an overbearing accountability system. As Halachmi (2014) states:

‘The paradox is that the accountability fervor meant to assure performance can have direct and indirect consequences that undermine it.’

Teaching and learning are elements of a much wider and very complex set of nested systems. Many universities now have a dedicated institute or learning development wing which has the responsibility for developing teaching. These departments have a crucial role in developing practice, and helping academic departments in taking teaching and associated activities forward. However, universities are becoming increasingly complex organisations, and are required to meet many agendas which are both internally and externally driven. This suggests the need for an ever wider perspective on teaching.

A complexity orientated perspective would suggest that any attempt to gain a deep understanding of teaching and learning approaches, together with the creation of innovative practice, needs a broad, transdisciplinary approach. This insight has led me to the idea of an ‘HE Studio’. In the diagram below, some of the main issues such a Studio would consider are identified. They are presented in concentric rings to reflect the idea that many of the issues of interest are interdependent but exist at different scales. For example, to consider the role and nature of assessment (defined not only by the assessments undertaken by students, but sense-making and evaluation of programmes etc) not only are other processes at this scale implicated (teaching, learning and curriculum) but processes and issues at larger scales. Assessment will be impacted by organisational policies and aims, by the use of technology, and above this, by government policy decisions and, on occasion, external partnerships. This means that to develop well-considered and robust teaching and learning environments, we need to develop holistic approaches to understanding and evidencing the web of processes which contribute to seminar room practice. In addition, such a Studio would also develop innovative practices based on practical insights and wider evidence-bases. But to do this well would require a transdisciplinary approach.

A Studio would need to capture diverse forms of evidence. Because of the interacting scales of processes involved, it would be necessary to develop qualitative, quantitative and mixed methods approaches to research. A range of perspectives would be important, from small-scale ethnographies and case studies, through programme-wide mixed methods approaches to larger-scale ‘big-data’ analysis at both organisational and sector-level scales. The development of fully transparent structured literature reviews would also play a role. I have chosen the word ‘Studio’ because the purpose of the research would be to help create foundations for innovation. It would not be a ‘laboratory’ as this would suggest a purely experimental approach, which whilst it might offer useful insight, would be deficient if used as the sole evidential base – it might be said to be necessary but not sufficient. Neither would it be an observatory as it would not be intended only to observe, measure and report. Instead, these would be an element of a wider set of practices, which aim to give rich, transdisciplinary insights which can then be used as the basis for introducing and refining new practices. Here, action research, design-based research, and where appropriate, quasi-experiments would become central.

The defining aim of an HE studio would be to consider, synthesise and create new practices in an emergent context. Process, experimentation, innovation and emerging insights would be the core focus of such work. In some quarters there appears to be an attempt to encourage the idea that teaching and learning are simple, easily defined processes which can be made efficient and understood through the use of a restricted set of (mainly) quantitative approaches. It seems to me that this ignores the inherently complex set of processes involved in teaching and learning, and the ecology of influences around them. The idea of a Studio approach is to put a varied spectrum of evidential bases at the heart of innovative development by accepting that useful insights can occur from a range of research traditions. It is how the evidence is synthesised and used as a basis for practical innovation which is important.             

Making the Case for a Centre for Change Studies

At the beginning of Duncan Green’s book How Change Happens, he makes the point that within universities there is no dedicated area researching and teaching about change,

‘It turns out they [academic disciplines] each operate with separate and often conflicting theories of change and there is no ‘department of change studies’ to sort it out.’  (Green, 2016)

As an employee of Oxfam, he commissioned a report considering how different disciplines understand the nature of change, it is well worth a read. The idea of a Centre for Change Studies makes a great deal of sense.

Green, as with an increasing number of academics, has found the use of complexity theory a useful lens in understanding the emerging processes inherent in change, as well as offering insights in how to influence and affect change. Complexity centres have already started to spring up in various guises within the university sector, the most famous being the Santa Fe Institute. These centres thrive on the interdisciplinary nature of their work, bringing different perspectives to bare on a single area of interest. These different perspectives, coming as they do from disciplinary starting points, offer both mutually supportive expertise and critical insight from many directions.

So why a Centre for Change Studies? Presently, a series of issues and processes are emerging which may fundamentally change the way in which we perceive and act in the world. The classification of the Anthropocene, worsening climate change and its often non-linear consequences, globalisation, social acceleration, the rise of artificial intelligence, medical and genetic advances, socio-economic inequalities within and between countries, the rise of post-democratic political systems, and signs of the beginnings of chronic resource depletion. These processes are suggestive of the possibility of radical and rapid and complex change. Many of the solutions and accommodations to these problems will and must emerge from innovations within specialist disciplines. However, understanding the processes of change and how we can understand and manage them require a specialist field drawing on expertise from many perspectives. We also need to help develop professionals to become sensitive to the complexities and processes of change, understanding these ideas and issues in ways that allow them to work more critically and assuredly within their own contexts.

At a time when the utility of academia is being called into question in some quarters, the development of a medium for researching, understanding and mitigating against the impacts of faster and more acute change processes seems like a good place to start in reconnecting with issues which have major practical and political implications.      

Some Initial Forays into Holiploigic Planning

I’m currently involved in getting ready for teaching two modules on a masters course after the winter break, and given my inability to just pick up sessions and resources from a previous year, I decided to work with a colleague to reimagine and redevelop the module content and approaches for a specialist pathway focusing on Innovation and Reform in Education. Because we are strong believers in praxis, we thought it would be a great opportunity to explore some of the elements of an outline for holiploigic development I have pinged across my twitter stream previously, shown again below.

The thinking behind this schematic is to try to bring a deeper approach, and explicit philosophical perspective, to curriculum planning. Curriculum must be seen as interpenetrating with the other complex systems of teaching, learning and assessment, all of which need to be allied with serious discussion about of elements of the process which revolve around and underpin, such as media of activity (face-to-face, blended, distance). We haven’t been able to spend as much time as we might with a new module, in part because we have little time between now and the start of teaching. However, yesterday we spent an interesting couple of hours developing the front end of the reimagined first module. In a two hour meeting we managed to get as far as a consideration of the interaction between curriculum-teaching-learning. The first 50 minutes were an underpinning discussion of what our aims and objectives are for the module. Usually, we might focus discussion on the learning outcomes, but these are in a sense performative and mechanistic – we wanted to think about what the deeper rationale of the experience should include, and decided that underpinning the module were a desire to help students:

1)     Understanding common/different educational issues which are important around the world

2)     Debate aims/definitions of education

3)     Define innovation as opposed to invention

4)     Understand models and perspectives on change – linked to processes & scales

5)     Further develop notions of criticality – notions of questioning and reflecting about experiences and research evidence

Having considered these, we also went on to discuss what our own values and philosophies are as teachers in our own discipline and what we feel are important approaches to curriculum, teaching and learning. We see this as discussion as crucial, as it stresses and emphasises the authenticity of our practice (see Kreber, 2013 and later posts). Here, after some discussion we decided we want to emphasise:

1)     The importance of disciplinary knowledge and understanding as a basis for developing applied insights

2)     Social learning

3)     Independent study 

4)     The development of a ‘narrative curriculum’ 

5)     Students as emergent researchers 

Having started to develop a foundation for our consideration of curriculum etc, we also wanted to think about the ‘personas’ we might be working with – a simple way of thinking about the variety of experience, expertise etc that students might bring with them as well as the things they might be hoping to gain from engaging with the module. This led to a list, possibly quite simple, but potentially useful for consideration of experiences:

1) Group A – no teaching experience, non-education degrees, experience of education predominantly as students.

2) Group B – previous teaching experience shaped within a particular context, but with little educational theory underpinning experience.

3) Group C – Passionate about a topic they want the programme to address and help them develop as an expertise  – particularly focused on SEND – some have practical experience or even familial/personal experience of their chosen passion.

It was only once we had discussed and captured these starting points that we started to think about the structure, focus and intertwining processes of the curriculum, teaching, learning and assessment (formal and informal). At the end of two hours, we had started to sketch out some of these elements of the process also deciding on some of the environments we would like to use in the module. At the moment we have only sketched out some of these ideas quite loosely as is shown below.

At the end of a two hour meeting, we had started to think about teaching and learning experiences, but assessment had not even been mentioned. We thought about narrative – how do we develop something that emerges, the idea of coherence, i.e. a bounded curricular space with structure whilst allowing enough freedom for individuals to pursue personal and professional interests (based on ideas from Davis and Sumara, 2006).

We still need to go away and work on the detail, we only have a shell of ideas and a loose structure. We are each taking elements of the emerging module to ‘play with’, before bringing them back together next week to begin to ‘stitch’ into a coherent whole. Two further elements we are taking away with us to reflect on and consider – what new elements of practice are we bringing to the discussion and what research evidence will we need to engage with to help develop those new practices, and during the teaching of the module what research insights do we need to help in the process of sensemaking as the module experiences of both teachers and students emerge?

References

Davis, B. & Sumara, D. (2006) Complexity and Education: Inquiries into Learning, Teaching, and Research. New York: Routledge.

Kreber, C. (2013) Authenticity in and through teaching in higher education : the transformative potential of the scholarship of teaching. London: Routledge.

 

 

An initial session on discussing the form of holiploigy

Holiploigy – an initial outline

As I stated in my last post, the idea sitting behind holiploigy (and hence its name) is the notion of the inherent complexity of the processes involved in helping students learn and become expert in their chosen fields of study. This involves a need to engage with, understand, apply and relate an ever larger body of concepts, knowledge, skills and competences, through a wealth of different media (the ‘holi’ – or holistic/complex). Working with students to enable this holistic understanding requires a wide range of approaches, some of which are more lecturer-led, others leading to independent and discovery-driven experiences; this wide spectrum of lecturer activity can be described as being analogous to navigation (ploigos).

Any attempt to capture the complexity of holiploigy in its entirety will fail, as will hopefully become clear as I consider some of its possible dimensions. This is because it is context driven, and will involve almost limitless combinations of activities. Different disciplines, under- and post-graduate, face-to-face, blended, or distance learning, will all lead to different practice. What appears below is an attempt to capture some of the possible central elements which act as fundamental features and processes. Such complexity reduction is bound to occur as any exhaustive model of a complex system would need to be as complete as the system itself which is not possible. However,

‘Just because a complex system is incompressible it does not follow that there are (incomplete) representations of the system that cannot be useful – otherwise how would we have knowledge of anything, however limited? Incompressibility is not an excuse for not bothering.’

Richardson and Tait (2010: 92-93)

An initial representative schematic of holiploigy is given below

At the centre of this conceptual framework is the interplay of a number of elements (1) which take into account assessment (2),  curriculum (3) (4), learning (5) (6) and teaching (7). These elements are experienced and shaped by the actions of tutors and students, and evolve over time, both in relation to each other, but also as a consequence of their interactions with the processes and characteristics which are positioned around this central field. Having discussed some ideas concerning learning, teaching, curriculum and assessment and their interaction in past posts, in this series I will focus on the sectors which lie around the periphery of the diagram above, and will also consider how all the elements of the holiploigic framework might play out in different practical contexts.

References

Richardson, K.A. & Tait, A. (2010) ‘The Death of the Expert?’ E:CO, 12:2, 87-97.

 

Navigating the complexity of education in universities – arguing for holiploigy

Introduction

In a number of previous posts I’ve tried to set out a loose framework for understanding how we might conceptualise the process of teaching, learning, etc in higher education. These posts were based on the idea that to argue for a discussion about ‘teaching and learning’ such as that in Scholarship of Teaching and Learning leads to a conceptual narrowing of the task at hand. Instead I proposed a simple diagram to outline a complex process:

I argued that we should move away from ‘teaching and learning’, and back to a reformed notion of ‘pedagogy’ (1) which takes into account assessment (2),  curriculum (3) (4), learning (5) (6) and teaching (7). As such I was calling this a form of ‘complex pedagogy’ due to the idea that each of these processes was, in their own right, complex, with their interpenetration making them all the more complex. I still think that this premise is correct for work in higher education, but the use of ‘pedagogy’ still concerned me; I was quite rightly challenged by someone who argued that pedagogy, by definition, focuses on the education of children. So what are the alternatives?

If we think about the meaning of ‘pedagogy’ it is actually composed of ‘paidos’, male child in ancient greek, and ‘agogos’, meaning to lead, so pedagogy means to lead a child. Doesn’t seem quite the right conceptualisation for working with young adults in undergraduate and postgraduate environments.

Two other terms which are used to describe teaching situations are ‘andragogy’ and ‘heutagogy’. Andragogy, comes from ‘andras’ man, leading to the ideas of teaching adults, i.e. leading men, and heutagogy relating to self-determined, student-centred, or discovery learning. In all these cases there is the notion of people being led – even heutagogy still refers to this.

As a result of reflecting on these ideas, I have decided that we need to think differently about the relationships between teaching, learning, curriculum and assessment, and between lecturers and students together with the terms of the spaces (virtual and real) in which such activities and relationships take place.

Outlining holiploigy

The concept of ‘holiploigy’ attempts to capture two fundamental aspects of work in higher education. The ‘holi’ element relates to the idea that the process of higher education needs to be considered holistically, and as a series of interpenetrating complex adaptive systems. This philosophy acts at a number of scales, and across a series of ideas. Firstly, there is the idea of the complexity of knowledge and skills within a domain, and increasingly their links across domains (inter- and trans-disciplinarity). Secondly, as laid out above, it includes the idea of teaching, learning, assessment and curriculum being inextricably linked, and of a complex nature (with lecturers and students at the intersection of the four). However, around this is the complexity of learning environments and how these processes operate across them. Teaching, learning etc operate differently in a face-to-face context when compared to being online, and yet increasingly, such blends will occur within a single course. How are the complexities of this to be understood and navigated?

And this leads to the idea of ‘ploigy’, from ploigos – navigate. Agogos, as used in pedagogy, suggests a role for the lecturer as leader, being at the centre of the educative process. At higher education level, this should not be the case – all of the time. However, if we see the lecturer as merely a guide – we might begin to move towards a process of ‘learnification’ (Biesta, 2012) which is potentially damaging. Biesta (2015) suggests the need for the teacher to be more central to the process of teaching and learning, but in a way that offers an opening up rather than a narrow leading. Navigating can be thought of as a process which sometimes needs more direct action, especially when moving through complex, dangerous and difficult waters. But at other times, such navigation requires less direct intervention, and can allow for much greater freedom, whilst still being a journey with a purpose. In some cases a journey might allow for detours, extra investigations of interesting, new places, but all the time the crew and navigator are working together to chart a meaningful course. And all the time, the navigator is inculcating the crew into the art of navigation for themselves.

Therefore, over the next few posts, I’ll outline what I see as a conceptual framework for the idea of navigating the complexity of the educative process and the knowledge and skills which it is used to explore, the process or holiploigy.

 

Biesta, G.J.J. (2012) ‘Giving Teaching Back to Education: Responding to the Disappearance of the Teacher.’ Phenomenology & Practice, 6 (2), 35-49.  http://www.ul.ie/eps/sites/default/files/Biesta%202012.pdf

Biesta, G.J.J. (2015) ‘The Rediscovery of Teaching: On Robot Vacuum Cleaners, Non-Egological Education, and the Limits of the Hermeneutical Worldview’. Educational Philosophy and Theory http://bura.brunel.ac.uk/handle/2438/10587

Education – a complexity perspective

An Initial schematic- complexity and education

This schematic is a first rough expression of trying to understand education from a complexity perspective. I’ll develop the explanation in the next few days, but the main structural character of the diagram is that concepts relating to complexity (blue) and reductionism (green) are picked out in colour. This is an attempt to position them as underlying concepts which are core to understanding how the ideas covered in the main diagram (black) interact.

David Snowden on Complexity