What do we mean by pedagogy? (Part 4) Thinking about Curriculum 2

In my last post I suggested that the masters framework developed by the QAA for HE in Scotland (2013) offered a very useful basis for developing pedagogy. I also argued that curriculum needs to be more than a list of content, instead seeing the roles of emergence and process as crucial to the work of masters students and therefore in designing curricula. In developing this perspective on curriculum, how might a practical framework look?

priciples of curriculum design

This model is an attempt to capture the complexity and process orientation towards curriculum which is informed by the work of Knight (2001). This model starts from a position of seeing knowledge as a central element of any curriculum. Knowledge is the building blocks on which debate and argumentation are based. Therefore, it is a crucial element in constructing any curriculum. However, by itself it is not enough. Of equal importance is the structure which supports these building blocks – the explicit discussion of concepts. Threshold concepts (Meyer, Land and Baillie, 2010) have become a useful basis for developing the overarching framework for a course, and indeed modules (whilst accepting that in any given module threshold concepts for many may remain liminal). At masters level there is every chance that students will move from a core area of knowledge to pursue and specialise in particular spheres within a module. The explicit use of threshold concepts allows this process to occur within a coherent, wider ‘field’ of study; whilst individuals may begin to investigate different subject areas and contexts the concepts ensure a level of coherence and allow a common point of contact for discussion and engagement with the work of others. The use of an explicit conceptual framework also gives a general scheme for the process of learning to operate within. In this sense, the interplay of a conceptual schema (boundary settings) with individual investigation and growth (freedom) is in keeping with Davis and Sumara’s (2006) identification of factors necessary for emergence in their work on complexity theory.

If knowledge and concepts are engaged with alone however, then there is a deficit in the applied/practical use of the emerging learning. Hence, application is also important as this is where schema, a developing knowledge base and understanding are utilised and ‘tested’.

It is at the intersection of the three dimensions of knowledge, concepts and application where curriculum as process (Knight, 2001; Stenhouse, 1975) can be made real. Together, they give the possibility for emerging understanding (here used in the way I’ve interpreted Van Camp (2014) to emphasise the connection of ideas and knowledge in networks) and application based on engagement with knowledge, concepts and their application. It is in this emerging interpenetration (Byrne and Callaghan, 2014) of these systems that both new insights and new knowledge can emerge. But at this level, this is a personal journey for each student with different contexts, interests and applications driving learning. Hence, curriculum as product (Stenhouse, 1975) makes little sense as the possible outcomes are hugely diverse whilst still operating within a loose framework and from common starting points (For an example of how this model for curriculum has been used in our work so far see an earlier post & our research methods pedagogy website)

As suggested in earlier posts, to understand and develop curricula where diversity and process are key, we need to have a clear understanding of the role of assessment in aiding the emergent process model, but as I’ll also reflect upon in future posts, interpenetration has major ramifications for the way we understand learning and teaching; to reiterate, to suggest that pedagogy can be the study of teaching alone makes little sense.


Byrne, D. & Callaghan, G. (2014) Complexity Theory and the Social Sciences: The state of the art. Abingdon: Routledge.

Davis, B. & Sumara, D. (2006) Complexity and Education: Inquiries into Learning, Teaching, and Research. New York: Routledge.

Knight, P.T. (2001) ‘Complexity and Curriculum: A process approach to curriculum-making.’ Teaching in Higher Education, 6(3), 369-381.

Meyer, J.H.F., Land, R. and Baillie, C. (2010) Threshold Concepts and Transformational Learning, (eds), Rotterdam: Sense Publishers.

Quality Assurance Agency for Higher Education (2013) What is mastersness? Discussion Paper. Retrieved from: http://www.enhancementthemes.ac.uk/docs/report/what-is-mastersness.pdf [Last accessed 5/7/15]

Stenhouse, L. (1975) An Introduction to Curriculum Research and Development. London: Heinemann.

Van Camp, W. (2014) ‘Explaining understanding (or understanding explanation).’ European Journal of Philosophy of Science, 4(1): 95-114.

Teaching Research methods – Some initial reflections

On March 20th, we finally finished teaching a research methods module which is a core element of our MA International Education (MAIE) course. As I outlined in this blog last autumn (here, here and here), I have been working with a colleague in the School of Education to develop a new approach to our research methods course.

Having finished the course and the data collection we have captured from a parallel research project of our own on the module, it feels like a good time to consider some initial reflections about our work. This is obviously an initial perception, we need to spend many months analysing the very rich dataset we have collected. Any reflections can’t be taken as a detailed and accurate account. However, several issues seem to have emerged across the module:

1) Thinking about threshold concepts. As we began to develop a curriculum framework we discussed possible threshold concepts in research methods as a basis for instructional design. In a past post, I listed threshold concepts identified by Kiley and Wisher (2010). They saw the threshold concepts relating to research methods as being:

  • argument
  • theory
  • frameworks
  • knowledge creation
  • analysis
  • paradigm

We started from this point, but through discussion emphasised the following concepts as being both central to understanding research methods and also having the potential to be transformatory. Consequently, our list of threshold concepts became:

  • criticality
  • theory
  • methodology
  • ethics
  • analysis
  • epistemology/ontology

In the event, we spent less time on theory as a concept than we had expected, but all of the other concepts became a major part of the course. In student interviews criticality was seen as central to developing an ability to read research and from this to writing well considered and careful texts. Methodology and analysis were also seen as being important for assessing papers as well as being central to a critical and deep understanding of how to carry out research. One student reflected that previously she had read the ‘start and end’ of papers to engage with the main messages; now she first engages with the ‘middle’ to assess the degree to which the research could be used or trusted. Ontology and epistemology were the most difficult concepts to tackle and at the end of the module I would argue that some are still in liminal space in this respect. Some students reflected that at undergraduate level the nature of reality and knowledge, as well as paradigms, were assumed and hence never discussed. As an interdisciplinary pursuit education needs to engage in these debates as researchers from many different traditions meet at this particular crossroads and there is therefore a level of philosophical complexity. Methodology, analysis and ethics were all equally important in aiding students to gain a deeper and holistic understanding on which to base their expanding knowledge and practical experience.

One additional concept which we had not included in our original list but which I would be minded to include having completed the module is that of ‘sampling’. Some struggle with this and yet good understanding often acted as a basis for logical, well considered and critical bridges between methodology and data collection tools. Where sampling was not well understood this bridge was less, if at all, secure and logical explanation of research design began to default to general description and a lack of criticality.

2) Importance of language. We have started to see the research methods module more and more as a language course. This is not only the result of developing a course which predominantly attracts international students, although this is obviously important. We have a number of English speaking students and yet they often commented on the difficulty of engaging with the language. Research methods language is conceptually rich and difficult; we are teaching this language and regardless of student origin, we need to ensure that students understand the language and the concepts underlying it.

3) Research methods as an applied activity. In our planning, we also developed a pedagogic model which sees conceptualisation, knowledge and application as equally important, and intertwined.

understanding elements of learning for a master's RM programme

At the end of the module I feel this is a very useful framework and has aided in developing a critical approach to the module. Conceptualisation is vital as a basis for constructing and developing knowledge. However, where these began to really make sense for students was when they actually enacted their ideas. The application of research methods started from day one of the course and revolved around two practical exercises. Firstly, students acted in pairs to consider the characteristics of good interviewing before developing a set of group research questions based on a research problem given to them by ourselves. From the research questions they discussed and agreed interview questions before splitting into pairs to complete their interviews. Once complete, the pairs then transcribed and encoded their data. This process shadowed their work in face-to-face sessions and therefore their emerging understanding of the module. The final exercise focused on comparing codes across the group to identify re-occurring themes as well as outliers.

Students then moved on to complete a module assignment which asked them to develop an area for research, develop research questions from this, before creating a research design which was then piloted. Subsequent to the pilot, students were then asked to reflect on their experience and how they would change their research design as a prelude to developing their dissertation work.

This proved very challenging, but also, according to some of the students, allowed them to consider how far their understanding of research methods developed.

I am currently discussing the potential for a new master’s degree focusing on praxis-based approaches to education. Having developed our work on research methods I fully intend to embed an emergent element of research methods across all modules of the programme, leading towards a specialist research methods module. Research methods needs to be engaged with over a period of time and within different contexts to give a wide critical and experiential basis for discussion and theoretical understanding.

These are some of the basic reflections from the course, but as I said above, these are only initial and need to be considered in far more detail as we begin to engage with the very large amount of data we have collected from this course. In my next post, I will continue my reflection, by considering the process of researching this module and the utility of considering the learning environment as being a complex adaptive system.

Designing a complex curriculum-reflections on knowledge, understanding, concepts and skills

If we are to develop an emergentist curriculum, as suggested in the last post, we need to make room for the emergence of meaning within the seminar room. But in defining a process of meaning making as emergent we cannot have ready-made goals, other than perhaps a loose field of interest within which we construct our work (i.e. the link between coherence and freedom in Davis and Sumara’s (2006) conditions for emergence). One critique that might be made of this approach is that it could lead to a form of ‘radical relativism’ with individuals following any direction they feel is warranted and ending up with very little to show for their endeavour. However, this is to fundamentally misunderstand an emergentist agenda. The coherence element as a foundation for for emergence ensures limits to the field of interest, but within this admits freedom. In addition, by questioning the meaning which individuals develop, as suggested by Osberg and Biesta (2008), the teacher is required to use information and knowledge to challenge thinking and understandings through a mixture of appropriate pedagogic strategies. Thus the goals of the curriculum might not be set closely, but this does not mean knowledge is not sought. I see knowledge as central to the emergence of meaning, but how that knowledge is understood and how it also emerges in the individual needs consideration.

Knowledge is central to any curriculum. But if this is the case then knowledge itself needs defining. The definition of knowledge at one level can be very simple, being the facts, information, and skills acquired through experience or education. However, this hides a very complex area of debate as the search for a definition of knowledge is a central strand of philosophical study and over millennia, has not managed to create a definitive statement which all can agree on, and which stands the test of philosophical scrutiny. The definition of knowledge is also made even more complex by the debate as to the degree to which it stands apart from, or acts an overarching term for, the notions of ‘concepts’, ‘understanding’, and ‘skills’. Each of these terms can be taken as a subset of knowledge (as a concept!). However, how they relate is again a contested area.

Van Camp (2014: 97) sees understanding is a type of knowledge, but nevertheless feels it important to distinguish it as an explicit idea, as he states,

‘To a large extent, much of the aversion to giving understanding any philosophical prominence comes from conflating concepts simply because of linguistic poverty.’

There is a debate over whether understanding is a form of knowledge or something different, and definitions of understanding themselves vary. For example, Kvanvig (2003:192) states

understanding requires the grasping of explanatory and other coherence-making relationships in a large and comprehensive body of information. One can know many unrelated pieces of information, but understanding is achieved only when informational items are pieced together by the subject in question.’

Likewise, Zagzabski (2001: 241) defines understanding as

‘involves seeing the relation of parts to other parts and perhaps even the relation of part to a whole.’

Both of these definitions see understanding as more than basic knowledge. It is characterised by a qualitatively different aspect, the development of a structure within knowledge which is relational. Van Camp (2014) suggests that this view of understanding is, therefore, incremental, and an individual can have more or less understanding depending on the degree to which relational connections have been made. He then goes on to argue that understanding is central to our development of causation,

On my account of understanding, information is better understood if it fits into that network of knowledge, and in tension with fundamental causal beliefs if it does not. So, while causation is not necessary for understanding in principle (other types of explanation, such as unification, can make connections in our knowledge), as a fundamental-perhaps native-worldview, phenomena which are not fitted to a causal framework remain conspicuously outside a comprehensive body of information, and thus not fully understood.’

Therefore, whilst understanding might be a form of knowledge, I would argue that it makes sense to retain it as differentiated from knowledge as a concept as it emphasises the explicit purpose of denoting the links and developing network of knowledge which we gain as we learn.

A simple diagrammatic way of showing this is


Concepts are likewise difficult to define. At a very simple level concepts can be defined as mental representations of classes of things (Murphy, 2004) inside the head. Mead and Gray (2010) develop this simple definition by considering how concepts might be understood within the wider context of ‘threshold concepts’. They consider the form and role of concepts within disciplines, emphasising the difference between private and public conceptions (or mental representations). They differentiate between the concepts we have inside our own heads, which are prone to change, and those which are shared (disciplinary) and which tend to be much more stable as change here requires negotiation and debate. They see concepts is providing the ‘underlying logic’ (p.99) used to develop and structure knowledge. Perkins (2006) in his discussion of troublesome knowledge uses Foucault’s notion of ‘episteme’ (any historical period’s way of configuring knowledge), referring to ‘a system of ideas or way of understanding that allows us to establish knowledge.’ (p.41-2). Concept is therefore positioned as a logical framework or system which allows us to structure knowledge in a way that supports and promotes understanding. Concepts by this definition become the foundation on which we structure and make sense of knowledge and understanding. As such I argue that they should also be the basis for building curricula. To add to the diagrammatic structure given above, concepts can be seen as underpinning knowledge and understanding.


Finally, there is the issue of skills. Skills again can be defined as knowledge – procedural knowledge, which is the knowledge exercised in the performance of a task. What is important here, regardless of the term used is the idea of application. Skills/procedural knowledge is concerned with the performance of something, be it driving a car (rather than just knowing how a car works), or being able to successfully search for information; procedural knowledge is therefore of a different form of knowledge when compared to declarative knowledge (knowledge about something).

In developing and enacting an emergent curriculum, I will define ‘knowledge’ as equating to declarative knowledge, which is made increasingly useful by the relational growth of understanding. How these nodes and relationships are given a structure occurs through the underpinning power of disciplinary concepts which provide the overarching logical framework for disciplinary knowledge and understanding. Finally, given that I have retained the term knowledge to refer to declarative knowledge, I use the term ‘skill’ rather than procedural knowledge to refer to the application of knowledge, understanding and concepts. Therefore, in developing the terms of an emergentist curriculum, the following conceptual diagram becomes a useful structure for thinking about the detail in developing such an approach.


In the next post, I will consider some of the practical ramifications of defining these processes in the way presented here, and how they interact with notions of curriculum and assessment to give a coherent approach to programme development.


Davis, B. & Sumara, D. (2006) Complexity and Education: Inquiries into Learning, Teaching, and Research. New York: Routledge.

Kvanvig, J. (2003) The value of knowledge and the pursuit of understanding. Cambridge: Cambridge University Press.

Mead, J. & Gray, S. (2010) ‘Contexts for Threshold Concepts (1) A conceptual Structure for Localizing Candidates.’ In J.H.F. Meyer, R. Land and C. Baillie (eds.) Threshold Concepts and Transformational Learning. pp. 97-113.Rotterdam: Sense Publishers.

Murphy, G. (2004) The Big Book of Concepts. London: The MIT Press.

Osberg, D. &Biesta, G. (2008) ‘the emergent curriculum: navigating complex course between unguided learning and planned enculturation.’ Journal of Curriculum Studies, 40(3): 313-328.

Perkins, D. (2006) ‘Constructivism and troublesome knowledge.‘ in J. Meyer and R. Land (eds.) Overcoming barriers to student understanding: Threshold concepts and troublesome knowledge. Pp. 33-47. Abingdon: Routledge.

Van Camp, W. (2014) ‘Explaining understanding (or understanding explanation).’ European Journal of Philosophy of Science, 4(1): 95-114.

Zagzebski, L. (2001) ‘Recovering understanding’ in M. Steup (ed.) Knowledge, truth and duty, pp.235-251. Oxford: Oxford University Press.

Troublesome knowledge: what is sauce for the goose is sauce for the gander?

Over the past 12 months, I’ve consciously started to move my own research agenda away from working on school-based projects to those focusing on curriculum and pedagogy in higher education contexts. This has been an exciting time as higher education provides a fertile ground for developing innovative pedagogical approaches based upon a notion that lecturers can be trusted to create, develop and execute modules and courses which relate to their subject expertise. The past 18 months has been the first time in my professional life that Ofsted has not cast a long and negative shadow over my professional autonomy and opportunities for innovative practice.

In re-orientating my pedagogical and, as a consequence, research interests, I have also started to attend a different set of conferences. I’ve just got home from my first attendance at the annual conference of the Society for Research in Higher Education which has been a very positive and thought-provoking experience. There has been a very wide-ranging set of presentations from considerations of organisational leadership and governance in universities, through developing reactions and alternatives to policy, international work, the work of lecturers and student experiences, to utopian perspectives concerning the futures of universities, learning technologies and digital universities as well as teaching and learning. Because my own research interests centre on pedagogy, curriculum and increasingly research methods, I have spent much of the last three days listening to and discussing issues as wide-ranging as the use of concept mapping to understand student conceptualisation of master’s dissertations (Dr E Buyl), and the opening up of the ‘Space of Reasons’ (Dr G Hinchliffe) (based on the work of John McDowell) as an alternative to understanding what is essentially a case of ‘epistemic access’ as outlined in Plato’s cave.

One session was presented by Ray Land who developed an interesting perspective on threshold concepts and troublesome knowledge as it focused on the role these concepts can play in disrupting and counteracting neoliberal discourses within the University. I’ve used these ideas in my own research for more than five years to help in developing curriculum models and pedagogies at both school and university level. Because their work is focused on the troublesome knowledge which students face in making sense of their studies as they begin to move towards crossing thresholds at a conceptual level I have tended to view this theoretical perspective in the same way. However, perhaps due to my relatively recent adoption of a focus on higher education pedagogy linked with the opportunity to listen to a lot of interesting and theoretically rich perspectives across the three days of the conference, it dawned on me that in understanding both my own emergent subject knowledge and my continued emergence and growth as a teacher troublesome knowledge and threshold concepts are as relevant to my own changing thinking and practice as it is for helping the development of knowledge and understanding in students.

In one of those moments where two ideas come together, I happened to be giving a presentation on the use of Lesson Study in higher education, in particular considering our current research into research methods pedagogy. One technique which we are using in this research is to ask students to create concept maps at the end of each session, which they are then asked to record a short audio commentary about before sending both files to us so that we can begin to gain another perspective on their emerging understanding. In some recent interviews a couple of the students talked about the importance of the concept maps and explanations in helping them to begin to get a more conscious understanding of the degree to which they do or don’t understand important principles and concepts within research methods. Having discussed this in the seminar with academic colleagues, it dawned on me that we all have conceptual and knowledge-based schemata that however knowledgeable or expert we are within a field, always offer new thresholds and troublesome knowledge with which to engage. By considering my own work and research in this way it is beginning to help me reframe the degree to which risk, grappling with the unknown and long periods of uncertainty and hard graft are as important for me as a researcher and teacher as it is for the students with whom I work. How often do we tend to inhabit a space which feels comfortable and from which we can feel a sense of authority, rather than searching out new areas of troublesome knowledge through which we can stumble towards new thresholds in our own understanding? Perhaps there is as much to gain from thinking about our own knowledge and work in this way as there is in considering how we engage students in their learning.

Thinking Through Learning and Research – 2

In the first post on thinking though learning and research I briefly outlined how we define the process of learning (through the work of Knud Illeris), as well as some of the contextual assumptions we make. In this post, I want to explore how these foundations translate into a conceptual model and from there to the curriculum model outlined at the end of the first post and which is augmented here.

Our first theoretical standpoint is that we are dealing with a complex adaptive system (as outlined in the first post) which leads to the acceptance that the processes involved are non-linear, interact in unpredictable ways and are therefore emergent in nature. We are also assuming that student learning rests on developing ever more complex and detailed schemata relating to research methods. These develop and coalesce around a small number of threshold concepts (which I first discussed here). The threshold concepts we believe a master’s level course should address are:

  • Criticality (in reading and writing)
  • Theory
  • Methodology
  • Analysis
  • Epistemology/Ontology/World View
  • Ethics

These are central to understanding, designing and competing small-scale research projects. They are therefore the basis for laying a strong foundation for those advancing to doctoral-level study.

A schema, or schemata, will emerge and coalesce around these concepts. These will then, hopefully act as an emerging framework for critical engagement with published research in the form of both empirical research and the research methods literature itself. In addition, the framework will be the basis for the practical application of these ideas and for the development and completion of small-scale research activities and projects which will eventually culminate in a master’s dissertation.

In considering the threshold concepts at the core of the course development, we decided at an early stage that we needed to give ourselves and students time to engage with both content and practical application. To do this we want to be able to introduce areas of research methods, for example methodology and its various forms, or ethics in a critical and in-depth way. In introducing each area we need to continually build links so that knowledge development is situated in both a wider schematic of research methods, whilst also being deeply rooted in relation to concepts. As a consequence, all research methods sessions will last for a minimum of one day (Figure 2).

By having longer, but less frequent sessions, time is given for introduction and discussion of new knowledge, the development of understanding of the links of that knowledge to concepts, as well as a consideration of practical application and use. This then suggests a set of inter-related processes (figure 1), such that

understanding elements of learning for a master's RM programme

The diagram in Figure 1 is an initial attempt to create a framework for research methods learning. It takes the growth of knowledge, the understanding of threshold concepts through liminal processes of thought, discussion, and reflection, and the application through the enhancing of practical skills as a holistic model of emerging research ability. All three elements are vital, and need to be intertwined to bring a critical understanding and practical ability in carrying out research. Each element is important for if there is any element missing there is less than a holistic approach. If students are given knowledge and told to apply this, then they may see application as a simple set of tick lists, a ‘mechanistic’ application as the knowledge instilled will tend to be technical in form where it is not underpinned by conceptual understanding. This also means that research quality will be compromised as when difficulties arise, or alternative approaches need to be developed in a particular context, the lack of deep, conceptual underpinning will lead to less flexibility and possibly to the use of inappropriate approaches.

Likewise, if concepts are explicitly discussed, but are not linked to a breadth and depth of knowledge, any link to application will be weak as a knowledge-base is important for practical application. Finally, the cross-over between concepts and knowledge is where I would place the recent surge of interest in ‘research literacy’. Here, engagement with the conceptual framework of research methods, together with a developing knowledge of approaches and examples, will lead to an emerging theoretical engagement and understanding. However, it will be devoid of practical skills in application and the ‘messiness’ of research as it is planned and executed. This is not a major weakness for those wanting to engage with the research of others, but will mean that some of the messiness inherent in research is not clearly or critically understood.

It is where all three elements of learning are focused on and developed that critical understanding and application will emerge. However, this has to be seen as an iterative process, one which extends beyond the end of any level of formal learning and training. It is the interplay of these elements which, over very long periods of research activity, lead to individuals who can be identified as ‘experts’ in research.

Taking the intertwined development of knowledge, concepts and application as the core of a research methods course, we have developed an approach which tries to engender these principles. As a result, the course has the following form (Figure 2).

RM course final

The course covers each of the main conceptual areas, beginning with a consideration of what we actually mean by research and moving forward to cover each area in turn, thus creating a ‘research methods’ narrative. These build on each other, and once an element has been covered it will be enfolded into later discussions. No session is less than a day long, and some are longer. For example, research design and tools is a 3 day session. This allows for revisiting of previous knowledge and conceptual bases, and the incorporation of research design as a further element of successful research planning. Having introduced these elements more formally, the length of the session will give time for discussion, planning, creation and problem-based and discovery learning. The result will be the development of a draft research design for a dissertation, as well as a research tool to be trialled as a pilot.

As well as the knowledge and conceptual bases being developed in the face-to-face and online materials, an application strand will start from early in the course. Initially this will take the form of pairs of students carrying out semi-structured interviews with established researchers to investigate their views concerning research, preferences of research approach, as well as some idea of research career history. Results will then be shared later in the course. Having completed this component, students will then need to carry out individual piloting of a research tool which they have created based on their work in the course. This will give them an opportunity to develop and critique research tools in a supported and structured environment. The data from both exercises will then act as the core for a two day session on analysis tools making use of the authentic data sets collected by the students themselves (supplemented where necessary to ensure engagement with both quantitative and qualitative analysis).

The development of application, knowledge and conceptual understanding throughout the course will culminate in the completion of a dissertation. By developing all strands together in a holistic approach, this will be the ‘acid-test’ of how well the students have developed a deepening, critical and positive potential in carrying out their own small-scale research.


The final post of this strand will consider how we can capture a useful understanding of the various elements and processes in the course.

Embedding assessment in course design: the case of a research methods course

In my last post on designing a new approach to research methods within a masters course I adopted a framework based on the use of threshold concepts. The focus of this theoretical approach is in aiding students to move between three stages in their understanding of ‘threshold’ concepts together with attendant knowledge and skills. This is a transition between:

  • pre-liminal
  • liminal
  • post-liminal

Part of the difficulty involved in assessing student understanding and learning within this framework is the complexity involved in making explicit the level of prior learning at the beginning of the module and the subsequent trajectory which individuals follow as they transcend each level. Land and Meyer (2010) argue that assessment should focus on the conceptual difficulties which students face whilst accepting that these are a natural part of the learning process, as such they argue:

‘… if, as we maintain, the transformations occasioned by threshold concepts are important, and require a rather different way of looking at the curriculum, then it follows that such transformations will require a more nuanced and generative model of assessment to help us purposefully identify variation in progress and understanding between individual learners.’ (Land and Meyer, 2010: 63)

They make clear that each individual will move through the process of ‘liminal shift’ at different rates and in different ways and therefore it is important that assessment is used to guide and establish for both teacher and student what the difficulties and emerging understandings are within their thinking and learning. This requires both a framework for understanding learning and also a way of attempting to make that learning visible to both teacher and student. One example which is given as a medium for achieving this is the use of concept mapping, drawing on the work of Kinchin and Hay (2000) (another very detailed consideration of concept mapping is that developed by Novak, 1998). Again, Land and Meyer (2010) suggest that the use of tools such as concept mapping allow for four distinct advantages in relation to making student understanding and learning explicit:

  1. they allow us to uncover what each student knows rather than attempting to anticipate this as might be the case in any test scenario where questions are created by the tutor thereby bounding and anticipating ‘correct answers’;
  2. they allow students to demonstrate what knowledge they possess, and importantly how they have arranged that knowledge within their own minds;
  3. they become a narrative on the developing understanding of the student rather than a series of basic sequential snapshots;
  4. they allow us to see which concepts remain resistant to change within the minds of students, whilst at the same time understanding how interrelations between concepts may have changed within the students’ thinking.

Importantly, the use of various tools such as concept mapping encourage the introduction of self-explanation theory which, dialogue with the self to encourage externalising of learning (for further consideration see Chi et al (1989) http://chilab.asu.edu/papers/ChiBassokLewisReimannGlaser.pdf, and for some approaches to developing self-explanation see Hausmann et al (2009) http://www.lrdc.pitt.edu/pubs/Abstracts/HausmannSelf-Explanation.pdf). In all of this consideration is the idea of understanding learning through the lens of variation theory. Theory of Variation is based on the work of Marton and Booth (1997) which argues there is no single way to understand, experience or think about a particular phenomenon, an argument which is based upon a phenomenographic tradition. Tong (2012: 3) emphasises that:

‘In learning, individual students make sense of new concepts in different ways, according to their existing understandings and frameworks of knowledge. This requires teachers to engage closely with their students to grasp the variations in understandings and knowledge so they can take account of this diversity in structuring the learning activities in a lesson (Marton and Tsui, 2004).’

Therefore, importance is given to the ‘object of learning’ which might be a particular concept or area of knowledge. Once identified, the object of learning needs to be understood in relation to its critical features, i.e. its particular characteristics which differentiate it from any other object of learning. Consultation with students through some medium is important here so as to understand the variation in prior learning and understanding which exists prior to teaching. Having understood both student conceptualisations and the features of the particular object of learning, activities can be designed which emphasise the features which are important in aiding student understanding. Ways in which this can be developed include the use of comparison making by use of contrast, separation, generalisation and fusion. In relation to assessment the central aspect here is the ability for both the teacher and student to be able to externalise learning thereby allowing networks of knowledge and understanding to be interrogated to allow for targeted and appropriate pedagogic approaches and subject content. It is in this way the assessment can begin to take the form of an embedded and constant element of learning rather than a staged snapshot.

By bringing together threshold concepts, variation theory and the principle of assessment as learning more practical ways of tracking student understanding become possible. Approaches might include regular concept mapping, the use of reflective diaries, the development of portfolios and the possibility of more technologically led capture of self-explanation. One example of this might be the use of screen capture technology such as Jing (http://www.techsmith.com/jing.html) or Screencast-O-Matic (http://www.screencast-o-matic.com/ ). This is following on from the work of Carl Simmons at Edge Hill University who has already made very positive use of video feedback to students. Students can be asked to create a PowerPoint slide or other artefact explaining a particular object of learning within, or at the end of, a session including a narration captured whilst the screen capture software is running which is then sent to the tutor. This allows for the capture of levels of conceptual difficulty and understanding, occurring as the students are involved in their learning, and embedding the notion of self-explanation. The advantage of this approach is that each reflection is a maximum of five minutes long (Jing only allows 5 minute captures), thereby allowing tutors to judge levels of misconception or emergent understanding very quickly. If linked to the use of simple podcast feedback on the part of the tutor very rapid support can be offered in a rich feedback environment. In addition, where a large proportion of students are obviously still struggling with the concepts and knowledge involved the tutor gains a more nuanced understanding of where the problems exist and can therefore adjust the features within the object of learning to aid in more targeted teaching in future sessions.

The use of tools for capturing the trajectory of student learning allow individuals to reflect upon those areas where they have begun to feel more confident whilst also highlighting areas where understanding is still partial at best. It also allows for more in-depth and nuanced discussions between individual students and tutors as there is a richer vein of evidence for understanding student learning, misconceptions and potential next steps.

By bringing together threshold concepts, attendant bodies of knowledge and systems based on assessment as learning and variation theory, a clearer, deeper and more critical framework can be developed in support of student learning. If ‘progression’ is seen from the perspective of trajectories from pre-liminal to post-liminal states then informal assessment becomes more focused on discussions concerning concepts, knowledge, and the relational aspects between them. This then gives a far more focused approach to regular feedback to students from tutors and also offers a clear framework for ever more critical approaches to self-explanation. In addition, where formal assessment is required, in our own case through the production of a conference poster and a formal written assignment assessment, assignment outlines can use the same language as that adopted within the wider course, and the feedback provided likewise; in this sense whilst still essentially summative these formal assessments then become a far more obvious extension of the course itself.


Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, M. W., & Glasser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182.

Hausmann, R., Nokes, T., VanLehn, K., & Gershman, S. (2009). The design of self-explanation prompts: The fit hypothesis. Proc. 31st Annual Conference of the Cognitive Science Society. pp. 2626–2631.

Kinchin, I. & Hay, D. (2000) ‘how a qualitative approach to concept map analysis can be used to aid learning by illustrating patterns of conceptual development.’ Educational Research, 42(1), 43-57

Land, R. and Meyer, J.H.F. (2010) ‘Threshold Concepts and Troublesome Knowledge (5): Dynamics of Assessment.’ In Threshold Concepts and Transformational Learning, J.H.F. Meyer, R. Land and C. Baillie (eds), Rotterdam: Sense Publishers, 61-79.

Marton, F. & Booth, S. (1997). Learning and awareness. Mahwah, New Jersey: Lawrence Erlbaum Associates, Publishers.

Marton, F. & Tsui, A.B.M. (2004). Classroom discourse and the space of learning. Mahwah,

NJ: Lawrence Erlbaum Associates, Inc., Publishers.

Novak, J.D. (1998) Learning, creating and using knowledge: Concept maps as facilitative tools in schools and corporations. Abingdon: Routledge.

Tong, S.Y.A. (2012) ‘Applying the Theory of Variation in Teaching Reading.’ Australian Journal of Teacher Education, 37(10), 1-19 (accessed at http://ro.ecu.edu.au/cgi/viewcontent.cgi?article=1800&context=ajte)

(Re)designing a research methods module for an education master’s

Over the past five years the MA that I have taught on has grown and gone from strength to strength. However, whilst most modules have become increasingly enriched, one module stands out as being a constant issue – research methods. In a full-time campus-based course research methods has a number of potential problems to overcome. It can sometimes feel that it is somewhat segregated from the rest of the course, and is often the first time our students (many of whom are international) have completed a module solely focusing on research methods. We’ve tried a number of different approaches, mainly focused on alternative ways of developing an in-course research project. However, I think there are wider issues in developing research methods within an education master’s degree centring on:

  • Differences between undergraduate experiences, particularly due to the different disciplines students have pursued and the epistemological traditions of different national backgrounds
  • How master’s degree provision maps onto doctoral studies. I think that master’s research methods courses are often ‘doctoral-lite’ with lots of breadth but less depth.
  • How emerging understanding is captured in practical application
  • How a core of knowledge, understanding and skills can be developed which are well embedded in groups which have a huge diversity of prior learning and cultural diversity with respect to research in general

This can lead to the development of a ‘Cook’s Tour’ approach where there is a list of content to get through which sketches out the vast majority of the research methods ‘oeuvre’ but which also leaves little time for real engagement and understanding. So how might a course be reconsidered and on what lines? This depends on what we want students to get from the experience of a research methods course. Do we just want them to have lots of knowledge, in which case the Cook’s Tour might be satisfactory, if of limited applied use, or do we want something different? I am going to argue for three main aims in a research methods course at master’s level:

  1. A clear and critical understanding of the ‘core’ of research methods at a conceptual level
  2. An emerging understanding and application of these concepts at a practical/applied level
  3. A good foundation on which to build at doctoral level whilst also providing a practical foundation for application beyond the academy for those who don’t further their studies.

This has led me to reconsider the development of a course using the idea of threshold concepts, explained as:

‘A threshold concept can be considered as akin to a portal, opening up a new and previously inaccessible way of thinking about something. It represents a transformed way of understanding, or interpreting, or viewing something without which the learner cannot progress. As a consequence of comprehending a threshold concept there may thus be a transformed internal view of subject matter, subject landscape, or even world view. This transformation may be sudden or it may be protracted over a considerable period of time, with the transition to understanding proving troublesome. Such a transformed view or landscape may represent how people ‘think’ in a particular discipline, or how they perceive, apprehend, or experience particular phenomena within that discipline (or more generally). (Meyer and Land, 2003: 1)

We are attempting to support individuals in becoming informed researchers, in transforming the way they think and act in relation to concepts such as evidence, argument and method. This cannot occur, I would argue, by simply presenting information, it can only occur through deep conceptual engagement. It is the difference between learning about research and becoming a researcher. One caveat is that I would argue that any one year course will not enable an individual to become an expert researcher, it will, at best, allow them to ‘cross the threshold’ into the world of research. Beyond this is a huge journey to grow and gain critical experience and understanding of the complexities by carrying out research over a long period of time. Becoming an expert researcher is a difficult, sometimes painful, and slow process.

In deciding the core threshold concepts on which to base a course, Kiley and Wisker (2010) offer a starting point. They set out a series of threshold concepts they believe are core to ‘learning to be a researcher’ (the title of their paper) through doctoral studies. They set out the following key concepts:

  • Argument
  • Theory
  • Frameworks
  • Knowledge creation
  • Analysis
  • Paradigms

These generally track onto a list which I had created before reading this paper after attending a Higher Education Academy social sciences conference which focused in part on research methods pedagogy.

concepts table final

With these core threshold concepts as the backbone of the course, the knowledge which is covered becomes the explication of those concepts. However, the concepts and the knowledge which overlays them need to be made concrete and need to be applied, and therefore research tools also need to be covered so as to allow the practical enactment of the emerging understanding which students gain. In a sense, the application of their learning is central to deducing for them and us the degree to which they have managed to grapple with the ‘troublesome knowledge’ (Perkins, 1999) of research methods.

In an earlier post (https://hereflections.wordpress.com/2014/05/12/thinking-about-course-innovation-and-evaluation-i/) I made it clear that I see curriculum, assessment and pedagogy as inseparable. And over coming posts I will consider the implications of this model for both assessment and pedagogy, but in terms of the curriculum element, the above argument leads me to a conceptualisation of a research methods curriculum as:

curriculu conceptulisation

Consequently, the basic curriculum map for the redesigned course is that given below:

RM outline

There are no longer any two hour seminars, one of the main features of the current course. Instead we have decided to have mainly whole, or multiple days at points across the year. This is to allow time to engage in different ways and at length. It also allows for more time for student engagement with texts and activities beyond the seminar room. Some core work will also be developed online as ways of revisiting and strengthening core threshold concepts such as epistemology and methodologies. Part of the research tools work in January will be used to develop a small-scale research project which students will need to pilot. What is central to the thinking here however, is that the ‘breadth’ of the course has been diminished to make way for depth and space to consider, tackle and play with troublesome knowledge and the core foundations of research methods. It is designed to create liminal spaces of difficulty and challenge to allow students to play and grapple with ideas as a way of helping them understand the nature of the thresholds they are attempting to cross and aid them in doing so. However, it is essential that approaches are found to link the various elements together explicitly to give a holistic picture, we need to guard against the insight from Eraut (2008, quoted in Land and Meyer, 2010: 75),

‘People always think that if you go into enough detail about something you’ll nail it. But you never can, and you lose sense of the whole context in which that something makes sense. You lose the big picture.’

It is important that in grappling with threshold concepts, we don’t lose sight of the bigger picture. This is where ‘assessment’ becomes important, and I am keen to develop an ‘assessment as learning’ model (see Dann, 2002) which attempts to

‘follow the movie of the personal journey rather than look at snapshots of it.’ (Land & Meyer, 2010: 65)

But more of that in the next post.


Dann, R. (2002) Promoting Assessment as Learning: Improving the Learning Process. Abingdon: Routledge.

Eraut, M. (2008) Research into Professional Learning: Its implications for the professional development for teachers in Higher Education. Unpublished seminar paper. HEDG Annual Conference, Madingley Hall, Cambridge, UK.

Kiley, M. and Wisker, G. (2010) ‘Learning to be a researcher: The concepts and crossings.’ In Threshold Concepts and Transformational Learning, J.H.F. Meyer, R. Land and C. Baillie (eds), Rotterdam: Sense Publishers, 399-414.

Land, R. and Meyer, J.H.F. (2010) ‘Threshold Concepts and Troublesome Knowledge (5): Dynamics of Assessment.’ In Threshold Concepts and Transformational Learning, J.H.F. Meyer, R. Land and C. Baillie (eds), Rotterdam: Sense Publishers, 61-79.

Meyer, J.H.F. and Land, R. (2003) Threshold Concepts and Troublesome Knowledge: Linkages to Ways of Thinking and Practising within the Disciplines Enhancing Teaching-Learning Environments in Undergraduate Courses, Occasional Report 4, May 2003. Accessed at: http://www.colorado.edu/ftep/documents/ETLreport4-1.pdf [last access: 25/5/2014]

Perkins, D. 1999. The many faces of constructivism. Educational Leadership, 57(3), 6–11.