What do we mean by pedagogy? (Part 2) Conceptualising assessment.

As outlined in part one of this strand of posts, I argue that we should see pedagogy as an overarching concept which positions teaching, curriculum, learning and assessment as interpenetrating systems which create a complex whole, making little sense when treated separately. Here, I want to consider how assessment might relate to other elements of pedagogy in master’s level study, particularly in education courses.

The explicit link between learning and assessment has become an increasingly accepted element of pedagogy at HE level, there is even a journal given over to the role of assessment at this level. Assessment, related to both teaching and learning, should begin at the start of a module, using diagnostic assessment to gauge and understand the complexities of prior learning within a group of students. How this is achieved is not the focus of this post, although there are a number of ways in which it can be developed. The diagnosis of prior learning is crucial as the insights gained should feed into both curriculum and teaching development, particularly challenging when working with widely diverse groups of students. By doing this, pedagogy becomes to a degree emergent, developing in response to student need rather than as prescribed by tutor assumption and preference.

Beyond initial diagnostic assessment is formative assessment, a vehicle for helping students take forward their learning, particularly through the use of targeted and well considered feedback/feed forward. The use of formative assessment is well embedded in higher educational practices, but can it be taken further? Dann (2002), working in the primary education phase, takes a view that formative assessment does not go far enough in redefining the link between learning and assessment. She argues that assessment should be fully embedded within learning, describing this as Assessent AS Learning. She sees this as a natural development of formative assessment, consisting of:

– assessment whilst teaching, leading to the directing and modification of that teaching;

– assessment by teaching, derived from an interpretation of Vygotsky’s zone of proximal development, where once the task has been set, the teacher gauges the amount and type of help required to ensure success and modifies teaching to give the greatest chance of this occurring.

This outline begins to blur the boundary between what counts as teaching and what counts as assessment. Indeed, this view of assessment in many ways suggests that the main focus is actually the process of teaching as informal and minute-to-minute reflection and assessment to inform the emergent direction and structure of the session itself. Hence, the interface between assessment and learning becomes dynamic, complex and collaborative; student involvement in assessment becomes a feature of learning.

The suggestion from the outline above is that there should be a clear synergy between teaching, the conceptualisation of learning and feedback discourses. The table below is a synthesis of the discussion developed by Askew and Lodge (2000) which links the role and approach of the teacher, the associated conceptualisation of learning, and the resultant tone for feedback discourse. To develop a feedback system where feedback is more informal (most of the time), and becomes an embedded element of learning, where a constant dialogue takes place to aid development and progress that is dialogic and iterative, a movement towards the Co-constructive end of the spectrum is a natural trend. In contrast, at the Receptive-transmission end of the spectrum, it can become necessary for staged, written input to make the ‘gift’ of tutor knowledge worthwhile and this may translate to a seminar room dynamic where grades begin to play a major role in feedback with little associated feed forward; students are told what the correct answers should be, with the degree of success mainly demonstrated through a summative indicator. The Receptive-transmission model can become more closely aligned with students playing only a passive role in extending their own learning.

Assessment

Masters level study, is in part, characterised by a drive towards increased independence and an ability to play a part in a community of inquiry. Given this intention, a Receptive-transmission approach seems out of place, and with it, more traditional forms of assessment. Instead, a more critical and appropriate approach is one which is co-constructive. But for this to be coherent there needs to be a synergy between teaching, views of learning and feedback/feed forward processes in both diagnostic and formative assessment.

References

Askew, S. and Lodge, C. (2000) ‘Gifts, ping-pong and loops – linking feedback and learning’ in S. Askew (ed.) Feedback for Learning, London: Routledge-Falmer.

Dann, R. (2002) Promoting Assessment as Learning: Improving the Learning Process, London: Routledge.

Advertisements

Embedding assessment in course design: the case of a research methods course

In my last post on designing a new approach to research methods within a masters course I adopted a framework based on the use of threshold concepts. The focus of this theoretical approach is in aiding students to move between three stages in their understanding of ‘threshold’ concepts together with attendant knowledge and skills. This is a transition between:

  • pre-liminal
  • liminal
  • post-liminal

Part of the difficulty involved in assessing student understanding and learning within this framework is the complexity involved in making explicit the level of prior learning at the beginning of the module and the subsequent trajectory which individuals follow as they transcend each level. Land and Meyer (2010) argue that assessment should focus on the conceptual difficulties which students face whilst accepting that these are a natural part of the learning process, as such they argue:

‘… if, as we maintain, the transformations occasioned by threshold concepts are important, and require a rather different way of looking at the curriculum, then it follows that such transformations will require a more nuanced and generative model of assessment to help us purposefully identify variation in progress and understanding between individual learners.’ (Land and Meyer, 2010: 63)

They make clear that each individual will move through the process of ‘liminal shift’ at different rates and in different ways and therefore it is important that assessment is used to guide and establish for both teacher and student what the difficulties and emerging understandings are within their thinking and learning. This requires both a framework for understanding learning and also a way of attempting to make that learning visible to both teacher and student. One example which is given as a medium for achieving this is the use of concept mapping, drawing on the work of Kinchin and Hay (2000) (another very detailed consideration of concept mapping is that developed by Novak, 1998). Again, Land and Meyer (2010) suggest that the use of tools such as concept mapping allow for four distinct advantages in relation to making student understanding and learning explicit:

  1. they allow us to uncover what each student knows rather than attempting to anticipate this as might be the case in any test scenario where questions are created by the tutor thereby bounding and anticipating ‘correct answers’;
  2. they allow students to demonstrate what knowledge they possess, and importantly how they have arranged that knowledge within their own minds;
  3. they become a narrative on the developing understanding of the student rather than a series of basic sequential snapshots;
  4. they allow us to see which concepts remain resistant to change within the minds of students, whilst at the same time understanding how interrelations between concepts may have changed within the students’ thinking.

Importantly, the use of various tools such as concept mapping encourage the introduction of self-explanation theory which, dialogue with the self to encourage externalising of learning (for further consideration see Chi et al (1989) http://chilab.asu.edu/papers/ChiBassokLewisReimannGlaser.pdf, and for some approaches to developing self-explanation see Hausmann et al (2009) http://www.lrdc.pitt.edu/pubs/Abstracts/HausmannSelf-Explanation.pdf). In all of this consideration is the idea of understanding learning through the lens of variation theory. Theory of Variation is based on the work of Marton and Booth (1997) which argues there is no single way to understand, experience or think about a particular phenomenon, an argument which is based upon a phenomenographic tradition. Tong (2012: 3) emphasises that:

‘In learning, individual students make sense of new concepts in different ways, according to their existing understandings and frameworks of knowledge. This requires teachers to engage closely with their students to grasp the variations in understandings and knowledge so they can take account of this diversity in structuring the learning activities in a lesson (Marton and Tsui, 2004).’

Therefore, importance is given to the ‘object of learning’ which might be a particular concept or area of knowledge. Once identified, the object of learning needs to be understood in relation to its critical features, i.e. its particular characteristics which differentiate it from any other object of learning. Consultation with students through some medium is important here so as to understand the variation in prior learning and understanding which exists prior to teaching. Having understood both student conceptualisations and the features of the particular object of learning, activities can be designed which emphasise the features which are important in aiding student understanding. Ways in which this can be developed include the use of comparison making by use of contrast, separation, generalisation and fusion. In relation to assessment the central aspect here is the ability for both the teacher and student to be able to externalise learning thereby allowing networks of knowledge and understanding to be interrogated to allow for targeted and appropriate pedagogic approaches and subject content. It is in this way the assessment can begin to take the form of an embedded and constant element of learning rather than a staged snapshot.

By bringing together threshold concepts, variation theory and the principle of assessment as learning more practical ways of tracking student understanding become possible. Approaches might include regular concept mapping, the use of reflective diaries, the development of portfolios and the possibility of more technologically led capture of self-explanation. One example of this might be the use of screen capture technology such as Jing (http://www.techsmith.com/jing.html) or Screencast-O-Matic (http://www.screencast-o-matic.com/ ). This is following on from the work of Carl Simmons at Edge Hill University who has already made very positive use of video feedback to students. Students can be asked to create a PowerPoint slide or other artefact explaining a particular object of learning within, or at the end of, a session including a narration captured whilst the screen capture software is running which is then sent to the tutor. This allows for the capture of levels of conceptual difficulty and understanding, occurring as the students are involved in their learning, and embedding the notion of self-explanation. The advantage of this approach is that each reflection is a maximum of five minutes long (Jing only allows 5 minute captures), thereby allowing tutors to judge levels of misconception or emergent understanding very quickly. If linked to the use of simple podcast feedback on the part of the tutor very rapid support can be offered in a rich feedback environment. In addition, where a large proportion of students are obviously still struggling with the concepts and knowledge involved the tutor gains a more nuanced understanding of where the problems exist and can therefore adjust the features within the object of learning to aid in more targeted teaching in future sessions.

The use of tools for capturing the trajectory of student learning allow individuals to reflect upon those areas where they have begun to feel more confident whilst also highlighting areas where understanding is still partial at best. It also allows for more in-depth and nuanced discussions between individual students and tutors as there is a richer vein of evidence for understanding student learning, misconceptions and potential next steps.

By bringing together threshold concepts, attendant bodies of knowledge and systems based on assessment as learning and variation theory, a clearer, deeper and more critical framework can be developed in support of student learning. If ‘progression’ is seen from the perspective of trajectories from pre-liminal to post-liminal states then informal assessment becomes more focused on discussions concerning concepts, knowledge, and the relational aspects between them. This then gives a far more focused approach to regular feedback to students from tutors and also offers a clear framework for ever more critical approaches to self-explanation. In addition, where formal assessment is required, in our own case through the production of a conference poster and a formal written assignment assessment, assignment outlines can use the same language as that adopted within the wider course, and the feedback provided likewise; in this sense whilst still essentially summative these formal assessments then become a far more obvious extension of the course itself.

References

Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, M. W., & Glasser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182.

Hausmann, R., Nokes, T., VanLehn, K., & Gershman, S. (2009). The design of self-explanation prompts: The fit hypothesis. Proc. 31st Annual Conference of the Cognitive Science Society. pp. 2626–2631.

Kinchin, I. & Hay, D. (2000) ‘how a qualitative approach to concept map analysis can be used to aid learning by illustrating patterns of conceptual development.’ Educational Research, 42(1), 43-57

Land, R. and Meyer, J.H.F. (2010) ‘Threshold Concepts and Troublesome Knowledge (5): Dynamics of Assessment.’ In Threshold Concepts and Transformational Learning, J.H.F. Meyer, R. Land and C. Baillie (eds), Rotterdam: Sense Publishers, 61-79.

Marton, F. & Booth, S. (1997). Learning and awareness. Mahwah, New Jersey: Lawrence Erlbaum Associates, Publishers.

Marton, F. & Tsui, A.B.M. (2004). Classroom discourse and the space of learning. Mahwah,

NJ: Lawrence Erlbaum Associates, Inc., Publishers.

Novak, J.D. (1998) Learning, creating and using knowledge: Concept maps as facilitative tools in schools and corporations. Abingdon: Routledge.

Tong, S.Y.A. (2012) ‘Applying the Theory of Variation in Teaching Reading.’ Australian Journal of Teacher Education, 37(10), 1-19 (accessed at http://ro.ecu.edu.au/cgi/viewcontent.cgi?article=1800&context=ajte)