Traditional Assessment vs Authentic Assessment

Based on an article by John Mueller.

Authentic Assessment is defined as

"a form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills"

Traditional Assessment (TA) includes things such as multiple-choice tests, true-false, matching and so forth. The idea behind this is that students must possess a certain body of knowledge and skills, courses must teach this body of knowledge and skills, students must then be tested to see if the course is successful. So, the curriculum drives assessment - the body of knowledge comes first, that becomes the curriculum, and the tests see if students acquired the curriculum.

Authentic Assessment (AA) is built on the assumption that students should be able to perform tasks in the real world, courses must help students become proficient in those tasks. So, to determine the success of the course students perform meaningful tasks that replicate real world challenges. In this case assessment drives the curriculum - first determine what the tasks are that need to be performed, then develop a curriculum that enables students to perform the task well including the acquisition of essential skills and knowledge.

How does this work in subjects that are not practical, or do not require performance? For example how do we use AA in History, or Sociology for example? We ask students to perform tasks that replicate the challenges faced by people doing history, or conducting social research. In sociology the rubric may make reference to, for example, Sociological knowledge, Sociological thinking, and Sociological research skills.

It seems that TA is more useful for formative assessment, while AA is used for summative assessment. Students should be able to perform well in both types of test - TA provides a good complement to authentic assessment.

What are the attributes of TA and AA?

Traditional --------------------------------------------- Authentic

Selecting a Response ------------------------------------Performing a Task

Contrived -------------------------------------------------Real-life

Recall/Recognition ------------------------------- ------Construction/Application

Teacher-structured --------------------------------------Student-structured

Indirect Evidence ----------------------------------------Direct Evidence

AA will use verbs that are towards the top of Bloom's taxonomy - students will be asked to 'analyse', 'synthesise' and apply their learning. AA allows students more choice in what to focus on and what to present as evidence of their learning. TA is more prescriptive. There are often multiple routes to a good answer.

I'm not sure I like the term 'Authentic Assessment'. It suggests that TA is not authentic, which is wrong. TA has its uses in formative assessment. I'd prefer to use one of the alternative titles suggested at the bottom of the article - Alternative Assessment, or Performance Assessment.

Writing Effective Online Discussion Questions

This is based on the CREST+ model outlined by Lynn Akin and Diane Neal.

The ability to participate in group tasks is an important variable in the success of an online course. It encourages a sense of community and is more likely to improve engagement. Discussion forums provide a good way to encourage participation, but it requires skill in asking the right sort of questions. Akin and Neal argue that the CREST+ model provides a framework for creating effective questions which lead to greater participation and a higher level processing of the course material.

The CREST+ model looks at the Cognitive nature of question, the Reading basis, the Experiential possibilities and the Style and Type of question.

Cognitive Nature

There are a range of learning theories and models on which to base questions, such as Constructivism, Androgogy, Bloom's taxonomy. With a a Constructivist approach the students builds meaning based upon the course content. Questions can be structured to reflect increasing complexity. Gilly Salmon's Five Stage model of online learning uses this approach, increasing the student's interactivity and collaboration via carefully constructed questions at the different stages to facilitate the process. Knowles's Androgogy looks at how adults learn and proposes that they want to know why they are learning, need self-direction and want to be responsible for their own decisions, and they bring their life experiences to the course with them. Questions should be constructed that address these needs and help them to learn what will help them in their lives. Bloom's taxonomy, updated by Anderson, ranks enquiry types into a hierarchy. Each level builds upon the other and the student moves to complex understanding and knowledge. The types of question that could be asked would be based on the different levels and where the student was at on the hierarchy. The hierarchy is shown below, with its updated version.



Each level has a set of terms that can be used to build questions, which are available from a wide range of sources online.

In summary the first step in building questions is for the tutor to decide the best type of question based upon the cognitive needs of the students and the desired learning outcomes. The aim is to encourage participation and engagement from the outset. Higher participation and engagement leads to increased cognitive presence, which enables students to construct meaning through sustained communication, and to engage in critical reflective thinking.

Reading

Many courses will have a text book which is a shared resource for the students. Initial questioning can be based upon the shared textbook. It is important to scaffold the questions so that student, online at least, can arrive at more complex understanding together. So for example the forums would be separated, first would be one which concentrates on more basic understanding, before moving on to another forum which requires more complex thinking and critical reflection. Students can learn from each other about how they came to their conclusions, and can learn why others might not be arriving at the same answer.

Questions can also be based on a wider reading of relevant literature. Students would be instructed to find alternative viewpoints and arguments, to share their findings, and resources and citations. It also encourages participation and collaboration, and engages students in finding our about current ideas and research in their field of study.

You should also try to incorporate questions that do not rely on a text. Use videos or podcasts and sound recordings, graphics and images, webquest, scenarios provided by the tutor.

Experiential Element

This is based on Knowles's Androgogy, and constructivist views. Adult students bring a lifetime of experience with them (well, all students do, naturally, but this refers to more mature and varied experience that is often not there in younger students). The tutor should tap into this by providing discussion forums which are based upon the experiences of students and where they can share those experiences, and ask each other questions. They will create their own meanings based upon their prior experiences and peer generated questions can help to build new knowledge. It also increases the sense of community, and builds the students' social presence.

Style and Type of Question

In this case the 'style' of question refers to the students answering questions in pairs or groups. Then changing pairs or groups to discuss the question further. One advantage of this is that it reduces the number of posts in a forum. It also involves collaboration which again will enhance the feeling of community.

Different types of question could include: Metacognitive questions, in which students question their own knowledge, make connections between former and current problems, and reflect on the process of solving problems. Follow-up questions in which students consider different perspectives, provide clarification of thoughts, identify outcomes and answer the 'so what' question within the discussion. Student-created questions can provide thought provoking questions, and puts the student in charge of their own learning. Evaluation and Reflection questions allow students to reflect on the course so far, or any section of the course. For example the 'one-minute' assessment in which students write something they learned form the session and one thing they struggled with. This can then form the basis of a discussion. The tutor may want to allow anonymous contributions.

Finally

The discussion forum should be structured. Students need to know when the discussion is open and when it closes. They should know etiquette and protocols expected in an online discussion.  In summary, the tutor should decide the cognitive value of the question, then whether it should be literature based or not. Once this is established, decide whether it should be an experience-based question, then design the style and type of the question, before deciding the parameters for the structure of the question within the discussion forum.

photo credit: Horia Varlan via photopin cc

Lecture Capture

There are moves here at Leicester Uni to use lecture capture. A blog post by Mark Smithers suggests why this might be a bad idea. First, a couple of plus points for lecture capture:
The capture can be broken down into smaller chunks, so that it's not just simply a re-run of the lecture. It can also have subtitles added, and questions can be provided to engage the student in some active participation rather than simply passively watching.

It is useful for recording guest lecturers and visiting subject matter experts.

So, on to the bad points made by Smithers in his blog. Lecture capture perpetuates a passive and outdated mode of teaching. It is using 21st century technology to present 1000 year old pedagogy.

Lectures are a certain length often to suit the timetabling requirements of a particular building, rather than for any pedagogical reason - is there any need for example for lectures to be 1 or 2 hours long? Furthermore no meaningful learning can occur in a lecture.

What's the alternative? Use video technology to record short desktop pieces that are about 10 minutes long, and which develop a particular point. Or any sort of content that gets across information and ideas efficiently. This fits with the attention span of students, and enables them to study in their own time. Research at Bath University, where they have used lecture capture, suggests that the students spend around 10 minutes looking at the capture. This suggests they are skimming for particular content, and it ties in with evidence for people's attention span when learning. However, Bath uses the Panopto software which has good searching and note taking facilities.

Overall - I suppose it's like any technology in education, it can be used badly, and it can be used well. Perhaps money could be better spent on staff development that encourages different methods of delivery, and more engaging ways to deliver lectures.

photo credit: I, Timmy via photopin cc

Audio Feedback Workshop

On Friday I attended an audio feedback workshop here at the University of Leicester. The general aim was to look at good practice in the use of audio feedback. We started by writing limericks about the Olympics (surprisingly tricky) and drawing a cartoon to ilustrate our limerick (unsurprisingly tricky). We then gave written feedback to each others' efforts and discussed our feelings about the feedback. We then repeated the excercise, based on our feedback, and this time used a variety of recording devices to give audio feedback.

The general view was that it is much more difficult to get started with audio feedback, as many felt inhibited by the 'performance' aspect of it. However, it was felt that this sort of anxiety could reduce with time and practice.

The plus points about audio feedback are:

• It is more personal, and feels more like a conversation. Even though it is a monologue, the person listening acts as if they were in a two-way conversation (i.e. they nod, that sort of thing).


• It is easier to get across more difficult feedback such as criticism - it is explained better.


• Audio feedback is like a mini lecture, there is more detail and information. And you can say much more in two minutes of speaking than you could do if you spent just two minutes trying to write.

Negative points about audio feedback:

• It could be time consuming, especially if, as suggested, students would prefer written feedback too.


• It requires an element of 'performance' which some people may not be able to manage - i.e. it could be monotonous.


• There was a worry that people's recordings could end up on YouTube.

I think that the worry about workload is obviously a valid one, but I wonder whether this desire for audio and written feedback from students is because they are used to written feedback, and expect it. Perhaps they will become used to audio feedback and the desire for written feedback will drop off. So maybe given  the positive elements of audio feedback it is worth persevering initially with this extra workload as a kind of 'loss leader'.

Blocking Pirate Bay - The Continuing Assault on the Internet

I shouldn't listen to radio 4 first thing in the morning. John Humphrys often puts me in a bad mood before I even get out of bed - although I suppose I only dislike his questioning when he's giving someone I support a bad time :o) Anyway...yesterday was no exception. He was doing his usual aggressive questioning, not allowing the responder to get a word in on the subject of the judgement that may force ISPs to block Pirate Bay. This time it was a person representing the ISPs who was arguing that it is not their remit to police what people do on the internet. A perfectly reasonable and correct standpoint. Humphrys however, was having none of it - he thought that they were 'assisting' people in downloading illegal material and pornography by providing a service. That's like saying the Highways agency are complicit in bank robberies by providing decent roads for the robbers to escape on. Ridiculous. What came out of the interview from Humphrys (and a Conservative MP, also interviewed), is that they simply don't understand how file sharing works, or how the internet works. This article fron the Guardian sets out the resons why a block won't work.

This is simply another skirmish in the battle for the control of the internet - Big Business wants to turn it into a one-way vehicle for pushing content at people, government and bureaurocrats want to crush the ability to organise and protest - doing so in the name of 'combating terrorism'. It's a battle that must be fought and won by anyone who values freedom of expression and the right of dissenting voices to be heard

The Reluctant Pragmatist

When it comes to rapacious corporations trampling over innovation and diversity in pursuit of a monopoly, you'll always find me in the crowd chucking rotten vegetables at them.  That's why I was as angry as anyone else at Blackboard's disgraceful pursuit a patent for their proprietary software.  I'm also, in my own small, quiet, almost imperceptible way a part of the background noise grumbling at their recent acquisition of Moodlerooms and Netspot.

Their acquisition of these two Moodle support services seems to be, according to people more savvy than me on these issues, an attempt to diversify their activities so that they become an all-round IT and educational services provider. They also have an advantage over their competitors because they already have access to data that is useful for educational analytics, and that's where their attraction to institutions lies, in addition to being an 'end-to-end' provider for the student learning experience, and that's why they won't be disappearing any time soon (although they might change their name). Essentially, their product will be the people who use their VLE, and the data they can provide.

Which brings me to the fact that I have just got a job as a Learning Technology Assistant, with my main function being to work on Blackboard. I'm not going to refuse to work on it due to my ethical or political objections, I have to get on with it and earn a living, and provide a service for the people who use Blackboard. Whether I think a mashup of free and open source products would be pedagogically better than Blackboard is irrelevant - the university uses Blackboard, and probably aren't going to get rid any time soon (they've just upgraded to the latest version).

So, does this leave me as a hapless, forlorn hypocrite - tossed aside and disregarded by yet another corporate behemoth as it trundles on relentlessly in pursuit of increased shareholder value? I don't think so, I have to be pragmatic, I have a mortgage to pay, and I want two foreign holidays a year :-) I will continue to make a small contribution to the background noise and, as the saying goes, it's better to be on the inside of the tent pissing out, than on the outside pissing in, even if I'd rather not be doing either.

The death of the VLE?

There has been a debate for a few years now about the future of the VLE. It seems that reports of its death have been greatly exaggerated, although the fact that there is a discussion is of interest to me, as someone whose job is to develop an institutional VLE, and to support staff in its use.

The argument against VLEs is that they are owned by the institution, and are there to restrict and control the activity of its users, and that they promote traditional approaches to pedagogy that do not reflect the age of digital collaboration and the new tools available to students and teachers. They are bloated and cumbersome, and are slow to adapt to the needs of their users.

Compare that to the flexibility of 'web 2.0' applications, that adapt according to the needs of users, are owned by users and reflect the way that many students use digital media and applications. They are cheap, adaptable and support learning far better than VLEs.

It is clear that in the longer term these collaborative tools will become ubiquitous, and that the days of the VLE are numbered - however, that is still some way off.  The reason is that the use of web 2.0 tools by students and teachers alike is exaggerated. Most are limited in their ability to use these tools, and the argument that there are 'Digital Natives' isn't backed up by evidence .  The fact is that most teachers, and even learning technologists, don't use these tools on a regular basis, and neither do students. Many are unsure about how to use web 2.0 tools, and social networking applications in education - the VLE offers them a safe starting point, and a route into digital tools in education. There is also the issue of data protection, and ownership of the material that is written and posted to internet tools, and these tools do not provide the institution with the sort of user data sets and usage statistics that institutions increasingly need as part of new funding regimes. The data sets are available from a VLE as a part of a wider managed Learning Environment.

These tools can, however,  be embedded into VLEs - the major VLEs have the ability to stream Twitter feeds and RSS feeds. They allow videos from YouTube and blogs to be embedded, and they have chat, discussion forums, wikis and blogs built-in.

The 'personal web' - the aggregation of tools adaptable by users to their individual needs, will win in the end, but for the moment the VLE is alive and kicking. The main question isn't whether the VLE is dead, but how institutions are going to manage the integration of increasing user-choice tools, and their VLE.

The Real Motive Behind SOPA, PIPA and ACTA

Google and Facebook block content in India after court warns of crackdown | World news | The Guardian

There's a quote near the end of this article:  

But, like many other governments around the world, India has become increasingly nervous about the power of social media.

which indicates the real motives behind the recent attempts to introduce legislation on copyright and internet piracy. Ostensibly aimed at protecting the interests of copyright holders, their effect would be turn the internet into a delivery portal for content from media corporations, and  enable governments to crack down on what they would view as inflammatory material. Governments have seen the role that social media played in the 'Arab Spring' and, despite praising the liberating role the internet at the time, they can see the danger it presents to them and their ability to control information, and the people they supposedly serve.

Apples iPad Textbooks: Everything You Need to Know About iBooks 2

Apples iPad Textbooks: Everything You Need to Know About iBooks 2.

When I first heard of the iPad and the digital readers such as Kindle, I could see the possibilities for both fiction and non-fiction.

The possibilities for non-fiction and text books are obvious - more interactive content, rich with multimedia, and links to other resources and possibilities for annotation and collaboration offering a better learning experience.

The possibilities for fiction are also interesting. It won't be too long before more interactive fiction appears, with multimedia content and links to external resources and material. I was thinking it might be analogous to video games - such as Grand Theft Auto IV. You have the main storyline, but alongside that runs other 'missions' and side stories which introduce a range of other characters and broadens the profile of the main character. I can see fiction including such references to the back story of characters, and side stories, as well as ambient sounds, news footage, metadata and so forth.

Do Learning Styles Really Exist? | doug woods

Do Learning Styles Really Exist? | doug woods.

I get the impression that Doug is conflating 'Learning Styles' and 'individual differences' here. We all have individual differences in the way we learn, but as far as I can see no one has yet defined what a Learning Style is - genetic predisposition? Preference? Aptitude? Attitude? If you can't define what it actually is how can you measure its effect - it can have no reliability or validity.

I suppose there might be some confusion about 'learning styles' and 'Learning Styles'. It's easy to conflate 'learning styles' and individual differences, but I think many of the proponents of learning styles are actually thinking of them as 'Learning Styles' (with capitals) denoting them as actual entities that exist, which can therefore be objectified, packaged, and sold in training courses and books. These people have a vested interest in the existence of 'Learning Styles' so can't be trusted to have an objective view.

I disagree that rejecting learning styles will lead to regressive teaching - a teacher can reject Learning Styles while acknowledging individual differences, and provide a rich learning environment that calls on different modalities.  The material provided should suit the content, not an ill-defined abstract notion about how an individual learns.

What we need in teaching is evidence based practice, and so far there is no evidence to support the view that presenting, for example, visual material to your 'visual learners' is more effective and leads to better learning than other ways of presenting the material.

My view is that 'Learning Styles' inhibit good teaching, because it narrows the focus of the teacher and can lead to labelling and self-fulfilling prophesies (I can't learn this because I'm a visual learner'). It can also lead to the production of content that is inappropriate for the subject matter.

BBC News - Children 'switching from TV to mobile internet'

BBC News - Children 'switching from TV to mobile internet'.

More reasons why schools and colleges need to embrace mobile learning and the use of mobile devices as learning tools. Rather than banning them.

There is an issue, outlined in the comments section to this story, that children are becoming socially isolated and failing to develop social skills. I'd suggest that they are simply developing new types of social skills and ways of communicating with each other.  People still socialise, they still talk to each other. Those who lack social skills would probably have been the sort of people who lacked social skills in any age.

Creating Online Learning

I'm creating the first online learning course in my new job. It is to teach people how to create good discussion forums in an online course - so we based it upon the Five Stage Model, put forward by Gilly Salmon. To structure our course we too are using the Five Stage model. So it feels like it's all a bit self-referential - using the model to describe the model.

One thing about online learning is that it takes a lot longer to put together than a face to face session. All of the materials have to be there in advance and on the  VLE, or wherever it is being held. It is 'scripted' much more than a face to face session, and there isn't much scope to 'wing it', or think of ad-hoc activities if anything goes wrong. The learners need a clear path through the course, and this needs careful planning.

There are also lots of revisions as I try to put myself in the shoes of a learner and think about the course from their point of view. There is also the difficulty in trying to make sure that the activities we plan fit the model that we are using.

This is a learning process for everyone involved. I'm sure the course will go OK, and I look forward to the feedback - surely it won't all be bad :o)

Flipped Learning

 I like this idea of flipped learning, and flipped seminars. There is so much time in training and teaching dedicated to exposition by the teacher - this could be better used by getting the learners to do stuff. It's a good constructivist model for education too - the learner builds their knowledge through collaboration with their peers, and with facilitation from the teacher. It's a good use of classroom time.