briansharland.com Comment on “Assessing pupils work for #digitalstudies – my examination of badges and levels”

Hi @Ianinsheffield

I think we all welcome a little bit of devil’s advocate. #digitalstudies is becoming a serious option for schools and therefore to be taken seriously we need to definitely respond to any constructive critique such as yours.

I think the flexibility from using badges comes through what badges pupils might be able to work towards. To start with the portfolios / projects may be quite heavily structured and pupils do take a fairly similar line of progress through the work. However I envisage as an example that within lets say a group of pupils who are working together on a website you have a pupil who is concentrating heavily on producing good quality imagery which meets the client brief, is designed well showing effective annotations and is then produced with great skills showing good use of advanced features and is optimised effectively for use on the web. In the exact same group you have a pupil who is instead working on the CSS and HTML which will pull in the imagery, display it on the page in the right way and create code which is compact, well written and does the job.

Using traditional levels you may be able to distinguish the two pupils but I feel that with the way levels are written it may prove cumbersome to do so and the teacher will simply assign a general level which fails to acknowledge the radical difference between the two pupils. By having two different badges available – one for the graphic creation and one for the CSS and HTML and assigning those separately to the two pupils I think is a much neater and more relevant way of acknowledging the skills they have shown.

So how does one show progress? Another analogy to use is that of the scout uniform sleeve. When I attended scouts in South Africa way back in the day we had a semi-militarised uniform so I have no idea what goes on now but our uniforms allowed us to sew our badges on our sleeves whenever we achieved a skill. These sleeves didn’t show levels or progression charts, they simply showed the amount of badges a boy had and let me tell you this, boy were we motivated to gain more badges to show off on our sleeves. I can remember feeling slightly embarrassed by my collection at one point and going through the little book we had of activities to do to gain badges to see which ones I could do to gain more badges.

So my first point about showing progress is that the collection of badges simply becomes your means for assessing progress. It is an easy way to show how much a pupil has done and because the full list of badges is made available before the portfolio starts they can see what they can still do. Tying into what I said further up about different types of badges pupils can also concentrate on collecting badges to suit their own skills (and this is where the teacher can obviously direct pupils who are unsure of their own strengths).

But what about levels? My simple answer to that is how many technology companies ‘level’ their employees one against another? I don’t think many if any do and yet we are preparing pupils for hopefully technology roles one day. A badge becomes a simple metaphor for the acquisition of skills which can easily then be translated into that first CV.

So I think ultimately comparative progress is largely irrelevant for #digitalstudies but certainly one can track the chronology of progress simply through when badges are assigned.

All the best
Brian

On 16 Apr 2012, at 11:59, Posterous wrote:

— Reply above this line to comment on this post — IaninSheffield just commented on the post “Assessing pupils work for #digitalstudies – my examination of badges and levels” on Brian Sharland

Hi Brian. Just catching up with what you guys have been discussing on the Badges front. Fascinating!I’m definitely in favour of this alternative way of recognising and celebrating achievements, but with your forbearance I’m going to play devil’s advocate for a moment. Can I ask for a little clarification of how you think Badges might provide more ‘flexibility’ and ‘increased choice?’ Since the Badge represents a token picked up on *completion* of a learning journey, I was just thinking that flexibility and choice might kick in at the commencement of that journey. Or to put it another way, the Badge is the destination, whereas flexibility comes from the routes available?

I too think that Badges might provide a good way of revealing progress, especially to a wider audience than with current systems, but haven’t yet resolved how best to show *rate* or chronology of progress … or doesn’t that matter? Or even comparative progress … or doesn’t that matter either?

Be good to hear your thoughts, as I’m wrestling with these issues internally at the moment … and wrestling with yourself is no fun at all! ;-)

Posterous Spaces is the place to post everything. Just email us. Unsubscribe from this comment thread

Assessing pupils work for #digitalstudies – my examination of badges and levels

Today saw a fast and fantastic discussion take place on twitter between a number of people on the topic of how to assess work within #digitalstudies. It began with @misterel asking how we would be showing progress by pupils – a very reasonable question. To answer it though I will be taking a sidetrack down my recent computer games playing experience.

Two of the last major computer games I have played have been the original Mass Effect and Modern Warfare 2 (I know I am slightly behind … I blame parenting and Minecraft :-) ). Modern Warfare 2 firstly is a classic example of an on the rails shooter. Yes you can run from one side of the path to another side and do a teensy amount of sight seeing but if you don’t follow Captain British Stereotype the next story trigger won’t trigger and nothing happens. You exist in a very strictly designed world where you need to progress through the levels and options as set out for you by the game designers. I know for those who are reading this I am probably stating the obvious but hear me out.

Mass Effect on the other hand is less of an on the rails shooter. You are presented with some degree of freedom in the choices you make and can see some effect of those choices. Choose to ask someone nicely and they might offer you something out of kindness or be horrible and they respond out of fear. Once you are on a mission there may be some form of ‘rails’ but some freedom persists and once out of a mission you can choose to sidestep optional missions and focus only on the most important ones. You are still following what the game designers have decreed but there is a lot more flexibility in the order of choices you can make.

Having mentioned Minecraft as an aside perhaps I can also bring that back in as well and say that even though it’s a classic open sandbox game where you are free to make your own choices you are still bound by certain rails and rules. In survival mode youcan’t break through the adminium blocks at the bottom of the level and you can’t build higher than a certain level.

So how do games like MW2 on the one hand and Mass Effect and Minecraft on the other hand apply to assessment and learning in education. I think that MW2 is an analogy for National Curriculum levels and Mass Effect and Minecraft are an analogy for badges as an assessment method.

As in a game pupils need to make progress through the tasks they have been set and they and the teachers should be able to show that progress. Levels as a concept can work as it allows a pupil to see where they are, what skills they have gained so far and be able to see where they should go next. However the definitions for the levels (and I am thinking just for ICT here) were so arcane i can remember spending forever translating them into pupil speak (and wondering why they weren’t in pupil speak in the first place).

Like an on the rails shooter I also thought that levels were restrictive of a pupils progress and ensured that even with separate strands of levels a pupil was in essence working through the same levels as everyone else. As I think @mberry put it today – pupils may be climbing different mountains (but sometimes the same foothills). As soon as a pupils is wanting to leapfrog aspects of the levelling criteria it becomes very hard to pin them down to a number which shows the ‘progress’ the levelling system is meant to show.

Mass Effect and Minecraft are therefore analogies of a better approach to learning and assessment. Pupils remain within a semi-walled garden as even with freedom of choice in learning paths teachers must still provide the guidance and structure to make sure that pupils are continuing to learn in a good way. But pupils are presented (like the missions in a game) with a choice of things they wish to proceed with. There may still be some order to it as i can think of a situation in Starcraft 2 where I was unable to proceed to a next mission unless I had completed a specific previous mission bit there is still some flexibility.

Badges to me reward a pupils choice, naturally promote flexibility, provide encouragement to proceed down a learning path but also allow pupils to choose a different path from some of their peers. I have always believed that the goal with this new focus on programming is not to turn every pupil into a programmer but give every pupil the chance and option to take that path and then provide other alternate paths such as digital law or digital authoring.

#digitalstudies is a portfolio based subject. At the beginning of a year I feel that pupils should be shown what they should ideally be producing by the end of the year. However in a portfolio such as the multimedia one I am working on pupils could either be focussing on using digital tools to design and create beautiful digital works of art in imagery, audio or video but doing less on the programming side or they could be doing a small amount of digital creativity but focussing more on coding an awesome multimedia website. This approach I think is a reflection of a typical startup where each person has a particular strength and focusses on that strength but is able to work with others. Badges suit this type of work approach as it allows each pupil to work towards their own path which may be independent to someone else even in their same group. Levels I think are highly individualistic and don’t adapt naturally to work done in a group whereas a badge I think could be easier to assign. Pupils then show their progress through an accumulation of badges which could start to indicate both to the pupil and the teacher what their natural ‘discipline’ within #digitalstudies is.

Although I had a go at MW2 earlier I do remember spending quite a bit of time trying to nail 3 stars in as many specops missions as I could. Even though I knew that others were also working towards it it was a badge for me and I really wanted it. I think this could also apply to how a pupil could feel motivated in class.

How does APP and AfL come into this? Although I have focussed on badges I do believe strongly in formative assessment. Badges although summative as well are I think a better summative assessment method than levels. Pupils would read a number or grade and ignore the comments so often my feedback is only formative and includes no summative grades. From my own understanding of AfL I think that formative assessment is a necessary part of AfL and therefore if you are doing the formative assessment properly you are naturally driving AfL forward as well.

APP though although I have never used it being at an independent school I think is too bureaucratic and too structured. It contains a pupil too much within a top down methodology and therefore lacks the flxibility of badge based assessment.

Badges therefore in conclusion provides the flexibility, gives the pupil a measure of independent choice in what they wish to learn and show progress in and ultimately provides a learning and assessment experience which I think comes closest to what exists in the workplace which is ultimately what we are preparing the pupils for.

Producing iBooks for #digitalstudies

After last nights blog about how to put everything together for #digitalstudies I think the next logical step is to thrash out a structure for an iBooks textbook focussing on a task which pupils have to do. I am developing a list of tasks on http://digitalstudieswiki.pbworks.com/w/page/51407927/Multimedia%20Portfolio%3A%20Animating%20Beowulfshowing what pupils will do throughout the year. Each task will have their own iBook.

The resources which go into each iBook will be available separately for anyone else to use in creating their own resources or digital textbooks under the appropriate CC licence.

So what should an iBook structure consist of? I am a novice to creating iBooks but this should be fairly logical.

Let’s take one of the tasks for the first multimedia portfolio ‘Create a script and a storyboard’ and look at what an iBook for this would entail.

  1. Title page showing
  1. #digitalstudies
  2. portfolio and task
  3. author

Contents page intro video description of task and the levels available for each pupil handy links to begin with chapters describing approaches and issues for that task

  1. use of widgets and videos

conclusion glossary

The majority of that structure is fairly obvious but it at least means that each textbook would appear similar in structure and layout. What is written in the chapters could be quite different between each textbook but I don’t think that will be an issue. I also prefer longer written text as I think pupils should still be ‘reading’ as much as possible. For pupils who struggle with reading if the textbook is built with multimodal communications this shouldn’t be an issue.

Right – now to get started with my first attempt …

Creating an iBook for #digitalstudies

After last nights blog about how to put everything together for #digitalstudies I think the next logical step is to thrash out a structure for an iBooks textbook focussing on a task which pupils have to do.  I am developing a list of tasks on http://digitalstudieswiki.pbworks.com/w/page/51407927/Multimedia%20Portfolio%3A%20Animating%20Beowulfshowing what pupils will do throughout the year.  Each task will have their own iBook.

The resources which go into each iBook will be available separately for anyone else to use in creating their own resources or digital textbooks under the appropriate CC licence.

So what should an iBook structure consist of?  I am a novice to creating iBooks but this should be fairly logical.

Let’s take one of the tasks for the first multimedia portfolio ‘Create a script and a storyboard’ and look at what an iBook for this would entail.

  1. Title page showing
  1. #digitalstudies
  2. portfolio and task
  3. author
  • Contents page
  • intro video
  • description of task and the levels available for each pupil
  • handy links to begin with
  • chapters describing approaches and issues for that task
    1. use of widgets and videos
  • conclusion
  • glossary
  • The majority of that structure is fairly obvious but it at least means that each textbook would appear similar in structure and layout.  What is written in the chapters could be quite different between each textbook but I don’t think that will be an issue.  I also prefer longer written text as I think pupils should still be ‘reading’ as much as possible.  For pupils who struggle with reading if the textbook is built with multimodal communications this shouldn’t be an issue.

    Right – now to get started with my first attempt …

    Update on #digitalstudies

    Over the last month it has been fantastic to see how the initial idea has grown and matured and become a serious possibility for use in schools. The work of people like Nic (@teachesict) and Chris (@infernaldepart) has been invaluable and far more than even I have been able to contribute.

    With about a term until the subject goes live in possibly three schools some concrete things need to be decided. For one Nic has been putting together an insane amount of resources and with the moodle site Chris has put together the question is how do we turn the resources into a structure which can be taught.

    A key principle of #digitalstudies is that for a year pupils are working on a series of tasks which revolve around the production of work for a single portfolio on a set topic. The first portfolio I am developing is a multimedia portfolio on animating images which will contain work for an entire year.

    I have been struggling to put together a way of presenting this to my pupils but now I think I am beginning to see a way forward. Based on a discussion I had at school about differentiation and particularly differentiation by outcome I have come up with a way of possibly doing it.

    In the accompanying image I have shown two examples of tasks which pupils would do with the multimedia portfolio. As you can see the use of the word ‘task’ is fairly broad and encompases work which will take place over a couple of weeks of 1 hour lessons each week. Each task will have an entry, mid and high level description to allow for differentiation and scope. Pupils are presented with all levels at the beginning of a task and can choose which level they wish to go for.

    Each task’s resources are drawn from the resource bank developed by Nic which in turn is based on the four strands. This means that every task can be traced back to the strands.

    So how will each task be presented to the pupils? Although I know that not many schools will have access to iPads I will probably begin building simple small ibooks for each task describing the levels, the resources and giving some idea as to what pupils could be producing. As I know that schools will use other methods to teach but may want to use the same resources I will try and release the exact same content e.g. images, text and videos and release that as a package for others to use in building #digitalstudies resources.

    This plan I hope will start to yield good quality resources and structure for the progression of #digitalstudies. Lots more to come …

    Update on #digitalstudies

    Over the last month it has been fantastic to see how the initial idea has grown and matured and become a serious possibility for use in schools.  The work of people like Nic (@teachesict) and Chris (@infernaldepart) has been invaluable and far more than even I have been able to contribute.  

    With about a term until the subject goes live in possibly three schools some concrete things need to be decided.  For one Nic has been putting together an insane amount of resources and with the moodle site Chris has put together the question is how do we turn the resources into a structure which can be taught.  

    A key principle of #digitalstudies is that for a year pupils are working on a series of tasks which revolve around the production of work for a single portfolio on a set topic.  The first portfolio I am developing is a multimedia portfolio on animating images which will contain work for an entire year.

    I have been struggling to put together a way of presenting this to my pupils but now I think I am beginning to see a way forward.  Based on a discussion I had at school about differentiation and particularly differentiation by outcome I have come up with a way of possibly doing it.

    In the accompanying image I have shown two examples of tasks which pupils would do with the multimedia portfolio.  As you can see the use of the word ‘task’ is fairly broad and encompases work which will take place over a couple of weeks of 1 hour lessons each week.  Each task will have an entry, mid and high level description to allow for differentiation and scope.  Pupils are presented with all levels at the beginning of a task and can choose which level they wish to go for.

    Each task’s resources are drawn from the resource bank developed by Nic which in turn is based on the four strands.  This means that every task can be traced back to the strands.  

    So how will each task be presented to the pupils?  Although I know that not many schools will have access to iPads I will probably begin building simple small ibooks for each task describing the levels, the resources and giving some idea as to what pupils could be producing.  As I know that schools will use other methods to teach but may want to use the same resources I will try and release the exact same content e.g. images, text and videos and release that as a package for others to use in building #digitalstudies resources.

    Digitalstudies

    This plan I hope will start to yield good quality resources and structure for the progression of #digitalstudies.  Lots more to come …