Why Marking Essays by Algorithm Risks Rewarding the Writing of 'Bullshit'

by Kai Riemer, University of Sydney, The Conversation: https://theconversation.com/why-marking-essays-by-algorithm-risks-rewarding-the-writing-of-bullshit-85910

You may have heard that algorithms will take over the world. But how are they operating right now? We take a look in our series on Algorithms at Work.

Will marking algorithms really reward good writing? Terence/Shutterstock

---

Picture this: you have written an essay. You researched the topic and carefully constructed your argument. You submit your essay online and receive your grade within seconds. But how can anyone read, comprehend and judge your essay that quickly?

Well, the answer is no one can. Your essay was marked by a computer. Would you trust the mark you received? Would you approach your next essay with the same effort and care?

These are the questions that parents, teachers and unions are asking about automated essay scoring (AES). The Australian Curriculum, Assessment and Reporting Authority (ACARA) proposes to use this program to grade essays, like persuasive writing questions, in its NAPLAN standardised testing scheme for primary and secondary schools.

ACARA has defended its decision and suggested that computer-based marking can match or even surpass the consistency of human markers.

In my view, this misses the point. Computers are unable to genuinely read and understand what a text is about. A good argument has little worth when marks are awarded by a structural comparison with other texts and not by judging its ideas.

More importantly though, we risk encouraging the writing of text that follows “the script” but essentially says nothing of worth. In other words, the writing of “bullshit”.

How does algorithmic marking work?

It’s not entirely clear how AES functions, but let’s assume, in line with previous announcements, that it employs a form of machine-learning.

Here’s how that could work: a machine-learning algorithm “learns” from a pool of training data – in this case, it may be “trained using more than 1,000 NAPLAN writing tests scored by human markers”.

But it generally does not learn the criteria by which humans mark essays. Rather, machine learning consists of multiple layers of so-called “artificial neurons”. These are statistical values that are gradually adjusted during the training period to associate certain inputs (structural text patterns, vocabulary, key words, semantic structure, paragraphing and sentence length) with certain outputs (high grades or low grades).

When marking a new essay, the algorithm makes a statistical inference by comparing the text with learned patterns and eventually matches it with a grade. Yet the algorithm cannot explain why this inference was reached.

Importantly, high grades are awarded to papers that show the structural features of highly persuasive writing – papers that follow the “persuasion rulebook”, so to speak.

Rewarding bullshit

Are the claims by ACARA that algorithmic marking can match the consistency of human markers wrong? Probably not, but that’s not the issue.

It’s possible that machine-learning could reliably award higher grades for those papers that follow the structural script for persuasive writing. And it might indeed do this with higher consistency than human markers. Examples from other fields show this – for instance, in the classification of images in medical diagnosis. It will certainly be quicker and cheaper.

But it will not matter what a text is about: whether the argument is ethical, offensive or outright nonsensical, whether it conveys any coherent ideas or whether it speaks effectively to the intended audience.

The only thing that matters is that the text has the right structural patterns. In essence, algorithmic marking might reward the writing of “bullshit” – text written with little regard for the subject matter and solely to fulfil the algorithm’s criteria.

Not simply lying, analysts use “bullshit” to describe empty talk or meaningless jargon. Princeton philosopher Harry Frankfurt argues that talking bullshit may actually be worse than lying, because the lie at least reaffirms the truth:

It is impossible for someone to lie unless he thinks he knows the truth. Producing bullshit requires no such conviction. A person who lies is thereby responding to the truth, and he is to that extent respectful of it … For the bullshitter, however, all these bets are off: he is neither on the side of the true nor on the side of the false. His eye is not on the facts at all, as the eyes of the honest man and of the liar are, except insofar as they may be pertinent to his interest in getting away with what he says.

Unlike humans, algorithms are incapable of truly understanding when something is nonsense rather than genuine ideas and argumentation. It doesn’t know whether a text has any worth or relationship to our world at all.

That’s why algorithmic marking, whether in NAPLAN or otherwise, risks rewarding the writing of bullshit.

Encouraging the wrong thing

Our politics, businesses and media are already flooded with empty arguments and jargon. Let’s not reward the skill of writing it.

Any application of algorithmic decision-making creates feedback loops. It influences future behaviour by rewarding and foregrounding some aspects of human practice and backgrounding others.

This is particularly the case when incentives are tied to the outcomes of algorithmic decision-making. In the case of NAPLAN, we know that the government rewards schools that score highly. As a result, there is already an entire industry geared towards “cracking the script” of NAPLAN in order to secure high marks.

Imagine what happens when students realise that genuine ideas and valid arguments are not rewarded by the algorithm

Kai Riemer, Professor of Information Technology and Organisation, University of Sydney

This article was originally published on The Conversation. Read the original article.

What it's Really Like to Mark a GCSE or A-level Exam

Children doing exams inside a classroom, 1940 (Wikipedia)

by Velda Elliott, University of Oxford, The Conversation: https://theconversation.com/what-its-really-like-to-mark-a-gcse-or-a-level-exam-64034

Every summer results day gives rise to a raft of headlines about the number of students who have to appeal because of inaccurate marking - though strangely enough, no-one ever appeals because they think their paper has been marked too highly.

Despite the headlines, the actual likelihood of a marker getting it totally wrong is actually pretty low. This is because markers do not know what the actual grade boundaries will be for each paper - their job is simply to apply the mark scheme as accurately as they can. So even if they are generous on principle, the effect will simply be to raise the grade boundaries. Besides, examiner training emphasises the need to be fair to all candidates, rather than sympathetic to a few.

As a former marker myself, and someone who has researched the topic, I know how examiner training, and the system used to check up on marking accuracy throughout the marking period, is designed to remove as much variability as possible. Although there is of course an element of subjectivity with all marking - and in some subjects more than others.

When it comes to marking test papers, it stands to reason that the more straightforward the question, the more straightforward the marking - and questions requiring short answers are naturally the easiest to mark reliably. With these types of questions, markers can compare the actual answer with the correct one and then can simply check for a match or mismatch.

But as answers get longer, it becomes more difficult to anticipate every possible response - meaning there is of course more scope for variability in markers’ judgements of the same script.

To be or not to be

English is notorious as being particularly difficult to mark - and “creative writing” pieces are especially subjective. Computer marking systems that are considered to be completely reliable are known for giving poor marks to what are widely considered great pieces of writing by the likes of Winston Churchill or George Orwell - because no matter what rules you lay down for making judgements, there are always exceptions.

Besides, what makes good writing is a subjective judgement. If it wasn’t, the same book would win all the literary prizes in any given year. What can be specified in writing, however, is spelling, punctuation and grammar - so grade criteria will include things like “using a variety of punctuation”, and “varying sentence lengths for effect”.

 
English exams: a marking minefield. bibiphoto/Shutterstock


But of course, interest and imagination in a written piece is not captured in spelling and grammar. And how a reader reacts to the creative element - such as word choice, ideas, or literary language - will depend on that individual and their life experience.

Essays are also difficult to mark - they are long, and markers must bear in mind the mark scheme, and model marked essays. This makes it a tough mental activity. Add in the time pressure and it becomes harder still, although exam boards are careful to remind examiners to mark at an “appropriate rate”. That is, slowly enough that they aren’t making mistakes, but fast enough to meet demanding deadlines.

Collaborative effort

Ofqual, the exams regulator in England acknowledged in its review of marking reliability in 2014 that “there would always be an element of subjectivity in some marking”. The difficulty for appeals is distinguishing between actual errors, and acceptable subjectivity.

But despite this element of subjectivity, in reality it would actually be difficult for one examiner to get it wrong enough to change the whole outcome. Not only because grade boundaries are unknown by the marker, but also because qualifications are often made up of more than one examined paper. Each paper at the very least will have different markers - and in many examinations now, papers are scanned and questions separated so that several markers will have input into one paper.

This should, in theory, increase the reliability overall because it reduces the chance of any paper being marked by an overgenerous, or overly mean, marker. It also means that examiners cannot give the benefit of the doubt where a candidate is on a boundary - because they don’t know how the candidate has done on the rest of the paper.

Additionally, the mark scheme which markers use in the examining process only refers to marks, not grades. Grade boundaries are set separately every year after the marking is complete, using statistical information alongside the proportion of each grade awarded the previous year. And this is informed by the expert judgement of senior examiners.

Off the mark

And yet, despite all of these efforts to avoid bias and remove as much subjectivity as possible, there is still a tension in the examination system in England between the “criteria-referenced mark schemes” - which imply that anyone can get an A, providing they work hard enough and meet the criteria - and the actual way in which grading functions.

The marking system can leave some parents and students feeling let down. Syda Productions/Shutterstock

Grading is not quite “norm-referenced” - where grades are allocated on a bell-curve, as is the usual practice in the US - but the number of As is limited by the proportion of students who are predicted to get that based on previous results. The number of As given in previous years is also taken into account - with a small margin for increased standards if necessary.

This leaves a mismatch between the expectations of students and parents, and the way the system actually works. But while it may sound harsh, we need qualifications to function at least partly as a sorting mechanism for entry to university or to work places - because if everyone could achieve an A grade, they simply wouldn’t fulfil this function - meaning these marks wouldn’t be worth the paper they’re written on.

Velda Elliott, Associate Professor of English and Literacy Education, University of Oxford

This article was originally published on The Conversation. Read the original article.

Informal And Formal Assessment With Multi-Ability Classes

by Richard D Boyce

Formal assessment often creates fear within the student such that it prevents them from performing at their best.

Therefore, it is important for the teacher to give students practice assessment items to allow them to gain experience in doing assessment under exam conditions to ease that fear.

This informal assessment helps both the teacher and student gauge how they are progressing and what they need do to prepare for the real thing.

In classes with a wide range of ability, there are strategies you can use to help prepare all your students to do well in formal assessment by using your informal assessment as preparation.

Below are ideas to consider as part of your informal assessment:

1. It is important to find out what understanding of each new topic your class has as a starting point for your teaching.
2. Use frequent, quick, short tests to consolidate the basics.
3. Practice any new assessment task before you use it formally.
4. Make sure the format of your informal assessment reflects the formal assessment format.
5. If you divide your class into ability groups, set tests to reflect their progress.
6. You might read the questions aloud to the lower ability groups to help them understand what has to be done.
7. If you set a common test for the class, you could set different starting points for each ability group. This would allow all students to gain some success and the more able to progress to the more challenging questions.
8. Alternatively, consider testing each ability group separately with their own test or section of the main test.

When it comes to formal testing, consider the following. If you are responsible for your class's total assessment program, then some of the ideas below will give you more flexibility than you would have if the assessment program incorporated many classes.

1. In formal testing, separate the testing of the basics from problem solving. This reduces stress on students allowing them to perform at a higher level more confidently.
2. Make sure each unit of the test begins with an easy example and progress through a range of difficulties. This means most students will get a start.
3. If you are prepared to be adventurous, you might set a test, graded in difficulty, to allow the students to choose where they start and finish.
4. With some less able students in your class, you might decide to give them a clue to get them started. You should record this on their paper and make adjustments to the marking scheme.
5. All of the above may need to be tempered to the rules set up by the formal testing procedures mandated by outside authorities in upper high school year levels.
6. Give separate skills and problem solving tests. The skills test would have a set time while the problem solving test might be more time flexible.
7. Remind students that the basic skills are paramount to gaining a pass mark and are the essential basis for problem solving.
8. Your assessment tasks should always reflect your teaching pedagogue.

Overarching these ideas are two important strategies that the teacher must engage in with their classes.

No student will do well in any assessment program, no matter how hard they work, unless they have an effective examination technique. Teachers need to teach their students how to do an examination and constantly review those techniques before each assessment task.

In addition, when reviewing the assessment task with their students, they must point out where students have made errors due to poor examination technique.

Finally, it is important for the teacher to model for students how to actually do each different type of assessment by using one of each and discussing how they would go about doing that type of assessment task. This should not be a one-off.

This article is one of many written by our author who had over 40 years' experience in the classroom. In his final years of permanent teaching, as Head of Mathematics, he was responsible for the total assessment program in his school.

All of his experience on many examination and assessment topics can be found in his eBook, "The Exam Book" on the website http://www.realteachingsolutions.com

Article Source: http://EzineArticles.com/?expert=Richard_D_Boyce
http://EzineArticles.com/?Informal-And-Formal-Assessment-With-Multi-Ability-Classes&id=7744324

Assessing Students Who Become Sick During Exams

Taking an Exam

by Richard D Boyce

There will always be conscientious students who are sick but will come to school to do an examination.

Teachers should discourage this practice both to the student and their parents as most, if not all schools, have protocols to overcome initial absences from examinations.

Encourage your students to tell you if they have been or are feeling sick prior to an examination. Then, with them, you can make a decision as to whether they will attempt it or not.

If the worst does occur and the student does get sick doing the exam, here is a process you might follow.

• Note the time this occurs on the student's exam paper at the point where the illness occurs as the student may like to continue at a later time.


• Treat the ill student in accordance with school protocol.


• When the exam paper is returned to you for marking, mark the parts of the exam that have been done prior to the illness occurring and any other work separately. Compare the two sections to see if they seem to be typical of the student's results and then discuss the results with your supervisor to decide on a mark or adopt normal school policy on how to give appropriate credit.


• Check also if the child was sick before the exam. This too could be factored into your decision about a final rating/mark for the exam. It may be that the results you already have from previous assessment items, give you enough data to give a valid level of achievement for the subject. That means you can ignore this exam result. School policy may suggest how you treat this situation.


• Always encourage a student to do the exam at a later date rather than do it when they are ill. The exam can be marked and used as a guide to their achievement as well as giving the student a real idea of how he/she understood the learning tested in that exam. This would allow him/her to do remedial work if necessary before continuing the subject or starting the next unit in the subject or the next year. These marks could be recorded with an explanation and may be used later to give extra data to help decide on the final subject rating on exit from the year or from school.


• Check for serial 'sick' offenders,.i.e. students who always seem to be sick on exam days. Some may have chronic illnesses and the school may have a policy for this situation. Those who have no history of chronic illness need to be 'forced' to do the exam ASAP under exam conditions with the same time or even less.

Perhaps they might be given a different test and the tests could be in their time, not class time. Again, this is a school policy issue.

It is worthwhile checking the history of these students with past teachers and referring the issue to higher authorities for further intervention, e.g. parent interviews with school administration.

It is important that no student be disadvantaged through genuine illness at exam time. Additionally, they must be seen to be treated fairly in the allocation of their final grading.

Here, a teacher must use their professional judgment based on the evidence of learning in class as well as the available assessment data. Here 'like' students can be a guide to where the student lies in the class order of merit in that subject.

The website http://www.realteachingsolutions.com provides an eBook that looks at all aspects of examinations and other types of assessment. The eBook is "The Exam Book".

Our author, during the last 16 years of his teaching career was the Head of a Mathematics Department where he was responsible for the assessment program.

He offers practical advice that works on aspects as diverse as running a class exam up to how best to mark alternative assessment tasks.

Article Source: http://EzineArticles.com/?expert=Richard_D_Boyce
http://EzineArticles.com/?Assessing-Students-Becoming-Sick-During-Exams&id=7728099

Survival Tips: Grading Essays

Bin full of coffee cups (Wikipedia)

by Erica J Thompson

Maybe it's just me - ha, yeah right! - but grading is one of the most difficult parts of this whole teaching thing.

I know I am not the only teacher who feels this way.

In fact I had a student tell me that he was thinking of being a teacher until he realized all his teachers complained about the grading.

I actually felt bad that I contributed to his negative thinking and apologized to him for complaining about my job.

I gave him honest reasons why teaching is amazing and why it can be a challenge.

I then decided to not say a word about grading to my students and simply keep my thoughts to myself, a few close friends, and you.

My aversion to grading is rather new. I never used to mind grading. I actually used to like it. I enjoyed seeing what my students learned and reading their thoughts.

It all changed this year. I know that there are ebbs and flows with everything and that teaching is one of those things. I know there are good years and OK years and years that make you think "Good gracious, when will it all end?" This year is no exception.

Overall this has been an OK year for many different reasons. However, when it comes to grading it has been a "good gracious, when will it all end?"

I have had many times where I question my ability to teach, where I consider going back to school and getting my MBA, or becoming a PE teacher - why didn't I do this to begin with? I could wear yoga pants every day!

In many ways this year I have adapted some survival tactics so that I can enjoy make it through my pile of essays that I will share with you now.

- Carry stacks of papers from desk to car to home to car to desk to home to car to desk to ...
- Grade two papers a day for a month, you might eventually finish.
- Before you grade think of the top 3-5 things you want the essay to do, only read those parts and grade accordingly.
- Take a break every 10 papers or so, but don't snack, you will gain 10 pounds in a week!
- Stop grading when comments start to become "did you listen to any of the directions I gave you?!?!"
- Remember that it is their paper, not your paper.
- Stack the deck - I like to look for student's who usually produce good work and place their papers strategically though the pile. It is a needed boost when I am considering walking off the job.
- Find the funniest simile, metaphor, personification etc. One of my favorites was "love is like a pot of wine, just one drink and you are passed out drunk." So much about this is beautiful. Who drinks wine from a pot? What kinds of wine renders a person drunk after one sip? How did an 18 year old senior know about this wine?
- Run a mile after every 20 papers. It keeps you awake and fresh. Works well when you are stuck grading at home and can run around your neighborhood or have a treadmill.
- Remember that these kids, even though they are not your kids, are someone's kids. They are daughters and sons and should be treated with worth.
- Assign grades based on how much I like them, how much they gave me in a Starbucks card for Christmas, how many excuses they gave when they turned the paper in, or blindly make piles of papers in a stack for As, Bs, Cs, Ds and Fs. Obviously, I do none of these, but it is fun to think of new ways to grade.
- Instead of getting mad, sad, depressed I now laugh. Just laugh it off and move on. The kids who do care will continue to do well and the ones who do not will change one day or not. You can't fix everyone.
- Pray. I honestly have to pray before I sit down and grade these days, it helps me remember to be kind.

No matter how much you want to throw the paper away or just give them all As there are plenty of ways to make it through that mountain staring you down. Do not get discouraged and do not feel like you are alone. This too shall pass.

http://www.ericajt.com
So Much Sunshine

Article Source: http://EzineArticles.com/?expert=Erica_J_Thompson
http://EzineArticles.com/?Survival-Tips---Grading-Essays&id=7685606

Computer Thinks You’re Dumb: Automated Essay Grading in the World of MOOCs

Pedagogical Lessons Learned as Students in Massive Online Open Courses (Photo credit: BCcampus_News)

by Mark Gregory, RMIT University

Let us consider the following scenario.

You have enrolled in a Massive Open Online Course (MOOC) offered by a world renowned university.

After four weeks of solid work you have completed your first assignment and you sit down to upload the essay.

Within a second of the essay being sent for grading your result appears declaring your essay to be a less than stellar effort.

But the essay might not have even been seen by a human, but instead been graded entirely by a computer system comparing your essay to sample essays in a database.

EdX, a non-profit MOOC provider founded by Harvard University and the Massachusetts Institute of Technology, introduced automated essay grading capability in a software upgrade earlier this year.

But should automated grading systems be used for essays? And how good are these systems anyway?

The MOOC phenomena

The hype surrounding MOOCs reached fever pitch last year. MOOCs were initially brought to us by prestigious American universities - offering the same content that students paid for, to anyone for free.

Australian universities soon jumped on board and homegrown MOOC platforms quickly followed.
Australian schools and universities have been using automated grading systems for multiple-choice and true-false tests for many years.

But EdX has moved one step further using artificial intelligence technology to grade essays - a controversial step given this approach is yet to be accepted.

EdX’s president, Anant Agarwal told the New York Times earlier this month that “instant grading software would be a useful pedagogical tool, enabling students to take tests and write essays over and over and improve the quality of their answers.”

Agarwal said the use of artificial intelligence technologies to grade essays had “distinct advantages over the traditional classroom system, where students often wait days or weeks for grades.”

Robot graders

Automated grading systems that assess written test answers have been around since the 1960s when the first mainframe computers were introduced.

The New York Times reports that four US states (Louisiana, North Dakota, Utah and West Virginia) now use automated essay grading systems in secondary schools and in some situations the software is used as a backup which provides a check on the human assessors.

Automated essay grading relies upon the system being trained with a set of example essays that have been hand-scored. It then learns from the example essays and results provided for other student’s essays, and includes an analysis of indicators about phrases, keywords, sentence and paragraph construction.

Automated essay grading systems can be fine-tuned by getting humans to grade a sub-set of the submitted essays. But this limits the ability of the automated grading system to provide instantaneous results and feedback.

The artificial intelligence technology can then step in to make the process more sophisticated, using knowledge gained from human marked essays.

Can a computer really grade an essay?

There has not been general acceptance for the use of artificial intelligence technologies within automated grading systems. And recent moves by online education organisations to use artificial intelligence technologies for high-stakes testing has caused concern among academia.

This concern culminated in an online petition against machine scoring of essays launched earlier this year by a group of concerned academics and research staff. Over 3,600 signatures have been collected so far including from high profile intellectuals like Noam Chomsky.

A statement on the petition website argues that “computers cannot ‘read’. They cannot measure the essentials of effective written communication: accuracy, reasoning, adequacy of evidence, good sense, ethical stance, convincing argument, meaningful organisation, clarity, and veracity, among others.”

Les Perelman, a researcher at MIT, is highly critical of automated grading systems and has provided a critique of the state-of-the-art of artificial intelligence technology use in automated essay grading systems.

Perelman states that “comparing the performance of human graders matching each other to the machines matching the resolved score still gives some indication that the human raters may be significantly more reliable than machines."

In June 2012, Perelman submitted a nonsense essay to the US Educational Testing Service’s (ETS) automated grading systems called e-Rater and received the highest possible grade.

ETS uses the e-Rater software, in conjunction with human assessors, to grade the Graduate Record Examinations (GRE) and Test Of English as a Foreign Language (TOEFL), without human intervention for practice tests.

Both these tests are high stakes - the former decides entrance to US graduate schools and the latter the fate of non-English speakers wishing to study at American universities.

Testing times

In the rush to adopt MOOCs, Australian universities may skip important debates on what forms of assessment are acceptable and how to ensure educational outcomes are valid.

Central to the value of MOOCs as a pedagogical tool is the method used to assess course participants. Artificial intelligence technologies have been advancing rapidly but have they reached the point where automated grading systems can replace teaching academics?

Please tell us what you think - do we need real live humans to grade essays or do you believe computers can do the job just as well? Leave your thoughts in the comments below.

Mark Gregory does not work for, consult to, own shares in or receive funding from any company or organisation that would benefit from this article, and has no relevant affiliations.


This article was originally published at The Conversation. Read the original article.

Instant Grades

Massachusetts Hall, Harvard University (Wikipedia)

by Dave Pell, Next Draft

Hi readers, this story presents a real problem - what do you think?

A non-profit enterprise backed by Harvard and MIT has introduced software that can instantly grade student essays (and provide feedback) without troubling any human professors.

So who are disappointed students going to beg for a higher grade, a Roomba?

Criteria-Based Marking In Mathematics

by Richard D Boyce

Traditionally, assessment in Mathematics at high school and beyond has been the timed, formal, pen on paper examination.

Often, talented Maths students have failed to show their expertise in these formal examinations due to exam nerves. Other students who learn in a non-maths/logic way found that the formal situation did not fit into their learning styles.

What has further pointed Mathematics teachers to use a variety of pedagogue and then non-traditional assessment techniques has been the recognition of the existence of Multiple Intelligences. This suggests strongly that students learn in different ways.

Additionally, more students are staying at school longer and studying Mathematics in the hope of gaining entry to more prestigious university courses.

These factors have lead in some educational areas to new Mathematics syllabuses that require assessment tasks to reflect a wide variety of pedagogue as well as the use of computer and calculator technology.

The traditional formal examination remains but has become one of many assessment techniques that may be used. This has led to criterion based marking in the non-traditional assessment tasks.

This article looks at one alternative assessment item that uses criterion based marking. It is summarised and the criteria are included and some further explanation added.

Year 12 Maths B Assignment

NETWORKS - CRITICAL PATH ANALYSIS

Select one of the following projects and use Critical Path Analysis techniques to complete the questions below:
A Compiling School Magazine
B Organising School Musical OR Rock Eisteddfod
C Organising School Senior Formal

What is above was the basis of the assignment. What followed this was the general description of what was expected in this assignment. Then the questions to be answered with regard to the Critical Path Analysis were specified.

They included questions which tested their basic understanding (Techniques) of Critical Path Analysis plus four questions of a problem solving variety. Here is an example of one of each of these.

A techniques question/requirement was:

Construct a network for these activities.

An applications question (problem solving) was:

Select a completion date for your project and show justification mathematically for the latest date on which you could begin the project.

The assignment was marked using three criteria. They were:

1. Communication (this criterion looks at how the student communicates his/her Mathematics);
2. Techniques (this criterion assess the student's knowledge of the core ideas/learning work of the topic); and
3. Applications (this criterion assesses how the student applies his/her knowledge of Mathematics to a problem solving situation, often in an unfamiliar context).

These criteria were the ones used when I wrote Alternative Assessment items late in my career. Obviously, the syllabus documents will specify and define the assessment criteria applicable in each system and how they are applied. I simply use these as an example.

Below are the assessment criteria I used for this assignment.

MARKING CRITERIA
COMMUNICATION
VERY GOOD
SATISFACTORY
UNSATISFACTORY

Presentation

Sentence construction
Spelling
Use of Mathematical symbols

Allocation of Standard
H Mostly Very Good S Mostly Satisfactory U Mostly Unsatisfactory

TECHNIQUES

STUDENT HAS CORRECTLY:
YES
SOME
NO

(a) Recorded activities
(b) Recorded time required to complete each activity
(c) Ordered the activities - stated in writing which it precedes
(d) Drawn a Network/s as appropriate
(e) Drawn Precedence Boxes
(f) Completed a forward Scan
(g) Completed a backward Scan
(h) Drawn in the Critical path
(i) Drawn a Gantt Chart
(j) Significant use of the Internet including listing of URLs

Allocation of Standard

A Completed all the above activities correctly
B Completed (a) - (h) correctly
C Completed (a) - (h) maybe some minor errors
D Some attempt
E No attempt

APPLICATIONS

QUESTIONS
1(S)
2(S)
3(C)
4(C)

F
No attempt
E
Little progress
D
Substantial progress but no solution
C
Incorrect solution caused by a major conceptual error
B
Correct solution and calculations shown (minor errors allowed)
A
Correct solution and justification of techniques used.

SIMPLE
/2
COMPLEX
/2

Notes:

1. In Techniques and Communication, you will notice all the basic skills are listed. This enables the student to know the least requirements to gain a satisfactory grading.


2. In Applications, the criteria are generic so as to not lead the student towards a solution. This makes it a real test of how the student uses his/her Mathematics knowledge. Specific marking criteria are developed for the marker to use and these are published for the students after the assessment task is returned to them.


3. In Applications, you will notice the words 'simple and complex' to describe the problems. The syllabus required a variety of unfamiliar problems from simple to complex in Applications to help delineate between students at the top end of the scale.


4. One of the important advantages of this type of criteria based marking is that you can use the same criteria over and over once you have revised them. All you do is to change the question.


5. Lastly, it is important to note that in discussing the assignment initially, the teacher should remind students to address the criteria at all times. This will ensure that they get the maximum grading for each criterion that they are capable of getting.

Our author, as a Head of Mathematics, led his staff in introducing criteria based assessment when syllabus revisions mandated the use of a variety of different pedagogue and assessment techniques. His school became known as an innovative school in the teaching of Mathematics.

He became a presenter of professional development on this issue and many schools borrowed examples of their assessment items to use as the basis of their early trials of these assessment techniques.

For a more detailed look at Alternate Assessment, go to http://www.realteachingsolutions.com and seek out the eBook, "The Examination and Assessment Book".

Article Source: http://EzineArticles.com/?expert=Richard_D_Boyce
http://EzineArticles.com/?Criteria-Based-Marking-In-Mathematics&id=7601449

Three Practical Steps For Academics To Take To Reduce The Impact Of Contract Cheating

Cheating (Photo: Wikipedia)

by Thomas Lancaster

Contract cheating is an undesirable undertaking within college and university education.

This form of cheating occurs when a students gets a piece of work completed for them which they then hand in as if they wrote this for themselves.

This is a dangerous trend within education since the student is getting academic credit, which may lead to an award, for a piece of work that they didn't create themselves.

This gives a false expectation about the skills that the student will have when they join the workplace.

The good news is that the literature related to contract cheating contains many case studies and suggested methods that will help academics to combat this growing problem.

This article identifies three popular practical steps which can be used to reduce the impact of contract cheating. They should be considered in light of the wider assessment processes within which this form of academic misconduct falls.

First of all. steps should be taken to make sure that work is completed within the classroom wherever possible. This means that students cannot leave and get their work completed somewhere else. This also allows academics some time to become familiar with their students and the work that they are completing. Where it is impossible to completely monitor this, checkpoints can be placed within assessment so that student progress can be monitored in some other form.

Second, the weighting of any piece of work on which contract cheating may be possible should be considered. It should be impossible for a student to pass a module based on contract cheating alone. In many cases, this will mean that coursework will need to constitute under half of the work on a particular module. This can be supported by other forms of assessments which are taken under controlled conditions; these include examinations, vivas and presentations.

One third method to consider when aiming to reduce the impact of contract cheating is of interest. Consideration has been paid to the use of Honour Codes, where students are compelled to report other students who they believe to be cheating. With this method, students who do not report cheating that they know about are said to be equally as guilty as the cheaters themselves. Where this method is plausible according to the local assessment regulations it is well worth considering further.

In summary, the growing problems associated with contract cheating do need careful attention within education to avoid this reaching the levels of wider student plagiarism. Coursework should be set so that it can be completed within supervised sessions wherever possible.

The weighting of work completed outside the classroom should be closely considered so that students cannot pass purely in this manner. The availability of Honour Codes should be investigated and used wherever possible. Only by such a multi-pronged approach can the impact of contract cheating be reduced.

Thomas Lancaster is an academic who regularly researches and presents on contract cheating. More details about Thomas Lancaster can also be found on his professional presence, which also contains many blog posts related to cheating and plagiarism issues.

Article Source: http://EzineArticles.com/?expert=Thomas_Lancaster
http://EzineArticles.com/?Three-Practical-Steps-For-Academics-To-Take-To-Reduce-The-Impact-Of-Contract-Cheating&id=7427317

Accountability "On the Line"

"Teacher Appreciation" featured photo (Photo: Wikipedia)

by Kathleen Kardaras

Formative assessment, summative assessment, portfolio assessment, "authentic" assessment - and, yes, standardized assessment, are all used to determine whether students are learning.

Why, then, would teachers not argue for the same approaches to evaluating their own performance in the classroom?

Systemically, education administrators determine what students should know, when they should know it, and how they should be able to demonstrate learning on a single standardized test. However, they insist that students not be graded simply on such standardized assessments of learning.

So, why should administrators determine teacher performance on the results of a single, standardized measure? And why are teacher's union leaders not fighting for simply the same kind of evaluation for teachers that have been developed and deemed appropriate for students?

With current legislation, teacher performance in most states is evaluated using standardized measures of student performance. If students do well on the state's standardized tests, teachers will be retained.

If not, teachers can be fired and schools may suffer re-staffing, re-formulation, or closure. Currently, teacher performance is evaluated on little more than student outcomes on a single measure, not on the multiple measures of learning that are used in classrooms.

Given these realities, one might ask, "Why are teachers and their union representatives not arguing for the same measures used to evaluate student performance, to evaluate teacher performance?"

Given the diverse ability and preparation levels found in any current classroom, diverse evaluation measures are appropriate. Why not the same for teachers? Teacher preparation is also diverse, as is teacher ability. School environments offer diverse opportunities,and present diverse challenges in access to computers, science labs, math, and language instruction.

If "authentic" assessment is good for students, why should it not be used to evaluate teacher performance? Why shouldn't observation, portfolio, lesson-demonstration, growth from one marking period to another (measured in teacher-created rubrics), and self-evaluation, all contribute to re-hiring, promotion, and salary increase?

Such evaluation measures could include, for example, an analysis of the lesson plan content prepared for each marking period. Do the lessons contain sequential content? Has there been adequate preparation for the delivery of the content in each lesson? Have there been provisions in each lesson to reach every student in the way s/he learns best?

Can the lesson be extended, enhanced for those students who are ahead, and does it allow for review for those who need it? Does the teacher "deliver" or "dictate" the lesson, or involve the students interactively in questioning, and participating in ways that promote deeper understanding?

Another evaluation measure might be a teacher portfolio that demonstrates growth and development over a particular marking period, or in a term. This would suggest that each year's evaluation with the principal, or parent board, or both, would set performance objectives for each teacher, based on areas of challenge and opportunity.

Such diverse measures would certainly provide a fuller, richer evaluation of performance, and, would offer parents, administrators, and taxpayers a better understanding of the complexities of the profession.

Accountability should not be abhorrent to teachers or their unions, it should not rest on a single performance measure, and it should provide a full assessment of the capabilities of individuals whose profession it is to determine when positive learning outcomes have been achieved.

Article Source: http://EzineArticles.com/?expert=Kathleen_Kardaras
http://EzineArticles.com/?Accountability-On-the-Line&id=7280685

Empowering Students: Add Choice to Assignments

Cinematographer Soumendu Ray teaching the students (Photo credit: Wikipedia)By Phil Tabernacle

Should students have a choice in what they learn and what work they do? The question reminds me of sitting in the doctor's office when I was a little kid.

Whenever I went to receive school shots as a child, I remember the doctor would always have me select from one of her two hands she held behind her back. In these hands held the fate of my morale, and somehow I had the amazing gift of consistently choosing the hand which held a square of bubble gum (I must mention that this was around the same time that I thought I had magical powers because I could fall asleep anywhere in the house and wake up in my bed the next morning).

The pain seemed to weaken with the promise of a thrilling Bazooka Joe comic. I bravely faced measles, mumps and rubella with the help of my impervious (delusional) sense of power at having just chosen the correct hand.

Reflecting on how I was given this special opportunity to choose as a child has made me think about the influence choice has on my students when brought into the classroom.

In today's world, "choice" as a positive is contested. Even when speaking in educational terms, choice can quickly divide a room. With this said, I'm going to suggest something that I feel is a bit less controversial, although I can already imagine objections similar to my own when my parents tried to tell me they had just been carrying me to bed.

Bringing Choice into the Classroom

Over years of creating projects and assigning essays to students, I've noticed that when given a choice, students tend to respond more positively. I started slow - students could choose a character in Of Mice and Men and decide if they were round or flat. I started loosening my grip on the gum and allowed students to choose a creative writing project over an essay.

This year, I've added multiple creative writing options, an essay, or a graphic organizer presentation which must be delivered to the class at the end of the unit. I'm not surprised that more students have completed this final assessment than in any year previous.

So let's break down the "gum" and "shots" in providing choice in assessments:

Weighing the Pros and Cons of Student Choice

The Gum

1. Students take ownership over a project (they can't complain that it's too difficult or too boring because they chose it)
2. They can demonstrate their knowledge in a way they feel they can succeed in. You also get to know your student's strengths by observing their choices
3. As a teacher, grading diverse projects is more interesting
4. Students are exposed to a variety of mini-lessons to support the work of the various projects

The Shots

1. Projects, while different, have to be equally rigorous and meet the goals set out at the beginning of the unit - this takes careful thought
2. Guidelines, supports and lessons must allow students to complete any or all of the projects which takes more time
3. Students won't choose to stretch themselves to practice and develop new skills. Everyone practices all the skills, but each chooses one to further develop

Despite the potential drawbacks, I find myself choosing choice more often these days, improving past projects and creating new ones. In order to make your students feel they have a say in the classroom, I suggest always having something they want in that hand behind your back.

Phil is a teacher and writes for TeachHUB.com. Established by teachers, for teachers, TeachHUB.com offers educators recommended classroom tools, professional development, daily lesson plans, and education news. The website is dedicated to improving the quality of education and invests in the opinions of teachers when providing resources and support for both inside and outside the classroom.

Article Source: http://EzineArticles.com/?expert=Phil_Tabernacle
http://EzineArticles.com/?Empowering-Students:-Add-Choice-to-Assignments&id=6942773

The Young Teacher's Guide to the Internet and Assessment

Image via WikipediaBy Richard D Boyce

This article is designed as a starting point for young teachers new to using the internet as a tool in their assessment.

Being proficient at using the internet itself is not a guarantee that you will immediately succeed in using it for assessment. Nor is being inexperienced with using the internet going to prevent you from using it to create an effective assessment task.

The advice that follows is the culmination of my experience introducing the internet as an assessment tool during the late 1990s and the early 2000s. It is important to understand that my staff and I began with varying degrees of experience with the internet.

Below is the advice I would offer to a young teacher joining my staff in the first year of his/her career.

• Make sure the assessment technique is not a test of how to use the internet.

• Always do the test yourself first, looking for 'glitches'.

• Allow more time than you feel is necessary. Computers and networks abide by 'Murphy's Law'. Websites can often be unavailable due to an upgrading process.

• Have more than one internet site available for use if possible.

• Always review the task after you have used it and rewrite it if necessary. Don't disregard it because it didn't work well the first time. It may just need fine tuning to make it a great assessment task. You may need to add extra websites or update your list.

• Try to write instruments that are procedural in nature and don't depend on the website to be visited. Thus, you can use the procedure over and over again changing only the topic/data and the websites to be used.

• Remember to check your answers to the assessment task before marking, as the website/s may have been upgraded since you wrote the assessment task.

• Learn to use a variety of search tools, e.g. Google and Yahoo.

• There are sites that offer assessment items that you might use or adapt, e.g. http://www.funbrain.com had lots of quizzes on a variety of topics in Maths and other subjects that students could do on the internet. The site would email the teacher the results of his/her class. This site may no longer be available but there may well be others of a similar nature.

• It is important to check the websites you use for any bias. If you still wish to use a site, discuss the concept of bias with your class before you give the assessment task based on that website.

• Where students are allowed to find their own websites, they must acknowledge them. This will allow you to test the integrity of the site, if necessary.

• For lower year levels and the less able, make sure the websites you use or recommend are easy to use to facilitate the students getting started quickly on the assessment task itself.

• It is important that the websites you use or recommend have reliable data. Check it yourself beforehand.

• For the most reliable results, all students should have access to their own computer. Sharing computers can be more time consuming and can lead to claims by students that they did not get equal time to do the assessment task. Inexperienced students do take longer to do the tasks.

Remember that you don't need to be an expert web surfer to get started. Start simply. If it doesn't work the first time, change it and try again.

You can always use the results as a guide rather than a final grade or not use them at all. It can simply become a learning experience, both for you and your class.

In conclusion, the internet offers you this great advantage. You get real life data and the relevance to life that your students so often crave in their learning situations.

Rick Boyce taught for over forty-five years. The last fifteen years before retirement he was the Head of Mathematics. He gained a reputation as an innovator in the teaching of Mathematics and introduced the use of internet into the Mathematics assessment program in his school and became a presenter of professional development on the use of internet in assessment to teachers. The article above is but a small taste of what the young teacher can expect to find in an eBook titled "Internet and Assessment" on our website called http://www.realteachingsolutions.com

Article Source: http://EzineArticles.com/?expert=Richard_D_Boyce
http://EzineArticles.com/?The-Young-Teachers-Guide-To-Internet-And-Assessment&id=6884037

How to Do a Simple Reading Assessment

Image via WikipediaBy Celia Webb

Perhaps you have volunteered to teach English to immigrants in your community or tutor a teenager in reading. Maybe you are concerned about your child's reading capability and want to determine whether you need to seek professional help for her. In all of these cases, you need a simple way to perform a reading assessment.

Reading assessments are the first thing to do before you start teaching or searching for additional help. Do not make assumptions about someone's ability. Find out. The answer gives you the starting point for your instruction.

Successful reading instruction depends on building upon previously learned skills. If you skip essential steps in the chain, students have a difficult time leaping from where they are to where you try to start them.

Reading Levels

Determining your student's reading level will guide you in selecting teaching material if you are working as an English language tutor. It will also aid you in working with your school district to further assist your child.

There are three reading levels:

Independent: The student can easily, accurately, and confidently read. The student should be able to correctly pronounce all words at this level.

Instructional: The student makes some errors but can still read most of the material.

Frustration: The student struggles, makes frequent errors, and shows symptoms of nervousness or dislike of the task.

Use instructional level materials during tutoring sessions and assign independent level reading material for homework. For homework assignments use vocabulary and spelling worksheets that reinforce the new vocabulary introduced in the instructional level reading. This approach builds confidence, reinforces the lesson, and properly prepares the student for the next session.

Read-Aloud Test

The easiest way to assess reading level is to do a simple read-aloud test. You can use a reading test available online like the Reading Competency Test sponsored by The National Right to Read Foundation. The test comes with instructions for administering it, grading it, and interpreting the results.

Alternatively, you can present your student with six or seven paragraphs of 25 to 50 words. Each paragraph should be a different grade level. You can determine the grade level by entering the paragraph into a word processor like Microsoft "WORD" and running the spell-check tool.

At the conclusion of the spell-check, a summary appears. At the bottom of the summary is the Fleisch-Kincaid Grade Level score. This score is equivalent to the school grade level of the writing.

So, if the score is 1.5, it means a first grader should be able to read that material. Have your student read the material aloud. The level at which the student makes more than one mistake for every twenty words will be the Instructional Level.

Phonics Survey

Reading is a process of decoding written symbols which represent sounds. Reading is a complete mystery if the student does not understand what symbols represent which sounds. This decoding system is not something people absorb, they must be taught. You can rapidly assess a student's understanding of phonics by doing the following:

Print each set of nonsense words below on a separate card.

Card 1.

TIF NEL ROM (Easy Consonants)

DUP CAV SEB (Short Vowels)

Card 2.

KO HOAB WAJE (Hard Consonants)

ZEEX QUIDE YAIG (Long Vowels)

Card 3.

WHAW THOIM PHER (Consonant Digraphs)

OUSH CHAU EANG (Difficult Vowels)

You will have three cards, each with six words. Explain to the student that you are going to show him a card with made-up words and you want him to pronounce those words. Note whether the student perfectly pronounced all the words, knew some, or knew none.

Anything less than a perfect score indicates more phonics instruction is needed. A "Knew some" score indicates the starting point for instruction. Although I have noted the phonics level in parentheses next to each line, the card the student reads from should only include the nonsense words.

Reading Comprehension Tests

Reading comprehension is the ultimate goal of reading. If you can pronounce all of the words in the correct order, but you do not know what they collectively mean, you can not read. There are reading comprehension tests and worksheets available online (search "reading comprehension test printables").

If you wish, you can make them yourself by providing short reading passages at appropriate reading levels. Next, ask questions which explore the following aspects of the passage:

Details in the passage.

Timeline questions (What happened first? What happened next? When did this take place?).

True and false questions.

What is the main idea of the passage?

Once you know the area of comprehension a reader has not mastered, you can make that area the main focus during the tutoring session.

If you are concerned about your child's progress in reading and the simple assessments point to a possible problem, begin solving the problem by first talking to your child's teacher. Teachers know many stumbling blocks present themselves when a student is learning to read.

A student may need glasses, or have dyslexia. They may not have been taught phonics or may have missed critical lessons due to absences from school, and so on. Most schools today have Reading Specialists on staff to help uncover the reasons why a student is falling behind.

It is important to advocate for your child as soon as you suspect a problem. Students can fall behind quickly and reading skills affect all subjects. Get help right away.

So there you have it, straightforward information on how to assess someone's reading ability. Of course, it doesn't make you an expert reading teacher, but it gives you a place to start and can help you discern if your student needs more professional assistance.

Celia Webb, President of Pilinut Press, Inc., publishers of advanced readers for children and ESL students. Check out http://www.pilinutpress.com for more free articles on developing reading-related skills, word games and puzzles, and activity sheets for the company's entertaining and educational books.

Article Source: http://EzineArticles.com/?expert=Celia_Webb
http://EzineArticles.com/?How-to-Do-a-Simple-Reading-Assessment&id=6867954