[2010 Ideas] Number Confusion on Scorecards
CharlieCooper (2010-01-27 18:41:58 +0000)
Recently I have encountered many difficulties in reading scorecards that have been written by judges. This causes a lot of problems for the people entering data into the spreadsheet as it becomes a lengthier process and even more worryingly, causes incorrect times in the database. Following a competition I organised in 2009 I had to check all of the scorecards again, a problem I know others have faced. One of the main problems seems to be the differences in European handwriting. We all appear to have been taught to write numbers slightly differently causing a lot of problems with 1/7 and 4/9 in particular.
I'm not sure how exactly this can be remedied, but I think it should at least be discussed. Telling people to just be "clear" in writing their numbers doesn't really work either because a "clear" 4 is a "clear" 9 to somebody else. Would a standardised list of numbers help? Could we print the numbers on the scorecards and then just circle them? I'm afraid I don't have the solution, but I just would like to highlight the issue.
Below are some examples. Some may think that there is no ambiguity in these, but the photos are of the actual cards that were entered incorrectly!
[img:2gg5b5w6]http://i47.tinypic.com/28vbthy.png[/img:2gg5b5w6]
MadsMohr (2010-01-27 21:35:32 +0000)
This is already covered by A7c as it is the competitors responsibility to check if results are written down correctly.
Perhaps we should make this more strict by requiring each competitor to sign each result not just penalties? It does not take that much extra time and it would decrease errors.
Lucas (2010-01-27 22:29:36 +0000)
[quote="MadsMohr":23837xfr]This is already covered by A7c as it is the competitors responsibility to check if results are written down correctly.
Perhaps we should make this more strict by requiring each competitor to sign each result not just penalties? It does not take that much extra time and it would decrease errors.[/quote:23837xfr]
We've been doing this in California for years, and if you asked any of us, we would highly recommend it as an official standard.
CharlieCooper (2010-01-27 22:49:39 +0000)
[quote="Lucas":2glhkjg0][quote="MadsMohr":2glhkjg0]This is already covered by A7c as it is the competitors responsibility to check if results are written down correctly.
Perhaps we should make this more strict by requiring each competitor to sign each result not just penalties? It does not take that much extra time and it would decrease errors.[/quote:2glhkjg0]
We've been doing this in California for years, and if you asked any of us, we would highly recommend it as an official standard.[/quote:2glhkjg0]
Sure, the competitor can confirm that the time is correct, but if he has the same handwriting as the judge it doesn't eliminate this problem as the person entering the data could still see ambiguously written numbers.
jazzthief81 (2010-01-28 00:23:34 +0000)
Charlie is right. Ensuring that the judge writes down the time legibly [i:2xrmrcne]to the competitor[/i:2xrmrcne] doesn't rule out a mistake later during data entry.
I've had to ask for corrections quite a few times. Sometimes I did it on the spot because I saw that a result was wrong on the result sheets that were hung out at the venue. On other occasions I saw that a result was wrong by browsing through the WCA database afterwards and I had to contact the WCA Board and competition organizers to rectify the mistake.
anders (2010-01-28 05:18:04 +0000)
I systematically double-check all the results from the scorecards the night after every comeptition were I have been the WCA delegate. And yes, some numbers are hard to interpret and I have to compare with numbers written by the same judge on other scorecards. Maybe let all judges write a sample of how they write numbers and hand it over to the WCA delegate?
TMOY (2010-01-28 08:48:20 +0000)
I think it might be a good idea. Take for example the 24.19 (on which I recognize my own handwriting perfectly well ): for me there's no ambiguity at all. The 2 cannot be a 8 because I never write 8s like this, same for the other digits. And IMHO a sample of digits written by me would have been enough for everybody to be able to eliminate the ambiguities as well.
qqwref (2010-01-28 09:32:39 +0000)
I agree that we need a standard; a judge saying "well I know this is a 2 because I'd never write an 8 like that" is not good enough if other people might think it's an 8. Having the competitor confirm that it's readable is also not good enough unless that competitor is entering the times into the spreadsheet (which rarely happens).
One possible standard would be as follows: write the numbers as they are displayed on the stackmat, with horizontal and vertical lines, not curves. So a 1 may not have a cross and a 7 must, 4 has three lines and a 9 has four lines, etc. This is an unbiased standard so I think we could conceivably all agree to use it in the future. The lack of curved lines also helps with the ambiguity, so unless the writer is very very sloppy it should be very clear exactly how many lines were drawn per digit, and where.
Pedro_S (2010-01-28 10:26:02 +0000)
Writing the numbers like the stackmat is kinda hard, doing all the little segments one by one...
I like the idea of having a sample from the judges, or simply telling all of them some standard before the competition begins. They probably won't follow it strictly during the whole comp, but this would help decrease the number of ambiguities.
Maybe choose a "trustable" computer font and ask judges to base their writing on it?
I'll check some of the fonts...
BryanLogan (2010-01-28 12:13:17 +0000)
[quote="qqwref":cfj26gyw]One possible standard would be as follows: write the numbers as they are displayed on the stackmat, with horizontal and vertical lines, not curves. So a 1 may not have a cross and a 7 must, 4 has three lines and a 9 has four lines, etc. This is an unbiased standard so I think we could conceivably all agree to use it in the future. The lack of curved lines also helps with the ambiguity, so unless the writer is very very sloppy it should be very clear exactly how many lines were drawn per digit, and where.[/quote:cfj26gyw]
Well, that might add a lot of time to writing things down, especially for the 95% of judges who can write decently. And the ones that can't write decently might not be able to follow this anyways.
I think this just comes down to telling the judges they need to write properly, and those that don't, could be told to do it in this tedious manner.
But I frequently will cross out the time and rewrite it myself if it's written very poorly before I sign.
CharlieCooper (2010-01-28 12:20:24 +0000)
[quote="qqwref":o451m59k]I agree that we need a standard; a judge saying "well I know this is a 2 because I'd never write an 8 like that" is not good enough if other people might think it's an 8. Having the competitor confirm that it's readable is also not good enough unless that competitor is entering the times into the spreadsheet (which rarely happens).
One possible standard would be as follows: write the numbers as they are displayed on the stackmat, with horizontal and vertical lines, not curves. So a 1 may not have a cross and a 7 must, 4 has three lines and a 9 has four lines, etc. This is an unbiased standard so I think we could conceivably all agree to use it in the future. The lack of curved lines also helps with the ambiguity, so unless the writer is very very sloppy it should be very clear exactly how many lines were drawn per digit, and where.[/quote:o451m59k]
This is by far my favourite suggestion so far. It makes entering the data into the spreadsheet just as easy, and there is no need to constantly check handwriting samples. If we had to check a sample we would have to check it every time for every scorecard, and even then this might not solve the issue if the person entering the data has a similar handwriting to the judge. If everybody knows that we have to write the numbers as they appear on the stackmat there is no need to check samples and will reduce errors. Another thought is that collecting samples from everybody that wants to judge would just be a nightmare and would almost be an incentive to have fewer judges just to avoid the complications, which is obviously not ideal.
[img:o451m59k]http://www.istockphoto.com/file_thumbview_approve/399523/2/istockphoto_399523-digital-numbers.jpg[/img:o451m59k]
Using those numbers combined with a real emphasis on competitors checking their times are correctly written would avoid errors.
[quote="BryanLogan":o451m59k]
Well, that might add a lot of time to writing things down, especially for the 95% of judges who can write decently. And the ones that can't write decently might not be able to follow this anyways.
I think this just comes down to telling the judges they need to write properly, and those that don't, could be told to do it in this tedious manner.[/quote:o451m59k]
In some cases though it isn't really a case of writing "decently" it's just a case of different people (or indeed countries) making use of slightly different numbers. As TMOY points out, what he writes is very obvious, but to me it isn't. We just have different standards of what is "decent". Actually, every issue that arose on those scorecards was due to English people misreading other European digits, or the other way around.
MadsMohr (2010-01-28 15:15:40 +0000)
It's true that having competitors sign the results does not eliminate problems for scorekeepers, but two eyes should reduce problems.
Sampling handwriting for judges are not really feasible. With a competition with 50 competitors the majority will be judges and then you would have to check 30+ samples?
Instructing judges to use a specific way of writing numbers are a good idea, but also not really feasible. It's still a problem to get people to read the regulations and I think that it would be hard to perform the instruction at the competition.
qqwref (2010-01-28 21:51:05 +0000)
Pedro, Bryan: I didn't mean that you have to write all the segments, just that the times should look the same. So a 0 is a box, a 1 is a single vertical line, etc. In fact, 0 1 2 5 6 7 9 can be written at once without lifting your pen off the paper. 3, 4, and 8 (box + middle line) will take slightly longer. I know it might take a bit longer to write like this than to use standard handwriting, but I think it will save time for every judge with questionable handwriting, and after a few rounds writing these numbers becomes easier.
Remember, it's only a suggestion - that we'd like the judges to follow this. I imagine that judges with bad handwriting will be asked to adhere to this standard or stop judging, while judges with very clear handwriting will be let alone.
unscarred (2010-02-06 23:57:00 +0000)
I'm not pretty sure but I thought the file Masayuki Akimoto sent me for scoresheets are used by many. I'm sure they will share it to everyone sooner or later. As far as scoresheets is concern, it worked remarkably. It really won't give scoretakers a hard time reading hand writings of any judge and they are similar to the one posted here but of course way better. I just hope they share it soon. To give you an idea, it will make encoding faster and very minimal error. You will never call the judge again to verify what number did he/she wrote on that scoresheet.
Pedro_S (2010-02-07 01:28:45 +0000)
I just had an idea (maybe is similar to what Masayuki did)
We could have "shadows" of the numbers, so all the judge has to do is follow the segments and "draw" the number
Have a big sheet of paper showing what the numbers should look like, and...done
8:88.88 (in a light gray)
So you just "fill" the appropriate segments to form the numbers
Lucas (2010-02-09 01:04:14 +0000)
[quote="Pedro_S":13qh3jr2]I just had an idea (maybe is similar to what Masayuki did)
We could have "shadows" of the numbers, so all the judge has to do is follow the segments and "draw" the number
Have a big sheet of paper showing what the numbers should look like, and...done
8:88.88 (in a light gray)
So you just "fill" the appropriate segments to form the numbers[/quote:13qh3jr2]
This is actually a nice idea.
Other judges around here write "13.37+2=15.37" for penalties, and there might be other cases where the score might need to be something else formatted differently from such a standard. But if it's light enough, it should work most of the time, and can be overwritten if necessary.. I'll mention this to Adam and Tyson, and see if it's feasible.
Lucas (2010-02-09 01:04:55 +0000)
[quote="Lucas":25iw7pbq][quote="Pedro_S":25iw7pbq]I just had an idea (maybe is similar to what Masayuki did)
We could have "shadows" of the numbers, so all the judge has to do is follow the segments and "draw" the number
Have a big sheet of paper showing what the numbers should look like, and...done
8:88.88 (in a light gray)
So you just "fill" the appropriate segments to form the numbers[/quote:25iw7pbq]
This is actually a nice idea.
Other judges around here write "13.37+2=15.37" for penalties, and there might be other cases where the score might need to be something else formatted differently from such a standard (I'm worried about current/future irregularities and inexperienced judges). But if it's light enough, it should work most of the time, and can be overwritten if necessary.. I'll mention this to Adam and Tyson, and see if it's feasible.[/quote:25iw7pbq]
Pedro_S (2010-02-09 01:14:01 +0000)
[quote="Lucas":1l0cvqf4][quote="Pedro_S":1l0cvqf4]I just had an idea (maybe is similar to what Masayuki did)
We could have "shadows" of the numbers, so all the judge has to do is follow the segments and "draw" the number
Have a big sheet of paper showing what the numbers should look like, and...done
8:88.88 (in a light gray)
So you just "fill" the appropriate segments to form the numbers[/quote:1l0cvqf4]
This is actually a nice idea.
Other judges around here write "13.37+2=15.37" for penalties, and there might be other cases where the score might need to be something else formatted differently from such a standard. But if it's light enough, it should work most of the time, and can be overwritten if necessary.. I'll mention this to Adam and Tyson, and see if it's feasible.[/quote:1l0cvqf4]
Just have them write the final time. Makes things so much easier...
I have a "prototype". Just asked my friend to change the font to a simpler one, with just lines (instead of the angled segments that appear on the stackmat) so it's easier to follow. Hopefully tomorrow I'll have it.
(note to self: tell all the judges to write just the final time)
Lucas (2010-02-09 06:19:34 +0000)
[quote="Pedro_S":1cixebwy]
Just have them write the final time. Makes things so much easier...[/quote:1cixebwy]
We like to record the fact that +2 was added. And the format for that is "15.37+", which apparently some people think looks like the +2 has yet to be added.
Perhaps a teeny checkbox for "penalty was added on this solve"?
qqwref (2010-02-09 08:09:09 +0000)
I'd rather we just didn't notate penalty in a competition. It just makes things more confusing, and it doesn't get entered into the spreadsheet anyway. If you really want to still be able to have it, you could always add an extra column labeled "penalty", but make sure to label the time column "final time" or "time incl. penalty" or something like that.
Lucas (2010-02-09 08:24:53 +0000)
[quote="qqwref":12qzodeu]I'd rather we just didn't notate penalty in a competition. It just makes things more confusing, and it doesn't get entered into the spreadsheet anyway. If you really want to still be able to have it, you could always add an extra column labeled "penalty", but make sure to label the time column "final time" or "time incl. penalty" or something like that.[/quote:12qzodeu]
Perhaps you're right, but I only agree because the penalties don't matter compared to the final official result.
How's this?
[img:12qzodeu]http://archive.garron.us/img/2010/score_card_light_digits_512.jpg[/img:12qzodeu]
MadsMohr (2010-02-09 09:11:56 +0000)
That's a pretty good idea and result sheet Lucas. Very nice.
[edit]
How would you enter "DNF" / "DNS"? And how would you enter results for multi bld?
Pedro_S (2010-02-09 10:26:03 +0000)
Indeed, I don't see any reason to notate penalty. It's not like solves with penalty will be treated any different.
I like your result. Still have to see how mine will turn out, hopefully later today (you know, I'm in a different time zone...just woke up )
qqwref (2010-02-11 01:25:58 +0000)
[quote="MadsMohr":38425ixg]How would you enter "DNF" / "DNS"? And how would you enter results for multi bld?[/quote:38425ixg]
One possibility:
- For DNS or DNF, just write it. Even if it is messy, it shouldn't be mistaken for a time.
- For multi, write the number of solved cubes in the first two positions, then / (or a space), then the number of total cubes in the last two positions. Assuming you know the scoresheet is for multi, this is completely unambiguous. I don't see anyone doing more than 99 cubes given the hour limit.
This scoresheet might have problems with mistakes, though. If you write a time wrong there is no place to put the correction. I don't know how to fix this problem (using pencils won't work because the scoresheets we use are carbon-copy type).
anders (2010-02-11 02:18:31 +0000)
We had similar score sheets as Lucas suggests at Malaysian Open 2009. It did not work because the light grey shadow numbers did not turn out well on photocopies and the whole thing became very messy, and in the end we did not use it. With a good pen and good printouts, I think it might work. Otherwise it will not work well.
Masayuki (2010-02-14 09:16:15 +0000)
Hi,
Attachment is the record form which we recently use.
The original record form sheet is from Kim. (Thanks Kim!)
We added 7 segment system but it was not so easy for judges to write numbers.
Several judges complained.
It is too bad to draw numbers following guide.
So we added 2 dots system and examples, too.
This record form was not bad.
If you like, we can share.
Masayuki
Pedro_S (2010-02-18 13:47:20 +0000)
[img:2ijztj6v]http://img251.imageshack.us/img251/9439/rubikssumulaabc2.jpg[/img:2ijztj6v]
Finally
I like it a lot, did some writing and works good. I left the sheet at home, will upload a picture once I'm there.