|
|
Posts: 199
 
Location: United States | kiwijam - 2011-10-10 1:57 PM
I just completed the practise test with entering multiple wrong answers for different puzzles, but only got a single -4 penalty at the end. I should have had more penalties.
Can you review my submission history (if it is saved) to check if there is a bug in this process?
Possible reasons I can think of might be from clicking the Submit All button rather than submitting one at a time, or from closing the page and logging in again later.
You probably ran into a feature, not a bug. It is hard to shut off the Submit All system for solvers, but there are strings we would never ever consider a wrong answer, for example "HI MOM" for the sudoku puzzle. If you manage to submit an answer based on Submit All that could not have otherwise been entered, it does not count against you. We expect the only answers that will fit this category is a solver typing the Easy in the Hard space, and only then when not using the individual submission button.
The only wrong submission I'm seeing from you was a test of axcaahbbec on tents that became axcaahbbeb. This does look like an answer to the tents, and was incorrect and became correct.
|
| |
|
Posts: 187
Location: New Zealand | Thanks for clearing that up, yes I did enter some other "HI MOM" answers to see what would happen. I assume all the Easy puzzles have different answer-key lengths to their Hard partners. |
| |
|
Posts: 199
Location: United States | kiwijam - 2011-10-10 2:48 PM
Thanks for clearing that up, yes I did enter some other "HI MOM" answers to see what would happen. I assume all the Easy puzzles have different answer-key lengths to their Hard partners.
Yes, within each puzzle type the keys are unique lengths. I'm not trying to get people negative scores after all! So common mistakes like typing the four number easy answer in the five number hard answer box will be met with no clickable button, or an X after Submit All for "wrong format", but not -4 points. Only answers that could be marked correct will be graded and earn credit or penalties.
There are still a few puzzle types with similar names (the two Loops, for example), where it might be possible to submit an answer in the wrong spot and be marked wrong. We will consider removing penalties in cases where it is clear that this was a result of such an error, but are hopeful that the formatting of the test and solution page will minimize this kind of mistake that does happen at a low rate on these tests. |
| |
|
Posts: 199
Location: United States | In response to helpful comments from play-testers, the graphical presentation of Almost Simple Loop will change from the original format in the instructions. Instead of black squares with white numbers and arrows, which can be harder to cross out when solving, the clues will now be in gray squares with black numbers and arrows as in the attached image. The word "black" in the puzzle instructions has also been replaced with the word "gray" to account for this presentation change.
A revised instruction booklet with new images for Almost Simple Loop and some other typographical fixes is now posted.
 (AlmostSimpleLoop.png)
Attachments
----------------
 AlmostSimpleLoop.png (2KB - 4 downloads)
|
| |
|
 
Location: India | Instructions in Chinese - http://www.sudokufans.org.cn/forums/index.php?showtopic=312 |
| |
|
Posts: 199
Location: United States | Password protected booklet uploaded. It has 10 pages, one page per puzzle type with the easy (20 point) and hard (50 point) puzzles on the same page. There is no cover page.
REMINDER: This test marks the debut of INSTANT GRADING, a new system of grading; if you have not yet practiced using this system, please go to this practice page before the contest using the instruction booklet answers for submission.
Edited by motris 2011-10-14 9:46 PM
|
| |
|
Posts: 3
Location: United States | The end date for the contest displays as October 16/17, 2012 instead of 2011.
2011 Double Decathlon ends at 10/16/2012 5:00:01 PM local time | 10/17/2012 12:00:01 AM GMT |
| |
|
Location: India | changed to 2011. |
| |
|
Posts: 30
Location: Canada | I LOVE the instant scoring. Thanks! |
| |
|
Posts: 199
Location: United States | figonometry - 2011-10-14 7:18 PM
I LOVE the instant scoring. Thanks!
Looks like it worked out for you exactly as intended (recovering points you might not otherwise have earned). We'll most likely run a poll right after the contest to gauge people's responses to the system. It obviously is not the easiest system to use for all puzzle tests, but seems to be an interesting compromise that online solving can allow.
EDIT: Actually, we've added the poll at the top of this thread now.
Edited by motris 2011-10-15 12:30 PM
|
| |
|
Posts: 328
Location: India | Wonderful...............thanks Thomas for such a nice puzzle test....ofcourse it was much beyond my capacity.... |
| |
|
Posts: 13
Location: Poland | Wow, either I'm out of shape or this test was really hard. Very nice puzzles, the ones I managed to crack at least. Also, really like instant scoring! |
| |
|
Posts: 2
Location: United States | So, I used the individual submit buttons, which kept updating a score on the left. But there was also something there that said "1 Correct, 0 Wrong" (or something like that). That figure didn't update. What was that supposed to tell me? |
| |
|
Posts: 199
Location: United States | dave8mcrae - 2011-10-15 11:32 AM
So, I used the individual submit buttons, which kept updating a score on the left. But there was also something there that said "1 Correct, 0 Wrong" (or something like that). That figure didn't update. What was that supposed to tell me?
That figure was telling you what was true of your most recent submission. It will only ever have more information like "3 correct, 1 Wrong" if you submitted more at a time using "submit all". This does seem like it could be slightly confusing so we can review the report for those doing individual submit if we use this system again.
Edited by motris 2011-10-16 1:18 AM
|
| |
|
Posts: 172
Location: ITALY | All contests on LMI are of good quality, but this one is well above the norm. Excellent stuff, I liked particularly the hard Loop the loop and battleship sudoku.
Also, the instant grading saved me a lot of points, two copying/typing errors and a genuine solving error which I could rectify. The negative effect is that probably I did not pay much attention when typing because I knew there was a second chance.....
stefano |
| |
|
CTC & TVC Author & Organizer
Posts: 183
Location: Turkey | First of all I solved all IB puzzles for preparation :) Normally I don't, but this time I tried to push myself to understand puzzle rules/competition rules before the contest. Also I should say that IB puzzle are really fun! After solving those I was looking forward to compete with real ones.
Secondly I am glad with my result although I couldn't get points that I should get, at least in my opinion. I solved one big puzzle in last 5 minutes but my time was not enough to submit my solution. Also I had solved one puzzle of all types but I lost myself in some puzzles and of course it caused me to lose my strategy, and I couldn't. But I am glad with my performance.
Your puzzles are great! They have nice looking, very satisfying solving paths; and of course new point system is cool! You made a great job, thank you so much, Thanks to LMI and shining man Deb :) |
| |
|
Posts: 31
Location: United States | Thanks for a great test! I liked the scoring system, too, both the structure of the bonuses and the penalties with the instant grading. I enjoyed all the puzzles, but particularly the same two that forcolin mentioned, though I still need more time to finish the rest of the test to see if there are some gems there that I didn't want to attempt with time pressure. |
| |
|
Classics & Regions (PR 2016/17) Author
Posts: 170
Location: Germany | Loved the test, and the scoring system. The instant grading might have made me a little more careless than usual. Three genuine mistakes in reading off the code feels like a lot for me. I'm amazed I was able to make the same mistake on both "Almost Simple Loop" puzzles!
It did seem the instant grading slightly affected my solving: On one or two puzzles, after I finished them up with some intuition, I used the submit button to verify the solution, instead of double checking by hand. |
| |
|
WSPC Organizer
Posts: 739
Location: India | Very good set of puzzles. Thanks Thomas!
I liked everything about the scoring system. I just wanted to throw open a point that comes to my mind. Should we have different penalties for different puzzles? (High-point puzzles have greater penalty?) Maybe not very large, but at least some amount of distinction. |
| |
|
Posts: 152
Location: United Kingdom | So with this new system, I think I was more careful about entering keys then normal, conscious of the four point penalties. And no mistakes!! (at least mistakes I didn't catch, seems I'm a little rusty from not doing any LMI tests in ages). So yeah, thumbs up from me on this system. I'm sure it'd save me lots of future grief, however I'm not sure it should be implemented on every test. Instantly knowing when you have a puzzle right or wrong doesn't accurately match up with an offline solving experience, for instance... |
| |
|
Posts: 199
Location: United States | Rohan Rao - 2011-10-16 6:27 AM
I liked everything about the scoring system. I just wanted to throw open a point that comes to my mind. Should we have different penalties for different puzzles? (High-point puzzles have greater penalty?) Maybe not very large, but at least some amount of distinction.
I thought a lot about different implementations; certainly the existing typo standard of 80% would suggest a larger penalty but I think, given the time put into solving the puzzle versus the time to enter the submission, it is excessively punitive (should it be 4 points and 10 points on this test, for example?). I will say that one change I would consider looking over the results is possibly an escalating penalty if making many errors on the same puzzle. It also seems possible to use the time to fix an error to split the cases (typos are fixed quickly, puzzle errors most often take 2+ minutes), but this could also be risky for some types of errors.
Considering all these options, I actually prefer the simplicity used here, just one kind of penalty and it is the same everywhere.
detuned - 2011-10-16 9:22 AM
So with this new system, I think I was more careful about entering keys then normal, conscious of the four point penalties. And no mistakes!! (at least mistakes I didn't catch, seems I'm a little rusty from not doing any LMI tests in ages). So yeah, thumbs up from me on this system. I'm sure it'd save me lots of future grief, however I'm not sure it should be implemented on every test. Instantly knowing when you have a puzzle right or wrong doesn't accurately match up with an offline solving experience, for instance...
I expect a few people to make this argument and it may be why several of the better solvers (uvo, melon, Para) have voted negative or neutral on this system. My view is offline contests are offline contests and online contests are online contests. They can borrow from each other at times and innovate and do new things at other times. I would compare this system to being in a playoff round at a WPC and turning in each puzzle as finished. After a fixed amount of time you get a signal if you are correct or not. So it is an offline test mode, just not one people have a lot of experience with. The penalty is set to act like the equivalent WPC penalty, which costs you a small amount of points/time, but puts you back in control of fixing whatever mistake you made.
The results in the stat page so far should reveal at lot of the errors solvers make are "online only"; their paper probably has a correct solution but entering a particular piece of information doesn't come through with high fidelity. Since I'm not grading entire grids, I'm happy to experiment with a system that helps remove the "online only errors" from other errors. I think this has gone very well on this test, and Deb has done a very good job realizing the scoring system I wanted.
Edited by motris 2011-10-17 12:05 AM
|
| |
|
Posts: 152
Location: United Kingdom | motris: I have a lot of time for your argument, and I don't think anyone can argue that this hasn't been one of the better LMI test scoring innovations (noting that as I've previously argued, these LMI tests are the perfect playground for these innovations). I'd definitely like to see this repeated in future tests. Just, I guess, not *all* of them. |
| |
|
Posts: 6
Location: United States | I got booted off the internet before I had a chance to enter my answers. I guess I should have been entering them as I solved them, but is there anything I can do now? |
| |
|
Posts: 199
Location: United States | jalbert - 2011-10-16 1:48 PM
I got booted off the internet before I had a chance to enter my answers. I guess I should have been entering them as I solved them, but is there anything I can do now?
This is unfortunately a problem that comes up with online tests and there is nothing we can do to give you an "official score" after the two hour clock has run out. Submitting before the end of the test (sometimes putting in what you have done after 90 of 120 minutes) is probably a good approach for the future. I hope you had fun with some of the puzzles despite the answer entry frustration. |
| |
|
Posts: 3
Location: United States | I enjoyed the test and the immediate scoring. I did notice a bit of a difference (about 45 seconds?) between the timer and the submitted time. I ran out of time on the last one according to the countdown, and submitted it (and the resubmit after a typo), and it accepted it, showing as within the 120 minutes in the scoring details. Not sure if that's atypical, or there's a short amount of cushion or something? |
| |
WSPC 2017 • Schedule • Participation • Location • WSC • WPC • Contact •Search •Recent
2011 Double Decathlon — LMI October Puzzle Test — 15th and 16th October
