WSPC 2017 • Schedule • Participation • Location • WSC • WPC • Contact •Search •Recent
 Puzzle Marathon — 21st-29th January | |
| LMI Tests -> Monthly Sudoku and Puzzle Tests | 168 posts • Page 6 of 7 • 1 2 3 4 5 6 7 |
prasanna16391 |
| ||
2021 World Sudoku+Puzzle Convention Organizer Posts: 1809    Location: India | debmohanty - 2012-01-30 8:41 PM Or, we can say for authors, it should be best 8 of 9, and then multiply the score by 9/8. Can we make mine best 4 of 9 for this one and then multiply?  Anyway, on a serious note, I personally was fine with this format because I considered it a bonus in itself to co-author and participate in the same test. However, if everyone feels that what Stefano or Deb have suggested is fine to implement next time, then all the better! | ||
| Para |
| ||
Posts: 315 Location: The Netherlands | |||
| Cyclone |
| ||
| Administrator |
| ||
  Location: India | |||
| Administrator |
| ||
Location: India | |||
| davmillar |
| ||
Posts: 44 Location: United States | |||
| macherlakumar |
| ||
Posts: 123 Location: India | |||
| Administrator |
| ||
Location: India | Time for some statistics - Number of Puzzles Vs Starts / Finishes / Bonus It is extremely pleasing to see that 124 players finished all 10 puzzles. Also, on an average (of 259 players who submitted at least 1 puzzle), a player finished 7.4 puzzles.  And because of the interesting bonus system, here is how the 20th / 50th score compared against the top score.  | ||
| Administrator |
| ||
Location: India | From the feedback page :  There is a bit of story behind Graffiti Snake. The first version of IB had "Paint By Number". We wanted to include PNB because, as per Rohan's word, PNB is king of big puzzles. But before I asked Serkan to make PNB, I suggested that he should make a Graffiti Snake instead. (I get 1% credit for selecting the type, and Serkan gets 99% credit for making a beautiful puzzle) Next 3 sets of graphs pretty much suggest that the corresponding aspects of the test were close to perfect.    Many players voted that they would use Online solving if provided. We will try to provide that next time, but no promises. We face enough issues while supporting both online and paper for Sudoku tests.  Of 149 players, 144 players voted that they would participate in future marathons, either as player or as player + author. | ||
| aldentea |
| ||
| reesylou |
| ||
| debmohanty |
| ||
 Location: India | debmohanty - 2012-01-30 6:30 AM Puzzle Marathon is now over. Congratulations to motris, MellowMelon and Kota for winning. Individual puzzle wise, there were 6 winners. Volxa (Kakuro), motris (Loop The Loops, Samurai Sudoku, Braille Word Search, Pentomino Areas), xevs (Graffiti Snake), Para (Black And White Loop, Small Regions), ACM (Tapa) And Serkan (Different Neighbours) Why did I have to wait till someone point me that?  | ||
| debmohanty |
| ||
Location: India | reesylou - 2012-02-01 6:10 AM I'd really appreciate someone giving a break down of an entry point into Different Numbers - I really struggle with these and got absolutely nowhere with this particualr one. There is cheeky start to the Different Neighbours at the top right corner. Note that X has to be 1 or 2, otherwise the top right is not solvable uniquely. Then transferring the 4 we get that the 2X2 cell can only be 3.  | ||
| macherlakumar |
| ||
Posts: 123 Location: India | |||
| debmohanty |
| ||
Location: India | |||
| debmohanty |
| ||
Location: India | aldentea - 2012-02-01 5:35 AM Yes, that is a bug. Too late to fix though I found a technical issue at the result page(score.asp) ... columns of each puzzle are sorted improperly when they are displayed in 'Points'(not 'Submission Time').  | ||
| forcolin |
| ||
 Posts: 172 Location: ITALY | I have done some analysis of the score and bonus system. First of all, in my opinion the scoring system in this contest was very good because every player had a realistic possibility of gaining bonuses on most puzzles, which means that the rank is very close to the sum of the times obtained in the individual puzzles. There is an exception, the Samurai Sudoku which was much more difficult than the remaining puzzles, and in which only 20 players were awarded bonus points, and which even the best solver (motris) had to drop as his worst result. Overall, the number of players gaining a bonus was 1108 out of 1927 (57.5%), relatively high, and this is the distribution among the various puzzles.  If we consider the percentage of players which gained a bonus as a measure of the difficulty of the puzzle, we must conclude that the easiest of the puzzle was the Braille wordsearch, with 84% of the submissions gaining a bonus. This was originally indicated as an AVERAGE puzzle. Which means that an attempt to allocate different times (or different bonus thresholds) to puzzle of different difficulties as proposed by detuned, may be affected by wrong evaluation of the difficulty. I have also analysed the proposal of awarding bonuses only to those players completing a puzzle within a fixed time (30 minutes) from the top solver. The total number of players earning a bonus in this case would be of 786, 40.8%, and the distribution is the following.  In my personal opinion, this system would be much worse. Not only the peculiarity of the Samurai sudoku is not solved (of course, 30 minutes margin on a very though puzzle means much less for a tough puzzle than for an easy one) but the total number of bonusus decreases dramatically, punishing the players earning 10-15 points with a solution time between 40 and 50 minutes. Overall, almost 350 submissions would earn no bonus at all, and this would be concentrated in the middle category solvers. Also, for those players, the average bonus would be reduced therefore the score would privilege a player with a very good time in just one puzzle against a player with decent times overall, and I do not think this is (or should be) the target of this competition. I have tried to develop a different system. This is based on assigning to the top solver a bonus of, say, 50 points, to define a bonus threshold to n times the time of the top solver, and to calculate the bonus by linear interpolation between these two values. I have prepared 3 scenarios, with n= 4, 5 and 6 respectively. Which means that a player would earn a bonus if his/her time was 4, 5 or 6 times the time of the best solver, or better. The following distributions are obtained    The total number of bonus scores is 905 (46.9%) for n=4, 1118 (58%) for n=5 and 1275 (66%) for n=6. All these calculations give a better distribution of the scores among the puzzles, (the number of players earning a bonus on the samurai is now comparable to the other puzzles) and the situation which better approaches the system adopted is for n=5. Of course this system would give different results in terms of final ranking, benefitting mainly those players which had a good time on the Samurai, but not good enough to gain a bonus on it. The negative consequence is that, with the system adopted for this competition, it was clear that after one hour from starting a player could put the puzzle in a corner to solve it the next day. With this system it could be possible (except for the very first player to start a puzzle) to show a “current bonus threshold” as an indication about when a player could give up, and also as an indication of the level of difficulty of the puzzle as required by Puzzlescot, but this indication may change with the time, as strong players will set up best times. Overall, I think that the system adopted for Marathon number one has the advantage of being simple, and could be adopted again without variation if only the organizers will avoid to use puzzles with a remarkable difference in difficulty, such as the Samurai Sudoku. If a new system has to be adopted, a calculation based on a bonus threshold of minimum 5 or 6 times the time of the best solver can be an improvement and could allow to use puzzles of different level of difficulty, but I am convinced that very difficult puzzles requiring the average solver much more than an hour to be solved, should be avoided for practical reasons. Excel Analysis : http://logicmastersindia.com/M201201P/MarathonSolvingTimes_forcolin... | ||
| debmohanty |
| ||
Location: India | |||
| motris |
| ||
Posts: 199 Location: United States | This is incredible data and I'm glad to finally have something like this in hand. I'm not sure what Stefano is trying to optimize (uniformity of percent achieving bonus? - is this really the relevant parameter?) and I haven't had time to dive too deep into the info myself. But I think the most fascinating graphs so far are just looking at the trends in time for each puzzle across the top 100 solvers and seeing how using the "nth solver" at any point is a good measure of the relative ranking of a puzzle's difficulty. I've linked to two images, one with a view of the whole test and one with just a view of the first hour which cuts Pentomino, Kakuro, and Samurai from the top 100 solver graph but gives a much better picture of the other puzzles. I think the data establishes a clear order of Tapa < Loop The Loop < Braille/Small Regions < Diff Neighbors/Graffiti < Black and White Loop < Pentomino/Kakuro < Samurai. Notice that top time is probably the worst of the 100 choices for ranking the difficulty of the puzzles (and therefore the worst to use to normalize by multiplication or other means). Looking at the 10th solver (95th percentile) seems much better though. The top time suggests Pentomino is slightly easier than Black and White Loop. The 10th place (or any spot from 10-100) shows it is a 20-25% harder puzzle than the Black and White Loop for the vast majority of solvers. These graphs also show me good characteristics to fit to either a rank-based or a normalized scoring model to get all the puzzles back on par with each other. The linear nature actually suggests rank may be best, with perhaps 150 to the top solver, 149 to the second, down to 100 for the 51st and later. For only Samurai, which we agree is too hard, would this system break down at one hour. But I do think you need to treat Kakuro and Pentomino differently from the easiest 6, and maybe even Black and White Loop as well. I disagree that only Samurai was an outlier on this test, and I'll let these graphs speak for themselves on that point. Edited by motris 2012-02-02 10:14 AM (top100.png) (top100-zoom.png) Attachments ----------------  top100.png (60KB - 0 downloads) top100-zoom.png (87KB - 1 downloads) | ||
| Realshaggy |
| ||
Posts: 69 Location: Germany | |||
| reesylou |
| ||
| detuned |
| ||
Posts: 152 Location: United Kingdom | |||
| macherlakumar |
| ||
Posts: 123 Location: India | |||
| motris |
| ||
Posts: 199 Location: United States | I've now gone ahead and played with the scoring model and tried the 150 for 1st, 149 for 2nd, 148 for 3rd, down to 100 for 51st through last finisher. I like this system a lot because it makes all puzzles equal for potential bonus. Each puzzle has a total of 1275 bonus points split between the top 50 finishers. This means an "easy" puzzle will not give too much bonus to too many solvers. A "hard" puzzle will not give too little bonus to too few solvers. Each puzzle has same final value. Obviously, the choice of top 50, and the linear progression of bonus, were arbitrary and can be adjusted for a given test. I kept the best 9 out of 10 approach. I've attached the stats for the top 20 (yellow shading is the dropped puzzle with rank scoring and I've shaded all tied options where those exist, red font is the dropped puzzle with current scoring). The average rank column is for the top 9 puzzles for that solver, but shows that deu and kota were very close and para and misko also very close in overall performance based on rank across the test. The time scoring had a different rank in these cases. I have also attached my excel spreadsheet if someone wants to play with this type of system further. I'm already looking at how to use it on my next decathlon test. Edited by motris 2012-02-05 3:29 AM (rank-example.png) Attachments ---------------- rank-example.png (55KB - 1 downloads) rank.xlsx (64KB - 10 downloads) | ||
| Tablesaw |
| ||
| 168 posts • Page 6 of 7 • 1 2 3 4 5 6 7 |
| Search this forum Printer friendly version |
