IHNN - 2022-01-14 8:50 AM

I've voiced my thoughts in a couple of other places, and think that the scoring method is the biggest flaw. I'll go into a bit more shortly, but I want to highlight the things I really liked about this series. Having replays, even if I didn't watch many, is a phenomenal addition that I'd love to see expanded- perhaps the solver could annotate certain moves or the actual times of inputs could also be logged to see where thinking times were? The puzzles themselves were generally very nice across the board showing a lot of variety in the genre, though I think there could have been another easy puzzle and another hard puzzle- there were a lot of middle difficulty ones (and to keep 16, two of these could have been saved for elsewhere). The comments area probably didn't need emoji as it made most of the comments a single emoji- dividing this into "reacts" and "comments" may work better for actually capturing comments? The system being in place is a huge plus though, and most of these are minor opinion based things. As an aside, I would definitely appreciate if there were 2 puzzles open at once instead of just 1, giving a longer period- even for 1 puzzle it's not always easy to fit it in.

What I think is less of an opinion is the scoring system. I do not like going purely by placement but acknowledge it's the simplest way to aggregate results with different amounts of finishers. Expanding the scoring to more solvers, or perhaps having 51st and 52nd get 50 points, 53rd and 54th 49 and so on (or a similar adjustment) may help include more solvers who may have finished near the bottom on most solves. For a first run, top 100 makes sense as I expect the volume of people participating far exceeded what was expected- a great problem to be having!
Even more egregious though is inconsistent penalties. A flat rate just doesn't work with this format, as 2 minutes on smaller or shorter puzzles could easily be 100 placements, while on the larger ones a penalty cost maybe a dozen placements or less. This ends up putting a much greater weight on the smaller and easier puzzles, which is backwards from what's typical. I firmly believe that going forwards, penalties should be based on either a percentage of your own time being added to it (perhaps plus a small flat amount to discourage guessing further) or based on the performance of others- as an arbitrary example, 25% of the 25th fastest error free time on a puzzle.

Scoring is, of course, only relevant to the competitive aspects. The community aspects, the general format, the technical side of getting this to work and the puzzles themselves are all on point already I think. Looking forward to the future of these for sure.