Login Register WSPC 2017 • Schedule • Participation • Location • WSC • WPC • Contact •Search •Recent





Sticky LMI Players' Rating System
   LMI Tests -> Monthly Sudoku and Puzzle Tests270 posts • Page 2 of 11 • 1 2 3 4 5 6 7 8 9 10 11
neerajmehrotra
Subject: Re: LMI Players' Rating System @ 2010-12-17 12:06 PM (#2919 - in reply to #1357) (#2919) Top




Posts: 328
10010010020
Location: India
Another Observation....
In top 10 all players have played 5 or more tests out of 8 rated. This shows playing consistently will keep u high in the rating..........
DreamRose311
Subject: Re: LMI Players' Rating System @ 2010-12-17 8:56 PM (#2922 - in reply to #1357) (#2922) Top




Posts: 24
20
Location: United States
The rating system is really cool, thanks for doing this. One issue though, on both 'change' columns, when you sort by that column, it doesn't quite sort correctly. For instance all numbers starting with 1, whether it's 1 digit, 2 digits, or 3 digits are all grouped together. (it sorta goes 1, 10, 100, 11, 110, 111)...
debmohanty
Subject: Re: LMI Players' Rating System @ 2010-12-28 12:37 PM (#3076 - in reply to #2922) (#3076) Top



1000500100100100202020
Location: India
DreamRose311 - 2010-12-17 8:56 PM

The rating system is really cool, thanks for doing this. One issue though, on both 'change' columns, when you sort by that column, it doesn't quite sort correctly. For instance all numbers starting with 1, whether it's 1 digit, 2 digits, or 3 digits are all grouped together. (it sorta goes 1, 10, 100, 11, 110, 111)...

The sorting works the way it does because of the square brackets around the numbers.
Rakesh, any reason why the numbers have to be bracketed?
rakesh_rai
Subject: Re: LMI Players' Rating System @ 2010-12-28 2:32 PM (#3077 - in reply to #3076) (#3077) Top



Mean Minis (2020) Author

Posts: 774
500100100202020
Location: India
debmohanty - 2010-12-28 12:37 PM
DreamRose311 - 2010-12-17 8:56 PMThe rating system is really cool, thanks for doing this. One issue though, on both 'change' columns, when you sort by that column, it doesn't quite sort correctly. For instance all numbers starting with 1, whether it's 1 digit, 2 digits, or 3 digits are all grouped together. (it sorta goes 1, 10, 100, 11, 110, 111)...
The sorting works the way it does because of the square brackets around the numbers.Rakesh, any reason why the numbers have to be bracketed?
The square brackets are not necessary. They can be done away with, if that helps in sorting on the 'Change' column.
rakesh_rai
Subject: Re: LMI Players' Rating System @ 2010-12-28 2:43 PM (#3078 - in reply to #3077) (#3078) Top



Mean Minis (2020) Author

Posts: 774
500100100202020
Location: India
The LMI Sudoku Ratings are now updated after the December test. Overall, 258 players from 41 countries are included in the rating list now.
rakesh_rai
Subject: Re: LMI Players' Rating System @ 2011-01-15 11:59 PM (#3179 - in reply to #2916) (#3179) Top



Mean Minis (2020) Author

Posts: 774
500100100202020
Location: India
The LMI Puzzle Ratings have now been updated after Puzzle Jackpot.

A total of 236 players from 38 countries find a place in the ratings, including 11 re-entries and 19 new players. Players who have missed the three most recent puzzle tests have not been included in the rating list.

The Top 6 remain the same. motris continues to lead the ratings at 996. The highest gainer has been Alberto who gained 186 points. Amongst the Top 50, pixl has been the highest gainer, gaining 151 points. The highest ranked new player is Valezius at #99.
neerajmehrotra
Subject: Re: LMI Players' Rating System @ 2011-01-26 7:41 PM (#3320 - in reply to #3179) (#3320) Top




Posts: 328
10010010020
Location: India
The LMI Sudoku Ratings have now been updated after the January Monthly Sudoku Test "Prime Exotica".
A total of 268 players are included in the current rating list, including 31 new players and 7 re-entries.

Thomas consolidated his #1 position with a top performance, and remains the only one with a four digit rating. Rishi lost 34 points but remained in 2nd position.

None of the Top 10 gained any rating points. WaterlooMathie was the only new name (re-entry) in the Top 20 at #16. David McNeill gained 17 rating points to move up to #15. Fred76, who tested Prime Exotica, gained 2 ranks as others around him lost a few rating points.

TiiT is the highest ranked new player at #71. Valezius gained 50 ranks to move to #70. In the Top 50, jonm, karin and Psyho were the major gainers.
rakesh_rai
Subject: Re: LMI Players' Rating System @ 2011-02-14 4:18 PM (#3501 - in reply to #1357) (#3501) Top



Mean Minis (2020) Author

Posts: 774
500100100202020
Location: India
The LMI Sudoku Ratings have now been updated after the February Monthly Sudoku Test "Double Delight". A total of 231 players are included in the current rating list.

The Top 3:
1. motris - 997
2. purifire - 913
3. jaku111 - 911

WaterlooMathie, the Double Delight winner, continued his rise with a gain of 60 points and 3 ranks. TiiT and bskbri were the two biggest gainers in the Top 100.

Top 3 from India:
1. purifire - 913
2. Rohan rao - 860
3. gauravkorde - 676

Overall, 37 countries are represented in the ratings. A distribution of the countries with the maximum rated participants is shown below:

distribution of the countries
figonometry
Subject: RE: LMI Players' Rating System @ 2011-02-19 5:43 AM (#3540 - in reply to #1357) (#3540) Top




Posts: 30
20
Location: Canada
I notice I'm missing from the rankings now. (I think I've done only one test since November.) How do you determine whether someone is 'active' or not? How long before you get removed?
neerajmehrotra
Subject: RE: LMI Players' Rating System @ 2011-02-19 9:22 PM (#3541 - in reply to #3540) (#3541) Top




Posts: 328
10010010020
Location: India
figonometry - 2011-02-19 5:43 AM

I notice I'm missing from the rankings now. (I think I've done only one test since November.) How do you determine whether someone is 'active' or not? How long before you get removed?


If you miss three consecutive tests then your name is temporarily dropped from the ratings. You will again enter in the list as soon as you play another test.
Sudoku tests / ratings and Puzzle tests /ratings are totally different entities....and are not mixed together.
neerajmehrotra
Subject: RE: LMI Players' Rating System @ 2011-02-19 9:24 PM (#3542 - in reply to #3541) (#3542) Top




Posts: 328
10010010020
Location: India
neerajmehrotra - 2011-02-19 9:22 PM

figonometry - 2011-02-19 5:43 AM

I notice I'm missing from the rankings now. (I think I've done only one test since November.) How do you determine whether someone is 'active' or not? How long before you get removed?


If you miss three consecutive tests then your name is temporarily dropped from the ratings. You will again enter in the list as soon as you play another test.
Sudoku tests / ratings and Puzzle tests /ratings are totally different entities....and are not mixed together.



Further if you have missed two consecutive tests then you will be shown as N... which is an indication that your name will be dropped in the next list if you dont play the next test.
rakesh_rai
Subject: RE: LMI Players' Rating System @ 2011-02-19 9:34 PM (#3544 - in reply to #3540) (#3544) Top



Mean Minis (2020) Author

Posts: 774
500100100202020
Location: India
figonometry - 2011-02-19 5:43 AM

I notice I'm missing from the rankings now. (I think I've done only one test since November.) How do you determine whether someone is 'active' or not? How long before you get removed?
Hope to see you back in the next ratings update. All your previous scores WILL be considered while calculating the new ratings.
figonometry
Subject: RE: LMI Players' Rating System @ 2011-02-20 2:30 AM (#3547 - in reply to #3544) (#3547) Top




Posts: 30
20
Location: Canada
Awesome, thanks. Maybe now I'll have the motivation to do better!
neerajmehrotra
Subject: Re: LMI Players' Rating System @ 2011-03-01 8:40 PM (#3632 - in reply to #1357) (#3632) Top




Posts: 328
10010010020
Location: India
Updated LMI Puzzle Ratings after Puzzle Zoo (February 2011 LMI puzzle test) are now available. Overall, 186 players are included in the ratings, including 14 new players and 8 re-entries.

Not much of topsy-turvy.......except a few changes in Top 10. Psyho climbs up to #7, gaining 54 rating points after Puzzle Zoo. Willwc enters the Top 10, gaining 14 rating points. Uvo sitting just behind deu and battling hard to reach the second spot.

Rob was the biggest gainer from Puzzle Zoo, gaining 221 points and 113 ranks. Amongst the Top 50, Valezius gained 181 points, and Nyuta gained 95 points. Notable re-entries to the rating list include #19 (Tarotaro) and #40 (ByronosaurusRex).
rakesh_rai
Subject: Re: LMI Players' Rating System @ 2011-03-23 2:32 PM (#3813 - in reply to #1357) (#3813) Top



Mean Minis (2020) Author

Posts: 774
500100100202020
Location: India
Updated LMI Puzzle Ratings after Puzzle Hybrids (LMI March 2011 Puzzle Test) are now available.

motris (992), deu (984) and uvo (955) occupy the top 3 spots.
rakesh_rai
Subject: Re: LMI Players' Rating System @ 2011-03-23 2:54 PM (#3814 - in reply to #1357) (#3814) Top



Mean Minis (2020) Author

Posts: 774
500100100202020
Location: India
The following table shows the highest ranked puzzlers from the countries that are represented in the Top 50:
(The countries are in alphabetical order)

Country Puzzler Rank Rating
Austria euklid 23 647
Canada ByronosaurusRex 39 558
Czech Republic Janka1 13 721
France godisdead 24 640
Germany uvo 3 955
Hungary Valezius 20 674
India Rohan Rao 34 587
Italy forcolin 28 617
Japan deu 2 984
Poland Psyho 7 808
Romania rubben 32 591
Serbia nikola 4 888
Slovakia pista 16 699
The Netherlands Para 27 621
Turkey yureklis 21 673
UK drsteve 29 601
USA motris 1 992
rakesh_rai
Subject: Re: LMI Players' Rating System @ 2011-03-30 11:18 AM (#3897 - in reply to #1357) (#3897) Top



Mean Minis (2020) Author

Posts: 774
500100100202020
Location: India
Updated LMI Sudoku Ratings after Spring Sudoku Test (LMI March 2011 Sudoku Test) are now available.

motris (998), nikola (892) and jaku111 (881) occupy the top 3 spots.
vopani
Subject: Re: LMI Players' Rating System @ 2011-03-30 4:46 PM (#3899 - in reply to #3814) (#3899) Top


WSPC Organizer

Posts: 739
50010010020
Location: India
rakesh_rai - 2011-03-23 2:54 PM

The following table shows the highest ranked puzzlers from the countries that are represented in the Top 50:
(The countries are in alphabetical order)

Country Puzzler Rank Rating
Austria euklid 23 647
Canada ByronosaurusRex 39 558
Czech Republic Janka1 13 721
France godisdead 24 640
Germany uvo 3 955
Hungary Valezius 20 674
India Rohan Rao 34 587
Italy forcolin 28 617
Japan deu 2 984
Poland Psyho 7 808
Romania rubben 32 591
Serbia nikola 4 888
Slovakia pista 16 699
The Netherlands Para 27 621
Turkey yureklis 21 673
UK drsteve 29 601
USA motris 1 992

Sad that India is nowhere near the top.
neerajmehrotra
Subject: Re: LMI Players' Rating System @ 2011-03-31 3:50 PM (#3902 - in reply to #1357) (#3902) Top




Posts: 328
10010010020
Location: India
But we have a better scene in Sudokus....with two persons in top 10.........
motris
Subject: Re: LMI Players' Rating System @ 2011-04-25 8:56 PM (#4240 - in reply to #1357) (#4240) Top




Posts: 199
10020202020
Location: United States
I meant to ask this question after Twist, but it certainly comes up again after the April Sudoku test too. For the purposes of the Puzzle Ratings and Sudoku Ratings, how do you account for the different scoring systems in play in different tests. Do you use strict score only, or time, or absolute rank, or some combination of these? Having a fair way to compare very different tests seems really important to keep this world leader board accurate, and I think many of us would like to know more about how the system works. As Nikola mentioned after Twist, it might be time to stop experimenting so much with scoring and start to use a consistent system that will apply to all tests, to keep the Ratings system as fair and even as possible.

Edited by motris 2011-04-25 9:02 PM
rakesh_rai
Subject: Re: LMI Players' Rating System @ 2011-04-26 6:57 PM (#4260 - in reply to #4240) (#4260) Top



Mean Minis (2020) Author

Posts: 774
500100100202020
Location: India
motris - 2011-04-25 8:56 PM

I meant to ask this question after Twist, but it certainly comes up again after the April Sudoku test too. For the purposes of the Puzzle Ratings and Sudoku Ratings, how do you account for the different scoring systems in play in different tests. Do you use strict score only, or time, or absolute rank, or some combination of these?
At the outset, we would like to thank you for asking this question. Within ourselves, we have been discussing exactly these same things recently. To answer your question, the current system uses strict score only. As long as the score is a function of time, we did not feel the need to include time as a separate factor. This was the case in most tests, where anyone solving the complete (or almost complete) test correctly would get some more points using a factor. But the April sudoku test did not use time as a factor for scoring - it did give 5 points extra but difference between 91 and 101 was considered the same as difference between 116 and 117, for example. We have discussed using rank also but ultimately we had decided against it.

Having a fair way to compare very different tests seems really important to keep this world leader board accurate
We cannot agree more. Thats our objective too - to keep the rating system accurate and acceptable.

As Nikola mentioned after Twist, it might be time to stop experimenting so much with scoring and start to use a consistent system that will apply to all tests, to keep the Ratings system as fair and even as possible.
There can be so many different scoring systems. And, unless we try them in a test, we do not know the efficacy of the scoring system. So, I do not necessarily agree that we should stick to one scoring system (it could become boring, in a way). Lets have the flexibility and freedom of having different systems at play in different tests. I think that the rating system should adapt to the scoring system(s)...and not the other way. Neither can the rating system remain static - it also needs to be reviewed from time to time for continuous improvement.

and I think many of us would like to know more about how the system works.
Currently, we are midway through the process of coming up with suitable improvements to the existing rating mechanism. Once finalized, the "how" part will be shared on the site as well.

At the same time, we would like everyone to share any ideas/suggestions/criticisms - small or big - from the ratings point of view on this forum. And, we will address each one of those while finalizing the system.
debmohanty
Subject: Re: LMI Players' Rating System @ 2011-04-26 9:38 PM (#4265 - in reply to #4260) (#4265) Top



1000500100100100202020
Location: India
As Rakesh mentioned, feel free to post ideas / suggestions on ratings, especially on
How to normalize scores in a test?
How to compare scores across tests?
How to deal with 0 scores / no submissions?
How to deal with players joining late or players not playing frequently?
etc...

Should anyone need sample data, please let us know.
motris
Subject: Re: LMI Players' Rating System @ 2011-04-26 9:45 PM (#4268 - in reply to #1357) (#4268) Top




Posts: 199
10020202020
Location: United States
Thanks for your reply. I'm sure I'm not the only one who will be interested to learn the methodology behind the system as it is a real "world leader board" these days.

My personal recommendation is to not necessarily lock in scoring systems (what puzzles are worth) as these can be varied by authors on different tests to affect strategy, but after the base score of a test is calculated, the only way for the rankings to use score fairly for complete finishers is to either include time in your formula, or always use N points per minute where N is close to "max value of test/time". I like creative scoring ideas, like in Puzzle Jackpot, or special bonuses as in Evergreens or in my 20+10 Decathlon for completing sets of puzzles, but we should realize scoring of puzzles is a different component of the test than correctly using the time of finishers to measure relative performance.

So on April Sudoku, I would certainly have just used ~5 points per minute (600/120). On Twist I would have used the same staggered "value of minutes" for the bonus, instead of going down to a nominal .5 points per minute. So I would have earned 1 minute at 100%, 10 minutes at 75%, 10 minutes at 60%, and 10 minutes at 50%. In each case, simply making the time worth the value of those minutes for other solvers allows the relative performance of all finishers to be correctly staggered for your ratings, for UKPA rankings, and so on. Unlike a one-time WPC where it doesn't matter as much, LMI has really become one of the big forums for monthly competition and so consistency is absolutely required in the ranking system.
motris
Subject: Re: LMI Players' Rating System @ 2011-04-26 9:54 PM (#4269 - in reply to #4265) (#4269) Top




Posts: 199
10020202020
Location: United States
debmohanty - 2011-04-26 9:38 PM

As Rakesh mentioned, feel free to post ideas / suggestions on ratings, especially on
How to normalize scores in a test?
How to compare scores across tests?
How to deal with 0 scores / no submissions?
How to deal with players joining late or players not playing frequently?
etc...

Should anyone need sample data, please let us know.


There are some interesting questions here.

I've thought about the 0/no submission score a bit. I think one option is to put a choice on the test upfront, something like "I want to play the test for the official ranking" or "I do not want to play the test for the official ranking". Either gives you a password and locks in a start time for your account. If you just want to see the test but not record a score, then you can now do so. But if you have solvers who are instead starting the test, and only putting in answers if they feel they did well, you can now separate those from each other. There will also always be the potential for technical problems, so I'm guessing that the rating probably does something like drop/minimize the value of the lowest recent test which can account for one time technical problems that led to a 0 result or particularly low score.

I've also always been surprised that the tests have a fixed time for last submission, as opposed to a fixed time for last start. I suppose advertising a 48-hour window is the point, but if that's the case I'd simply always run the tests for 50 hours. Changing to the latter format would eliminate the situation (that is always frustrating for the solver) of realizing they started too late and have a low score. While this won't be common among the frequent players, for people taking one of their first tests you do not want to cause a negative experience or they may not want to come back. A check on test start times is a huge improvement in my opinion.

Edited by motris 2011-04-26 10:03 PM
debmohanty
Subject: Re: LMI Players' Rating System @ 2011-04-26 10:34 PM (#4270 - in reply to #4269) (#4270) Top



1000500100100100202020
Location: India
0 score/No submission -
I've never thought about that details because I always assumed players don't submit either a) they never intended to submit or b) they had some technical issues.
In context of ratings, it may be possible that some players submit only if they think they did well enough. I've no idea if that is a reality, but we can definitely add an option as you describe.
As of now, we simply ignore 0 scores for ratings computations.

Fixed Start Time Vs Fixed End Time -
This has been pending for a long long time even confusing regular players sometimes. It will be fixed in May Puzzle test (and onwards)
I would like to mention that it does not happen frequently because I always push the end time if I believe some serious player started the test later than when they should have. But I've missed it couple of times ( e.g. here & here). So it is better to be clear upfront.

270 posts • Page 2 of 11 • 1 2 3 4 5 6 7 8 9 10 11
Jump to forum :
Search this forum
Printer friendly version