Tournament skill level math formula

[Brian]

Brian,

Sorry i haven't gotten back here yet, but I am looking forward to getting into some tournaments. Sadly..i am forced to compete on Hard here because I passed Tier 7.  I can barely pass hard songs at this point, and am borderline competitive on Medium at the gh.com tourneys (top 10's mostly).

Anyways, our tour group just had one which was really cool and well done...but takes a bit of research.

What the guy running it did was, he picked 3 songs and it was  multi difficulty tourney made fair by "handicapping".

The method was simple and the stats on it are really tight, I did some research..

I will quote our tour group members post on the tourney:

I've done some research on The Old Dudes leaderboard, and I'm using that research to grade (or handicap) scores from different difficulties.  For example; the highest score in The Old Dudes leaderboard for "Closer" on Expert is 273,314, the highest score for Hard is 201,884, Medium 185,808, and Easy 80,282.  If you take 80,282 (the highest Easy score) and multiply by 3.40442 you get 273,313.6 (which rounds up to 273,314- the highest Expert score).  The multiplier for Medium is 1.47094, the multiplier for Hard is 1.353817.

Let's say "Closer" is one of the songs for this tournament- Player A decides to play the tournament on Easy, their high "raw" score for "Closer" would be multiplied by 3.40442 and that "net score" would be compared to Player B, who decided to play on Expert (with no multiplier).  The same goes for Player C who decided to play on Medium (with a 1.47094 mulitplier), and Player D who played on Hard (with a 1.353817 multiplier).  What you end up with is a more level playing field for players of all skill levels.

Don't worry- I'll do all the math, determine all the multipliers, and keep all the scores.  I will not be participating in this tournament.  I will determine different mulitpliers for each song in the tournament (which is neccessary, because different songs produce different multipliers- for example: "Sabotage" multipliers are roughly 3.95, 1.73, and 1.06).

So each song has it's own multiplier, based on difficulty level, and it ended up working out eminently fair.  The stats I did on it pointed to use the average of the top 3 scores being even more reliable...but either way it ends up allowing any difficulty to be played.

Our tour group would probably be game for any style tourney though...but the big thing is multi difficulty fairness. The above worked very well.

Once i find a tourney I can possibly compete in, i will be here to play!

Pete

[Brian]

Hmmm...interesting --- I think I'll have to read that a few more times before it all sinks in.

But basically, you just use a formula to convert every score into a "common" score...? So then it's fair to compare an Expert score to an Easy score within the same tournament?   And that multiplier is constant across all songs?

Wow...impressive figuring all that out, but if it sticks and doesn't have any holes in it, then that will work great. That'll change everything. I might be mailing you with more specifics. I'd like to try to apply this at Edison.

[dataguru9171]

Well, it is exactly so an easy score can be compared to an Expert one.

Each song/difficult has its own mutlipler which is basically the highest expert score divided by the highest <other level> score. The guy running this one used the TG leaderboard.  I would probably use the optimal calculated score.  No one played in easy..

This is how Even Flow worked out in our tournament:

Rank    Player            Level    Even Flow       
1      Dataguru1971    Medium    310,745       
2      Basurablanca    Medium    310,424       
3      Dibernardo77    Medium    307,835       
4      Sirenz              Medium    306,319       
5     cenachaingang54Medium   304,558       
6      quevven           Medium    304,496       
7      Frumious          Expert      296,589       
8      Dataylor57       Medium    294,193       
9      Jryall               Medium    291,117       
10    terrence33       Medium    291,055       
11    Rocker74         Hard        284,020       
12    OldmanOil       Hard        274,776       
13    Axegrinder1976Hard        269,907       
14    Littlebruce67    Medium    261,445       
15    Axegrind          Hard        255,551       
16    Barbobus         Hard        251,820       
17    BlackCherry883 Hard        243,037       
18    PantherJim       Hard        226,130       
19    Schottgunn       Hard        222,671       
20    Wabbit99         Hard        213,164       
21    Stormcheese    Expert      208,153       
22    Angrygames     Hard        197,141   

My score was only about 165.5K.  There were two other songs and it worked out okay in those too.  I ended up 10th overall, and the top 10 was equal parts medium/hard/expert.

The tourney ended up seemingly favoring the medium skill, but it has to do with the highest score used in the math.  In our group, a lot of the members scores aren't in the leaderboard he used (the TG leaderboard)..so the actual highest score was not accurate I think.  You could create the mutlipliers based on the calculated optimal scores for easy, medium and hard from the bradleyzoo.com information, or scorehero.

It can probably skewed a bit differently too...instead of the straight multiplier, you might adjust it -10% for medium and easy due to the much more difficult FC's in hard.

[Brian]

Hmmm...well, if every level has a multiplier and Expert is the maximum of scores, can't you just take the multiplier and do each person's score until they get all at the Expert level?

So if there is an Easy player, just multiply their score by 3.404...?

Basically bring everyone's score up to Expert level and then compare? Seems like that would work because the multipliers should be constant.

I'll run some tests on the pool of scores I have for some songs and see if they end up looking reasonable.

[dataguru9171]

Basically that is the idea of the math.  Divide highest expert score by highest easy score to get the easy to expert "conversion", same for medium and hard.

That way they roll-up to the expert level of score.

BUT, there should be an adjustment since optimal on Expert isn't the norm.  That is why in our's the highest scores within the group were used.

Every song is different though...but with a small setlist it is easy to "set" the multipliers.

[Brian]

Ahh...I see. So to do the calculations I'd need to have a conversion table for each song, for each difficulty and run each score through it per tournament.

I looked at the tables from bradleyzoo.com and have come to assume that the text file associated with each score is the output of the song being deconstructed programatically somehow so the scoring is accurate and not based on someone playing it.

I'd probably use the snapshot score from the tables there as that appears to be the optimal average. I know that perfect is far from norm but it keeps things constant for all players. Using the highest score per set is like grading on a curve, no?

Thanks for the continued feedback. This whole theory is very interesting and think it has some legs. And if your experience with using the formulas is not completely out of whack then it must have some stability.

[dataguru9171]

Yes, the scores at bradley zoo are mathematically calculated, and generally within a few points of optimal.  I would say to use the non-squeeze SP path score.

The guy in our group who came up with it (DeanoBoz) had built an excel file that made it easy to keep track (we just sent screenshots of recently played).

I would presume the website database could just have a reference table which had the conversions..that way it would be simple join or lookup to take score to it's adjusted value.

Let me know if you want some help building the table..it can be done pretty easily in excel and imported into any db.

[Brian]

Actually, yeah, if you could build a handy Excel table I can use that in the backend when I get there. The system all runs with PHP so the logging of scores would be done with the class I've written and includes the table. But from Excel I can easily make it a CSV or other form that is easily parseable.

And when it comes down to making it public, I'll certainly give credit to the guy and the table site. We can call it the "DeanoBoz Equivalence Formula" or something. I should hopefully have some time this weekend to try out the conversions on some large data sets.

I'll post here with results and such.

Thanks for your help!

[dataguru9171]

Brian,

All set...took a few minutes is all.

I sent you an excel file with the source optimals (all versions) based on the X360 version of the site. Pretty self explanatory.

If anyone else wants the excel file (wish I had done it before...will come in handy to see the optimal scores..doesn't have the paths however), just let me know.

[Brian]

I just applied the multipliers against a set of score data for La Grange and got the following results:

http://www.angryedison.com/download/lagrange.html

All in all it looks pretty alright. I don't see any outliers that suggest the multis are not reliable. The new score is rounded, so I'm sure that fudges some scores, but at least they're all fudged, so it evens out.

I'm really impressed and ecstatic that this is so promising. This will have a direct impact on tournaments in the most positive way possible. Hopefully the long holiday weekend coming up will give me some time to work on things a bit. This type of awesomeness will take priority over the other Edison projects right now.

I'll try the math against some other songs and post more result tables here.

- Black Magic Woman : http://www.angryedison.com/download/blackmagic.html

[dataguru9171]

The only real issue is the expert tier.  Since optimal is borderline insanity on the GHIII Tier 7 and 8 songs...the expert tier should get some lee-way as well. 

The problem is that optimal is so much easier on medium compared to hard or expert.  You could reduce the medium multipliers by 5% each easy enough, and do add 5% to expert.  That would almost even it out.

[dataguru9171]

can post them sorted by final adjusted score?  YOu want to make sure the top 10 isnt "medium heavy" as per my above post.  However, this works so efficiently, someone can pick whichever level they want to try and play (regardless of skill)--so an expert isn't forced to play on expert.  That person could play on medium and still compete.  That is the beauty of this. Again..it was "DeanoBoz"'s masterminding for our TG tourney that led to this..

[Brian]

Thinking about it today, my only concern with this solution is the fact you bring up --- anyone can then play any level. So an Expert player could play Medium and nail every note with every SP and just rock it. There score would be awesome while a true Medium player competing would not be. But, the system now isn't perfect either and players can do the same thing if they don't play on career mode prior. But this method almost makes that easier. At least by using career percents, if someone wants to maintain a name across the GH.com standings and Edison standings, they'll have to play in career.

It's a tough call the more I think about it.

The evening up solution would open tournaments to more players, which would result in more full tournaments. But I worry about the perceived fairness of such tournaments. While I know the math would work out good, I'm not sure people would accept it any more or better than the way things are setup now.

Any thoughts?

[dataguru9171]

I think it would have to be tested out against existing result sets maybe...perhaps using results from the top couple in one of the weekly tourneys at Gh.com...and just plugging in the songs.  Since the set lists are the same, we could look at the top 5 of each difficulty, use the multipliers and see how it works.

And you are right about the ability to FC medium for an expert player...but to that end, a medium player "can" FC it too.

The point is...max out the points on the song for the difficulty level..if you do that, you can win..even if you max out medium against a max out in expert.

Why don't we take wednesday's Humpday shred scores and send them to you in excel..you can test against the top 5 from each tier of difficulty as a "test case" ?

Like I said, the multipliers can be adjusted.  In our tournament Deano used scores from the actual players in our TG as the "set" points.  In my scenario..it is using optimal..which penalizes expert somewhat.

I am sure the math approach can be done fairly..it is just a matter of taking the existing mutlipliers and having a true "difficulty" adjustment--specifically on the tier 7 and tier 8 set lists and some of the bonus songs.

I see a fairness adjustment of reducing medium's multiplier, and increasing experts..."how much" of an adjustment may be by song.

If I had all the data..I could really make this super-duper fair because I can do some statistical analysis on the whole population.  One thing I considered was looking at scorehero song rankings and taking the top 30 scores for each song and adjusting the multiplier by 1- the percent of FC's

For explanation....something like Slow Ride has 30/30 of the top scores are FC's.  So there would be NO adjustment to the existing multiplier.

As a comparison.. Numbver of the beast as 28/30 FC's (93.3%) for top scores. So the Expert adjustment would 1.0 (existing) + 1-.93 or 1. + 0.07 or 1.07 for the song Number of the Beast.

Do you see my logic there?

playing on expert gets a bump of 7% because getting the FC is so much harder there.

It may be more practical to go to top 100 scores per song, which would just take some time to research.

If you can access the GH.com database for pulling song info..maybe using an average of the top 50 scores to determine the multiplier (instead of "optimal") would make more sense?

[Brian]

Hmmm...well, I have to admit some of that is over my head. I'm honestly not a numbers guy when it comes to statistics. It takes me a while to grasp those concepts - but that sounds like your thing, so I hope you don't mind being the "data guru" haha...

I don't have access to the GH scores whatsoever, at least not anymore than you or another average visitors would have. But if your thoughts include variable multipliers based on score, then it would be more fair to use scores logged at Angry Edison. So rather than have people play against numbers from someone that doesn't compete on Edison, they are numbers that are within context of our tournaments. That data is obviously easier for me to come by.

However, if it's possible to be decently fair, I'd prefer a constant multiplier for each song at most. That consistency will be 1) easier to program, and 2) easier for players to understand and trust. If the numbers being applied to their scores change from one tournament to the next, people might start suspecting something...I know I would.

If you could help me come up with a good solid table of song+multiplier numbers, that would be a HUGE help. I'd trust your numbers and formulas more so than I would anything I try to whittle up. I'll then set up a few side tournaments that people can enter to test and we'll see what happens...

Whatdaya think?