A 5 year review of world wide elite scores

HortonHorton Posts: 27,160 Administrator
edited August 12 in Other Stuff

I set out to review the data from the last 12 months. I expected to find that some countries or sites that had abnormally high scores a year ago did not this year. This turned out to be somewhat true but when I sliced the data a number of ways the bigger dataset turned out to be the most interesting and perhaps the most defendable.

The graph below was created with the following criteria:

  • Data was mined from http://www.iwsftournament.com/homologation/idopenslalom0419.php
  • Skiers #2 through #11 from the current world standings list - Smith was excluded because he skis in Europe less that the other skiers and because his scores tend to be an outlier
  • Scores were achieved between April 2014 and August 2019
  • Countries shown on the chart had at least one score of 70 balls (4 @ 41) or greater

How to read the Gray bars

  • These 10 skiers posted scores from 1717 rounds between April 2014 and August 2019
  • The grey bar is the number of rounds per country divided by the 1717
  • The grey bar is the percent of total rounds per country
  • Example – There are 220 scores in Italy. 220/1717 = 12.81% of all scores in the data set

How to read the colored bars

  • The bars include the number of times any / all of the skiers scored at or above 68, 70 or 72 balls in each country.
  • That total is then divided by the total number of scores from in each country.
  • Example – in the USA, 68 balls (2 @ 41) or more was run 414 times and there were 775 total scores in the USA data set. 414 / 775 = 53. In other words these skiers ran 2 or more at 41 balls 53% of the time.
  • The bars show the percentage of scores at or above 68, 70 or 72 balls per country.


Support BallOfSpray by supporting the companies that support BallOfSpray

Barts ★ Connelly ★  DBSkis ★ Goode ★ Hobe Lake ★ HO Syndicate 

MasterCraft ★ Masterline ★ MOB ★ Performance Ski and Surf ★ Reflex ★ Radar ★ Stokes

ballsohardThan_Boganaupatkinglpskierbbrannan19skierRazorRoss3FWinter
«1

Comments

  • HortonHorton Posts: 27,160 Administrator
    edited August 11
    A few addition comments
    • There is no proof about what did or did not happen in Spain last summer but by all accounts the same event this year appears to be beyond reproach.
    • Any conclusions drawn from this data should not be a criticism of the skiers. The point of looking at this data is to highlight any events that are not at the same standard as the rest of the world.
    • It is my understanding there is a shift of culture and a desire to tighten up the specs at many of the European venues.
    • If anyone wants to look at my source data all they have to do is ask.

    Support BallOfSpray by supporting the companies that support BallOfSpray

    Barts ★ Connelly ★  DBSkis ★ Goode ★ Hobe Lake ★ HO Syndicate 

    MasterCraft ★ Masterline ★ MOB ★ Performance Ski and Surf ★ Reflex ★ Radar ★ Stokes

  • HortonHorton Posts: 27,160 Administrator
    edited August 11
    Last thing.... I struggled to find a best way to show this data. If you think you are confused the Bold bullets in the original post should help. Otherwise just ask.

    Support BallOfSpray by supporting the companies that support BallOfSpray

    Barts ★ Connelly ★  DBSkis ★ Goode ★ Hobe Lake ★ HO Syndicate 

    MasterCraft ★ Masterline ★ MOB ★ Performance Ski and Surf ★ Reflex ★ Radar ★ Stokes

  • HortonHorton Posts: 27,160 Administrator
    At least temporarily this post is in a section of the web site where only USAWS or members of other federations can comment.

    Support BallOfSpray by supporting the companies that support BallOfSpray

    Barts ★ Connelly ★  DBSkis ★ Goode ★ Hobe Lake ★ HO Syndicate 

    MasterCraft ★ Masterline ★ MOB ★ Performance Ski and Surf ★ Reflex ★ Radar ★ Stokes

  • LuzzLuzz Posts: 273 Open or 55K Rated Skier
    edited August 12
    Interesting way of thinking about this data. Can you retrieve the number of sites and tournaments per country this data is based on? Probably a ton of extra work, but should be able to obtain it from tournament codes.
    Organizer of the San Gervasio Pro Am (recap videos from 2018, 2017, 2016, 2015, and 2014)
    Co-organizer of the Jolly Clinics
    See what I am up to on Twitter and Instagram.
  • bishop8950bishop8950 Posts: 1,103 Open or 55K Rated Skier
    Interesting. What if you normalize the number of scores to a fixed number of events? In other worlds the gray bar is the same for all countries and the score bars adjust proportionately.
    ScarletArrow
  • HortonHorton Posts: 27,160 Administrator
    @bishop8950 I am can't figure out how to graph the data the way you are saying so here is the source data for anyone who wants to play with it. There are 1543 rounds shown. If you want to include rounds where 68 balls was not achieved the total is 1717.
      Number of rounds 68 balls or more 70 balls or more 72 balls or more
    AUT 16 7 2 0
    AUS 153 29 2 0
    GBR 70 31 4 1
    ESP 75 54 13 5
    ITA 220 106 27 4
    FRA 234 149 47 15
    USA 775 414 70 13

    Support BallOfSpray by supporting the companies that support BallOfSpray

    Barts ★ Connelly ★  DBSkis ★ Goode ★ Hobe Lake ★ HO Syndicate 

    MasterCraft ★ Masterline ★ MOB ★ Performance Ski and Surf ★ Reflex ★ Radar ★ Stokes

  • Than_BoganThan_Bogan Posts: 6,347 Mega Baller
    edited August 12
    @bishop8950 I actually think @Horton has already done what you've asked for. I want to loudly applaud him presenting this data, but I agree with his self-assessment that he hasn't quite figured out the best way to present it. The biggest problem is that everything is a percentage, but in a fundamentally different way! On the grey bars, it's a percentage of all tournament rounds. For each of the other bars, it's a percentage of success out of tournament rounds in that country.

    When the denominators aren't the same, percentages don't relate to each other in any useful way.

    To me, the grey bars are almost meaningless, except to suggest statistical significance.
    And even that is indirect, because pure quantity is all you need for that: 200 rounds in one country is statistically significant even if there were to exist another country with a million rounds. And 2 rounds would not be statistically significant, even if there had only been a total of 4 rounds and thus the 2 represented 50% of the total.

    What your eye should be drawn to is the unusual height of the green bars, which indicate the percentage of running -41. These are unusual in four countries: In Australia and Austria, they are outlying on the low side (both 0 in fact). In Spain and France, they are outlying on the high side -- more than triple the percentage of completion of -41 than in the US, Germany, and Italy.

    As always, all sorts of explanations can be possible. For the low ones, Austria is not statistically significant, because at the US rate, the expected number would be about 0.3, making 0 well within the realm of probable outcomes. And Australia is probably heavily skewed by the extreme challenge of the Yarra River.

    For the high ones, I do not have ready explanations.
    Nathaniel Bogan -- GUT Padawan
    ski6jonesjayski
  • Than_BoganThan_Bogan Posts: 6,347 Mega Baller
    Sidenote about statistical significance: When you are taking about improbable events, you need more data points than usual in order to make the type of conclusion one usually wants to make. Suppose after 100 trials, something has happened 48 times. It's extremely likely that the thing I've just sampled is in the range of 40% to 56% likely. But suppose after 100 trials, it hasn't happened at all. In this case, it's not crazy to think it might be 1% likely, but it also might be completely impossible. I thus don't have any insight into whether it is actually 1% or actually 0.00000001%. If I want to be sure I'm within say a factor of 2 of the real probability, I need to keep gathering samples until I have quite a few occurrences. And if the real probability actually is 0, then I can never use random sampling to get with a factor of 2 of that, and instead will eventually have to give up and say something like "it's 95% likely to be less than 0.001%."
    Nathaniel Bogan -- GUT Padawan
    andjules
  • HortonHorton Posts: 27,160 Administrator
    @Luzz Here is sanctions per country. Is this what you are asking for?

    AUS 17
    AUT 2
    cAN 12
    chi 3
    Col 3
    cze 2
    DOM 4
    ESP 8
    FRA 34
    GBR 13
    GRE 1
    ITA 42
    jpn 1
    MEX 6
    nor 1
    PER 3
    pol 1
    ukr 1
    USA 158

    Support BallOfSpray by supporting the companies that support BallOfSpray

    Barts ★ Connelly ★  DBSkis ★ Goode ★ Hobe Lake ★ HO Syndicate 

    MasterCraft ★ Masterline ★ MOB ★ Performance Ski and Surf ★ Reflex ★ Radar ★ Stokes

  • bishop8950bishop8950 Posts: 1,103 Open or 55K Rated Skier
    @Than_Bogan I would not dare to challenge your nerd skills when it comes to this sort of thing. But the way I was looking at this:

    ESP: 75 rounds, with 5 over 72
    FRA: 234 rounds, with 15 over 72
    USA: 775 rounds, with 13 over 72

    So per round, this says these skiers are ~4x more likely to ski 72 or higher in Spain or France than in the USA (without taking into account boats, drivers, sites, events, etc).
  • LuzzLuzz Posts: 273 Open or 55K Rated Skier
    @Horton kind of. I'd be interested in knowing the same event over the years. E.g., 18ITA001 and 19ITA001 would all fall within "San Gervasio Pro Am". As I said, lots of work, so no problem if it's unfeasible
    Organizer of the San Gervasio Pro Am (recap videos from 2018, 2017, 2016, 2015, and 2014)
    Co-organizer of the Jolly Clinics
    See what I am up to on Twitter and Instagram.
  • Than_BoganThan_Bogan Posts: 6,347 Mega Baller
    edited August 12
    @bishop8950 I agree. And the green bars show exactly that. The only slight caveat is that I'd say Spain is "borderline" in terms of statistical significance, because the expected number, at the U.S. rate, would be 1.2. So if 4 random people just happened to totally kick ass one time each, we could arrive at 5. That's a little improbable, which is why I call it "borderline." I would consider France to be statistically significant, because having 11 "random amazing performances" beyond the expected value is not nearly as plausible.
    Nathaniel Bogan -- GUT Padawan
    bishop8950
  • HortonHorton Posts: 27,160 Administrator
    @Than_Bogan @bishop8950 @Luzz

    There are really all kinds of problems with this data because it is a small data set and because of other unknown factors. This data is an indicator of a problem but not statistical proof of anything specific. My suspicion is that standards are more variable in parts of Europe.

    One of the top pros in the world told me that he does not see it as a country issue but a site issue. One site might be dead on the rules and the next site 10 miles away is playing fast and loose with everything.

    There are all kinds of downsides to the events with inflated scores. The World standings lists become meaningless. All scores become less meaningful because we do not know which are real and which are fake. Sites that do put on legit events are looked at with the same suspicions as those who do not. When skiers do achieve something special at a legit event it is undermined by fake scores. Freaking Standards Matter

    From my office here in California I am trying exert pressure on the European federations to just pay attention.

    Support BallOfSpray by supporting the companies that support BallOfSpray

    Barts ★ Connelly ★  DBSkis ★ Goode ★ Hobe Lake ★ HO Syndicate 

    MasterCraft ★ Masterline ★ MOB ★ Performance Ski and Surf ★ Reflex ★ Radar ★ Stokes

    Than_BoganGus
  • HortonHorton Posts: 27,160 Administrator
    edited August 12
    @luzz here is the raw version of what I think you are asking for.
    42 events 220 Rounds 68 balls or more 70 balls or more 72 balls or more
    19ITA013 2 1 0 0
    19ITA009 1 0 0 0
    19ITA001 27 18 4 0
    18ITA016 1 0 0 0
    18ITA015 3 1 1 0
    18ITA014 3 3 1 0
    18ITA013 3 3 0 0
    18ITA011 3 2 1 0
    18ITA001 27 15 2 0
    17ITA013 4 1 0 0
    17ITA012 2 0 0 0
    17ITA011 2 2 0 0
    17ITA010 4 3 1 0
    17ITA008 3 1 1 0
    17ITA007 3 0 0 0
    17ITA006 2 2 0 0
    17ITA003 2 1 0 0
    17ITA002 2 2 1 0
    17ITA001 21 11 3 0
    16ITA020 2 0 0 0
    16ITA016 3 2 0 0
    16ITA013 2 0 0 0
    16ITA009 6 4 2 0
    16ITA008 16 10 5 2
    16ITA007 14 11 3 2
    16ITA006 4 2 0 0
    16ITA004 2 0 0 0
    16ITA003 3 0 0 0
    15ITA018 2 0 0 0
    15ITA014 4 1 0 0
    15ITA013 4 1 0 0
    15ITA011 3 0 0 0
    15ITA009 3 0 0 0
    15ITA008 3 2 1 0
    15ITA005 10 1 0 0
    15ITA002 5 0 0 0
    14ITA013 2 1 0 0
    14ITA012 2 0 0 0
    14ITA009 6 3 0 0
    14ITA006 6 1 1 0
    14ITA003 1 1 0 0
    14EAC001 2 0 0 0

    Support BallOfSpray by supporting the companies that support BallOfSpray

    Barts ★ Connelly ★  DBSkis ★ Goode ★ Hobe Lake ★ HO Syndicate 

    MasterCraft ★ Masterline ★ MOB ★ Performance Ski and Surf ★ Reflex ★ Radar ★ Stokes

  • LuzzLuzz Posts: 273 Open or 55K Rated Skier
    @Horton pretty much. So, for example:
    San Gervasio Pro Am

    105/220 rounds
    57/106 68+buoys
    13/27 70+buoys
    2/4 72+buoys

    Pretty much 50% of the rounds are accounted by one pro event (the only one in that country). This data can initially make people raise eyebrows. However, I believe it highlights the simple fact that pro events at a good site are likely to generate a lot of high scores.

    I'm pretty sure that if one was to see the data like this for the US, similar results would show up, namely all pro events at good sites would account for possibly 50% of the scores.

    I believe the efforts of understanding why some tournaments in the past produced an unusual number of high scores are justified. At the same time, I believe that providing standards and making sure those standards are followed is the issue here, and a point that can be made without fishing for data interpretation.

    Truth to be told, the E&A region assigns judges for RC events. While this may not be enough of a scrutiny and control system, it is already a start to make sure that no intentional rule bending occurs. I know this is not the case in the PANAM region, not sure about A&A. The issue is that not all tournaments are RC, and if you ski a score in any of those categories (including [email protected]), as long as it is not a record, the regional or world TC doesn't have to review it.

    One of the issue with this is that our sport doesn't have professional judges, something that if it were in existence would drastically diminish conversations about scrutiny. By the same token, we are a super small sport, so the whole volunteer judging system we have in place is somewhat necessary to run events.

    In short, more scrutiny is needed, pro tournaments should all be RC, and thank you to the judges and drivers that invest their own time to help the sport :smile:

    ( @Horton I know you are not on the other side of the points I am making. These are just my thoughts after thinking about this. )
    Organizer of the San Gervasio Pro Am (recap videos from 2018, 2017, 2016, 2015, and 2014)
    Co-organizer of the Jolly Clinics
    See what I am up to on Twitter and Instagram.
    bishop8950ALPJrThan_Bogan
  • HortonHorton Posts: 27,160 Administrator
    edited August 13
    @luzz I hate that the above graph data infers that all sites in Europe or in any one country may not be within bounds but I am not sure how else to make my point.

    EVERYTHING I have heard about the San Gervasio Pro Am is that the event is run by the rules and is a model tournament.

    I also have to say that it appears that the Seseña event was by the book this year. It was the Seseña event in 2018 that brought the issue to a head.

    The question I want answered is why some guy with a website in California has to point fingers before the people in charge of international start asking questions?

    For the integrity of the sport and for the elite skiers lets all ski to the same standards.

    Support BallOfSpray by supporting the companies that support BallOfSpray

    Barts ★ Connelly ★  DBSkis ★ Goode ★ Hobe Lake ★ HO Syndicate 

    MasterCraft ★ Masterline ★ MOB ★ Performance Ski and Surf ★ Reflex ★ Radar ★ Stokes

    andjules
  • LuzzLuzz Posts: 273 Open or 55K Rated Skier
    @Horton . I just talked about that tournament because you pulled up that data. I know my event is legit.

    All I am trying to say is that there is more scrutiny in Europe than in the US, and more questions are asked than you or I or people we both know are aware of.

    That doesn't mean that questionable scores can pop up anywhere, Europe, US, or elsewhere.

    One of the things that could be considered is not to officialize any non-RC score at 43off until review. That is the case right now for 11k+ scores at non-RC events. However, that is also contributing to the lower number of scores in tricks, lower tournaments, etc.

    Again, the point is well-taken and necessary, but I am not sure that an analysis like the one you are doing is going to strengthen it.
    Organizer of the San Gervasio Pro Am (recap videos from 2018, 2017, 2016, 2015, and 2014)
    Co-organizer of the Jolly Clinics
    See what I am up to on Twitter and Instagram.
    bigtex2011
  • HortonHorton Posts: 27,160 Administrator
    @Luzz my analysis might be very flawed but if I get people talking about the subject and looking for solutions then my job is done.

    Support BallOfSpray by supporting the companies that support BallOfSpray

    Barts ★ Connelly ★  DBSkis ★ Goode ★ Hobe Lake ★ HO Syndicate 

    MasterCraft ★ Masterline ★ MOB ★ Performance Ski and Surf ★ Reflex ★ Radar ★ Stokes

    LuzzRoger
  • Than_BoganThan_Bogan Posts: 6,347 Mega Baller
    I think it goes too far to call your analysis "very flawed." I think about the harshest criticism that should be applied is "cursory" -- i.e. it hints at some things to look into but doesn't provide further details.

    Well, also I'm deducting points for displaying things on the same scale that are totally unrelated, but that is trivially fixed by just removing the grey bars. :)
    Nathaniel Bogan -- GUT Padawan
    andjules
  • mshaw200mshaw200 Posts: 75 Baller
    I would be curious to know how the scores in 2019 have been affected by the new MasterLine ropes.
    And my hair was perfect!
    bigtex2011
  • andjulesandjules Posts: 798 Crazy Baller
    edited August 15

    Thread summary, courtesy of @Than_Bogan (what @Horton is really trying to say?):

    What your eye should be drawn to is the unusual height of the green bars, which indicate the percentage of running -41… In Spain and France, they are outlying on the high side -- more than triple the percentage of completion of -41 than in the US, Germany, and Italy.
  • HortonHorton Posts: 27,160 Administrator
    edited August 15
    @HighAltitude
    Your golf comments would be true if we eliminated the world standings list. The current expectation is that all events are run to as close to the same specs as possible. Your comments make sense if buoy width, boat speeds and rope length we're not exactly specified universally / worldwide

    When some ski sites run fast and loose with the rules and inflate the scores the athletes who obtain those scores move up the standings list compared to athletes that only compete at sites that are exactly on spec. I have heard Elite skiers talk about the need to go to specific events so they can get ( illegitimate ) high scores to maintain their world standings. That is ridiculous and makes the sport look stupid.

    Support BallOfSpray by supporting the companies that support BallOfSpray

    Barts ★ Connelly ★  DBSkis ★ Goode ★ Hobe Lake ★ HO Syndicate 

    MasterCraft ★ Masterline ★ MOB ★ Performance Ski and Surf ★ Reflex ★ Radar ★ Stokes

  • HortonHorton Posts: 27,160 Administrator
    @HighAltitude there is a link in my original post to all of the source data. If you're interested knock around in there and see if you find any patterns.

    Support BallOfSpray by supporting the companies that support BallOfSpray

    Barts ★ Connelly ★  DBSkis ★ Goode ★ Hobe Lake ★ HO Syndicate 

    MasterCraft ★ Masterline ★ MOB ★ Performance Ski and Surf ★ Reflex ★ Radar ★ Stokes

  • HortonHorton Posts: 27,160 Administrator
    edited August 15
    @Than_Bogan I don't really think that my analysis is flawed but I am clearly not a statistician. I would not fall out of my chair if somebody reviewed my work and found some error. I double-checked it more than once and I'm pretty sure my methods are correct but I did not get a second set of eyes to review.

    What I am confident of is that no matter how you crunch the data you would come to a very similar conclusion.

    The best thing for the sport is if more people took a hard look at the raw data.

    Support BallOfSpray by supporting the companies that support BallOfSpray

    Barts ★ Connelly ★  DBSkis ★ Goode ★ Hobe Lake ★ HO Syndicate 

    MasterCraft ★ Masterline ★ MOB ★ Performance Ski and Surf ★ Reflex ★ Radar ★ Stokes

  • rfarfa Posts: 242 Baller
    Now that I have learned stats (and pole jumping) from @Than, I feel I can comment here...LOL.
    I should disclose that I was never a fan the original tread about the "unusual scores" in Spain. I had a similar "bad taste" around the same time, about another tread on "unusual" high scores at an amateur tournament site in NJ. My discomfort with those was the innuendo that some type of "fraud" was responsible for the high scores. I have no idea if any "fraud" was present (narrow course, slow boat, bad boat path, lenient judges, whatever), so personally, I would not bring it up without some proof. Conversely, I agree with @Horton that "questions should be asked" and answered.
    My comment on this analyses is not whether it is perfect or flawed. Looks like a good compilation of available data. (as a less nerdy R&D engineer now that I am old, I still love "data"). My view is that this does not increase the potential justification to ask whatever questions might be legit about the Spain tournament. It was a one tournament situation. I suspect that if no answers were given to date, this will not provide any further impetus. @Horton says "it appears that the Seseña event was by the book this year". Does "by the book" mean that no unexpected 41's were run this year? Or do we have some new relevant information about how it was run last year?
    What if one looked at the graph and asked about the lack of 41's in AUS and AUT? Wide course, fast boat, "too good" boat path, over zealous judges, whatever).
    Bottom line, great information compiled and presented but, in my opinion, not changing the need (or lack thereof) for questions about Spain 2018.
    Rui Afonso
  • HortonHorton Posts: 27,160 Administrator
    edited August 15
    @rfa ideally I would have left Spain out of this data set but as soon as I start including excluding parts of the data I think it becomes less valid. Furthermore grouping the data by country is also not ideal. The suggestion that one particular nationality is honest and another is not is ridiculous. With a data set so small there has to be silos and unfortunately the logical silo is by country.

    I can't say this enough. My goal is to make people at the high end of the sport look at the data and see the anomalies.

    Support BallOfSpray by supporting the companies that support BallOfSpray

    Barts ★ Connelly ★  DBSkis ★ Goode ★ Hobe Lake ★ HO Syndicate 

    MasterCraft ★ Masterline ★ MOB ★ Performance Ski and Surf ★ Reflex ★ Radar ★ Stokes

  • HighAltitudeHighAltitude Posts: 145 Baller
    @Horton I'll try and look things over when I have time, but just looking at what you have here, I'd say there isn't much more to analyze. Yes, we could show there is a statistically significant difference between sites and we might be able to use some sort of confidence interval to show how unlikely something is, but....I think it will become clear that some sites ski better than others. The real question is why. Are some playing "loose and fast" with the rules or is it consistently better conditions?

    I think one boat manufacturer currently has more world records behind it than the others (I haven't seen those ads lately, so I'm not sure if its true anymore). Does that mean that boat is better? Could that be the difference? I don't believe that but it goes to show how many variables there still are despite an attempt to minimize them.

    BTW... pro golf has world rankings as well.
  • bigtex2011bigtex2011 Posts: 454 Crazy Baller
    @Horton I'm intrigued by the new hard hitting Horton. Are you going to focus your new fact finding hard hitting expos'e to equipment that fails (broken skis, ropes that are too long or break, bindings flaws) How many and what kind. I'd love to see your stats on AWSA membership or what happened to TRex was it an asteroid or global warming.

    Back in the DAY or Lapoint and Mapple I feel confident some sites were probably more skier friendly that others. In the end, it all works out.



  • HortonHorton Posts: 27,160 Administrator
    edited August 15
    @HighAltitude Boat manufacturer is not a factor. The brand with the most top skiers on salary has the most records. I do not remember the last time a world record was set behind a brand of boat who was not paying that skier.

    Golf has ranking but the criteria is totally different*

    ( * assumption )

    Support BallOfSpray by supporting the companies that support BallOfSpray

    Barts ★ Connelly ★  DBSkis ★ Goode ★ Hobe Lake ★ HO Syndicate 

    MasterCraft ★ Masterline ★ MOB ★ Performance Ski and Surf ★ Reflex ★ Radar ★ Stokes

Sign In or Register to comment.