Results 1 to 1 of 1

Thread: SRL Judges Panel

  1. #1
    Join Date
    Jul 2007
    Location
    Right now? Chair.
    Posts
    8,488
    Mentioned
    3 Post(s)
    Quoted
    12 Post(s)

    Default SRL Judges Panel

    Too long, Didn't read: A plan to ensure the prosperity and continuity of SRL Competitions.

    As many of you will know, I'm doing a management course. And in that management course, one of the things that was said was the following "People will be happier to agree to an idea if they know exactly what that idea is." - So I'll be letting you know what is on my mind.

    Background
    SRL used to have scripting competitions. This was started by Sb100, he ran a few competitions, and then several people tried to pick it back up but never quite at the level Starblaster did. The day he left we lost a great person in our community.

    The Aim
    My aim is to have these competitions restarted. And not have them falter again, whenever I happen to leave. "True leadership raises other leaders to follow your steps."

    The Why
    Competitions are great for honing your skills, learning from other people and great to get scripts out. This obviously also increases activity, as a Domino. Scripter -> Script -> Testers -> Updates -> New users -> New scripts.

    The How
    This is where it becomes interesting.


    General Competitions

    Background
    This was an idea that I saw a long time ago. Then as SSRL I felt I had authority enough to run one and started it up. When I got to admin the judging was finished. This obviously wasn't efficient.

    The Aim
    To give JMs yet another reason to script. Learn. Improve. Strive.


    The Prizes
    SRL Membership. Competition Winner Cup.


    The How
    This is where it gets interesting.

    SRL Member's Competitions

    Background
    Back when it was Starblaster running this, these competitions would be open to all. This time around, I'm sorry to say it, but devs might be excluded - I still have to find a propper way around for this.

    The Aim
    To get SRL Members scripting at their best. This will make it easier to see who has Dev, Master or Scripter potential, allowing people to grow through the ranks more easily.

    The Why
    The amount of learning by itself is reason enough for the why.

    The Prizes
    Competition Winner Cup.


    The How
    This is where things really do get interesting.


    The How - This is the really interesting part

    The Judging
    All Judging will be done by SRL Members +
    Judges will grade harder competitions depending on how they performed the previous time.

    The Judges Panel
    This is where innovation really comes in. I believe this will be the key to success. There will be a judges panel. What will this mean?
    The panel will have a hierarchy to it. From bottom to top:
    1. General Competition Judges - These will be the people judging General Competitions. As JMs are usually less-able scripters than SRL Members, the newer members should be just fine to judge these. This will be a rank as much about learning to judge as for those who do not feel confident enough to judge fellow members. There will of course be some highly-able scripters also judging these competitions, to ensure fairness and that nothing is missed out.

    2. SRL Members+ Judges - These will be the people judging SRL Member competitions. This will require a higher level of skill as well as amount of work that is put into the judging, to notice the subtletie's each scripter is bound to have. These competitions will be judged by members of all levels to ensure fairness.

    3. The Table - These are those who will be in charge. The Table will be resposible for setting up the competitions, dealing with the judges, assigning judges, promoting judges and so on. Members of the table will be those who ensure the competitions are kept running. This will NOT be restricted by forums rank. Show you deserve it and you'll get it.



    How the Ranks will work
    Top 20% go up, Bottom 10% go down, remaining 70% stay as they are.
    Meaning:
    • Judges who did not perform well will go down a rank or leave the the panel completly.
    • Judges who performed best will go up a rank.
    • Any new appliers will always be given a slot, wherever The Table believes they would be best placed.
    • NONE will be promoted directly to The Table.


    The 20% judges who put most work in (quantity&quality) get moved up to grading more complicated competitions, as they've shown themselves capable of doing such things.

    10% down is just to avoid people who just want to be part of the panel for the "coolness" of it

    Why?
    This ensures the best of the best make it to the top. As simple as that. Work hard for something and you'll get it. If only life was always this easy.

    What if I only want to do it once or twice?
    You are free to do this as many or as few times as you'd like. Although there is a system in place to prevent unfair judging, leaks or any sort of disruption you are more than welcome to judge as many times as you like - given that you were not the source of any trouble caused on purpose.


    The Marking System
    This is somewhat necessary to ensure fairness. How will it be done? The Panel will plan how they would go about creating a script, keeping a certain skill bar in mind. (Scripters Cup for JMs, Dev / Masters for SRL, MSI SL for MSI) Once having assessed the best methods to use, they will then consider alternatives. Once this is done they will assign a score to each, generating somewhat of a 'generic answer'. This will be done at the same time as the competitors are writting their scripts.
    [ If anyone is actually a teacher, some further insight into the usual way of marking exams would be appreciated ]


    Marking - First Steps
    Scripts will be posted in the appropriate places, with any refferences to authors removed. Judges will then post their assesments, without actually posting their grade. The grades will be submitted to the panel - This is to ensure no one loses their files, as has happened before. Once everyone has posted their comments, judges can choose to re-assess the grades they gave (and then send them again to the panel).

    Marking - Weighting
    There will always be harsh and lenient judges. This was overcome last time by weighting. Each Judge's score is multiplied by a weight over all the scripts, just for it not to be too drastic. This does not affect the overall effect on the judging as it will applied to all scripts. This is to overcome the cases where "One person may find all scripts great, while another will find them all to be crap." - The weight applied to each judge will be set according to somewhat of an average of the expectation of the script, by a member of The Table. One single member of the table will decide on weights for each judge in one of the competitions, to ensure equal standards.

    Wrapping up the competition
    This will be done by a member of The Table. Basically includes gathering the comments and grades, making a thread with the results and appropriate presentation and letting the SRL Staff know where cups are due, along with any cup suggestions they might have.



    Setting up the Judges Panel

    Applying
    Post saying you are interested. Feel free to also throw around any ideas. PM me, (or in the future - A Table Member), saying which competition(s) you find yourself fit to judge, why, why do you want to do this, your availability and any relevant experience.

    Choosing
    Anyone who applies will be given an opportunity to judge. You will be assigned to a competition at a whim, you can then feel free to work yourself up the ranks.

    Starting Competitions
    More on this will be within the Judges' forums. Things such as criteria to choose what is suitable or not for a scripting competition, the preparation of threads and rules, etc.

    The Rewards
    Other than the obvious prestige, self-satisfaction and community adoration? It is likely that some cups will be given around as seen fit.

    Charity
    Anyone who does not feel like doing this long-term, but rather just once or twice, is more than welcome to do so. All help is appreciated.


    Extra Information
    • The competitions will last no longer than a month each.
    • Judging will last no longer than 3 weeks.



    Now, lets see some comments and applications, as well as some enthusiasm and motivation. I'll be expecting a flooded inbox. I shall end the thread here.

    Lots o' Love,
    ~RM
    Last edited by BraK; 10-02-2012 at 03:30 PM.

    I & I know Zion. It is in the spirit, body and mind of every one of us
    RMouse(obj: TMSIObject): boolean;

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •