Multiple realm ratings

For Wizard Kings (and higher) to discuss the Realm Editor and the creation of realms.
User avatar
SpiteAndMalice
DEMIGOD
Posts: 6276
Joined: Sun Apr 06, 2014 10:45 pm
Location: UK
Contact:

Multiple realm ratings

Post by SpiteAndMalice » Tue Dec 15, 2015 6:43 pm

I think the trick is to get a balance between asking enough for enough ratings so as to get meaningful feedback for the designer (and other players), and not asking for so many ratings that the player doesn't bother to answer them seriously.

To which I'd suggest no more than 3 or 4 in total and maybe asking about areas such as these:
Writing
Story/Theme
Difficulty
Encounters
Design
Fun
Overall Rating

Can any think of better categories?

It shouldn't be all of the above, and some areas could be combined. (For instance Writing and Story, Writing and Encounters, or Encounters and Design).

The other option is to let the realm designer ask a question or 2 themselves and/or receive written feedback.
Image

The Guild of the Cephalopods is now recruiting - We embrace all.

- Chief Tentacle Arranger in The Guild of the Cephalopods.

User avatar
Mazy
Posts: 1789
Joined: Mon Oct 06, 2014 8:24 pm
Location: UK

Re: Multiple realm ratings

Post by Mazy » Wed Dec 16, 2015 1:45 am

Aye, Categories might be one way to go, It's no biggie ticking a few boxes or leaving a tiny bit of feedback after completing a realm really. I can see the logic behind having one simple overall realm star rating system but I guess it doesn't tell you much, i.e, was it a low rating because it was bad (what is bad?), uninteresting? or too difficult?

...to be completely honest I pretty much give every realm 5 stars because:

1) I know how long it takes to make a realm, it doesn't matter how it plays out, they deserve 5 stars for effort.
2) I know and like the people that create the realms, of course I'm going to rate them highly when being asked to judge them.
3) Some of the best realms were getting the lowest ratings so was duty bound to rate them up.

You could say those last couple of points are flaws in the system. Plus I don't see realms as either 'good' or 'bad'.

...maybe realms don't even need a rating mechanic? people create them, people play them then they move on to the next. When you've completed all the listed realms you're going to play the next released realm no matter what, it's a new map, some new battles and a new set of towns to find loot in (a good or bad rating isn't going to influence your decision). After all, moderators will be rating them simply 'playable' or 'unplayable', this is criteria enough.

Or how about, radically overhaul the realm scoring system to simply give players who complete the realm a star rating (1-5) based on their performance instead of an almost random numerical score (based on routes taken) then the average player performance rating within the realm can be the overall rating for the realm, rather than some subjective thoughts (a low rating would mean difficult/challenging, high rating would mean easy).
Last edited by Mazy on Wed Dec 16, 2015 1:56 am, edited 1 time in total.

User avatar
NoWorries
GOD
Posts: 5039
Joined: Fri Apr 18, 2014 4:17 am
Location: Baldrlux.com
Contact:

Re: Multiple realm ratings

Post by NoWorries » Wed Dec 16, 2015 1:55 am

Some objective bits of information might be more informative for players looking for a realm to play. For example:
  • 1) Completion Rate (i.e., % of players who start a realm who complete it at some point);
    2) Number of Encounters (serves as a proxy for RP content);
    3) Number of Wizard Lords (battles!);
    4) A tag (moderator set?) which confirms whether the Realm makes a nod to official game lore or not.
Like there was before, there could also be an Alchemist report which informs how many Citadels, etc. are in a Realm. Again, useful data for when a player is determining what kind of Realm they are selecting.

If we have all of that information, then a single rating might work just fine.
___________________________
  • One Theme to rule them all, One Theme to find them, One Theme to bring them all and in the Chaos bind them.
    NoWorries plays as Baldr, God of Wonder & Progress | Compete together with Baldr in the Order of Baldrlux

User avatar
SpiteAndMalice
DEMIGOD
Posts: 6276
Joined: Sun Apr 06, 2014 10:45 pm
Location: UK
Contact:

Re: Multiple realm ratings

Post by SpiteAndMalice » Wed Dec 16, 2015 7:23 am

Mazy wrote: ...to be completely honest I pretty much give every realm 5 stars because:

1) I know how long it takes to make a realm, it doesn't matter how it plays out, they deserve 5 stars for effort.
2) I know and like the people that create the realms, of course I'm going to rate them highly when being asked to judge them.
3) Some of the best realms were getting the lowest ratings so was duty bound to rate them up.

You could say those last couple of points are flaws in the system. Plus I don't see realms as either 'good' or 'bad'.
I don't think you'll be alone in doing this Mazy, any feedback system where people are asked to make value judgement will to an extent become a 'happy sheet'.

That's partly how multiple categories can help, it's not so much about comparing your scores with other designers, as comparing the categories against each other. If your 'encounter' scores are constantly lower than your 'writing' scores then you get some idea of which area of realm design to work on.
Mazy wrote: Or how about, radically overhaul the realm scoring system to simply give players who complete the realm a star rating (1-5) based on their performance instead of an almost random numerical score (based on routes taken) then the average player performance rating within the realm can be the overall rating for the realm, rather than some subjective thoughts (a low rating would mean difficult/challenging, high rating would mean easy).
This would be good: I'd say A-E - If doing this you either need to know the maximum possible score for a realm too if you're giving absolute grades (grades which persist no matter how other players do) or you need a formula to compare players grades against each other (at which point grades can shift as more players complete a realm).
Image

The Guild of the Cephalopods is now recruiting - We embrace all.

- Chief Tentacle Arranger in The Guild of the Cephalopods.

User avatar
SpiteAndMalice
DEMIGOD
Posts: 6276
Joined: Sun Apr 06, 2014 10:45 pm
Location: UK
Contact:

Re: Multiple realm ratings

Post by SpiteAndMalice » Wed Dec 16, 2015 7:25 am

NoWorries wrote:Some objective bits of information might be more informative for players looking for a realm to play. For example:
  • 1) Completion Rate (i.e., % of players who start a realm who complete it at some point);
    2) Number of Encounters (serves as a proxy for RP content);
    3) Number of Wizard Lords (battles!);
    4) A tag (moderator set?) which confirms whether the Realm makes a nod to official game lore or not.
Like there was before, there could also be an Alchemist report which informs how many Citadels, etc. are in a Realm. Again, useful data for when a player is determining what kind of Realm they are selecting.
These would help regardless.

2&3 don't need to be given as absolute numbers, it could be just be along the lines of 'High' 'Medium' or 'Low'

EDIT - In fact 1 could also work this way; starting with 'Unknown' for new realms and then moving to 'Easy' 'Average' 'Difficult.'
Image

The Guild of the Cephalopods is now recruiting - We embrace all.

- Chief Tentacle Arranger in The Guild of the Cephalopods.

User avatar
NoWorries
GOD
Posts: 5039
Joined: Fri Apr 18, 2014 4:17 am
Location: Baldrlux.com
Contact:

Re: Multiple realm ratings

Post by NoWorries » Wed Dec 16, 2015 5:15 pm

SpiteAndMalice wrote:EDIT - In fact 1 could also work this way; starting with 'Unknown' for new realms and then moving to 'Easy' 'Average' 'Difficult.'
A low % completion Realm with a low star rating may correlate with a grind Realm which is uninteresting. While a low % completion Realm with a high star rating may correlate with a very challenging but rewarding Realm. So the raw number should perhaps remain transparent so players can apply their own meaning.
___________________________
  • One Theme to rule them all, One Theme to find them, One Theme to bring them all and in the Chaos bind them.
    NoWorries plays as Baldr, God of Wonder & Progress | Compete together with Baldr in the Order of Baldrlux

anjovi
Posts: 1515
Joined: Thu Apr 10, 2014 5:34 am

Re: Multiple realm ratings

Post by anjovi » Thu Dec 17, 2015 1:40 am

I could see multiple categories to rank stars on to be reasonable.

just have a few sub categories under the overall score to give people a good representation of were the realm succeeds/lacks.

I suppose my ideal would be

Overall: the overall score
Writing: encounters/descriptions
Gameplay: More of an intuitive thing. Just thinking about general pacing of the realm and how balanced the villages/lord battles were

So i guess just 2 subcategories, An optional written review for the creator to read would actually be great! As I've still got no idea what people think of the realms i've made...Is the writing bad? is the pacing off? was that mountain section in my aestril realm horrible? it'd be cool if people could leave a short little blurb similar to how mods can when reviewing your realms.

Granthor
Posts: 377
Joined: Sun Dec 14, 2014 11:42 pm

Re: Multiple realm ratings

Post by Granthor » Tue Dec 22, 2015 11:54 am

+1 to this idea, can't see any major downside really - if the idea is to give feedback to the designer, the more that we can tell them the better.

User avatar
NoWorries
GOD
Posts: 5039
Joined: Fri Apr 18, 2014 4:17 am
Location: Baldrlux.com
Contact:

Re: Multiple realm ratings

Post by NoWorries » Tue Dec 22, 2015 3:59 pm

Granthor wrote:+1 to this idea, can't see any major downside really - if the idea is to give feedback to the designer, the more that we can tell them the better.
The primary purpose of ratings is to help players determine which realms they would enjoy playing. So any rating fields should be designed with that in mind. Feedback to realm designers is a secondary purpose.

Also, while I'm not against more, each ratings field added risks decreasing participation in the rating system altogether. So we should be thoughtful in anything added. Hence my suggestion to add objective data sets where practicable that do not necessarily require user inputs.
___________________________
  • One Theme to rule them all, One Theme to find them, One Theme to bring them all and in the Chaos bind them.
    NoWorries plays as Baldr, God of Wonder & Progress | Compete together with Baldr in the Order of Baldrlux

User avatar
SpiteAndMalice
DEMIGOD
Posts: 6276
Joined: Sun Apr 06, 2014 10:45 pm
Location: UK
Contact:

Re: Multiple realm ratings

Post by SpiteAndMalice » Tue Dec 22, 2015 5:37 pm

NoWorries wrote:
Granthor wrote:+1 to this idea, can't see any major downside really - if the idea is to give feedback to the designer, the more that we can tell them the better.
The primary purpose of ratings is to help players determine which realms they would enjoy playing. So any rating fields should be designed with that in mind. Feedback to realm designers is a secondary purpose.
Has this primary purpose been stated somewhere?

Either way written feedback to realm designers lies outside of the scope of allowing players to chose the best realm to play. Written feedback actually serves a separate purpose of allowing realm designers to get feedback on their realms. This will in turn lead to better realms being created and thereby lead to a better player experience.

NoWorries wrote:Also, while I'm not against more, each ratings field added risks decreasing participation in the rating system altogether. So we should be thoughtful in anything added. Hence my suggestion to add objective data sets where practicable that do not necessarily require user inputs.
With 2000 play-throughs of a realm the percentage of players who participate in leaving feedback is less important than the usefulness of the feedback given. Quality over quantity. Objective data sets would be a useful part of that, but they're not the complete answer.
Image

The Guild of the Cephalopods is now recruiting - We embrace all.

- Chief Tentacle Arranger in The Guild of the Cephalopods.

Locked