Dear friends,
Many of you have expressed concerns over usability of the
current rating system. Through your feedback, you identified and explained
various problems with ratings and helped us better understand your needs and
your goals.
Many of you have spoken in favor of radical reforms to
address the issues of revenge, intimidation and manipulations that
unfortunately have become part of the process when it comes to giving and
receiving ratings.
To address these concerns, we have started the process of revamping
the system of ratings, and many aspects associated with it. We recognize that
many of the issues with ratings as they are today are inheritably built in the
current environment itself, and we have made it our goal to bring in freedom of expression,
professionalism, and unbiased, constructive and meaningful feedback -
everything that the ratings were meant, but practically failed, to be.
The new system we would like to introduce is designed to correct the flaws of
the existing rating system in the very core of its design. It is based on the
best suggestions we have received from our members, and we are enormously
grateful to everyone who spoke out and proposed constructive, well-thought out
ideas. Our goal is to give you the new system before the middle of April 2005.
While designing the changes, our priority was to keep the
best parts of the system, while adjusting aspects
that were hindering it. The new system preserves social interactivity, ease of
use, rewards, and many other aspects you are accustomed to.
The new approach is targeted to address many of the negative
aspects of the ratings: fear of retribution, revenge ratings, hate-mail,
offensive or meaningless comments, over-rating, biased ratings and any sort of
manipulations. The new system breaks the old habit of expecting nothing but
tens, and brings a new meaning into ratings.
Many photographers are striving to receive honest and
constructive feedback - they realize that it is the
only way their work can ever improve. With the new system, constructive
criticism is in the foundation of all ratings. Many of the very good
photographers are accustomed to tens - the wonderful tens and nines, although great for
one's ego, do not help deal with the problems the photos may still have. Other
members who may be able to provide helpful comments feel intimidated as attacks
have unfortunately become part of the everyday reality for anyone who has been
trying to be objective.
Meaningful and constructive feedback - the foundation of
learning and self-improvement - will be possible under the new system.
Rewarding excellence in photography and expression of appreciation of true
talent will be possible under the new system. Under the new system,
- Offensive, rude, or meaningless feedback will no longer lurk
under covers of the first rater anonymity - every photographer will be able to
simply get rid of such feedback.
- No one will need to be an active rater in order to get more
ratings, and there will be no need to rate high in order to get rated highly.
- Everyone will have an equal opportunity to get fair ratings,
regardless of whether the name is on someone's favorites list or a member just
joined the site, regardless of whether someone is "popular" in the community or
not.
The key areas we concentrated on to achieve these goals are:
- increasing participation in the process of ratings without
resorting to force anyone to rate, and
- using the two-way anonymous numeric rating system to encourage
unbiased and fair feedback.
We apologize that the list below is rather lengthy. We believe
that when changes of this magnitude are introduced, it is essential to provide
every possible detail about all of them.
Comprehensive List of Upcoming Changes
I New Benefits for Being an Active Rater
1. Introduction of the new home page section called "Spotlight." Spotlight
will be showcasing a photographer, displaying his/her name, a few lines from the
profile, and a single highest rated photo. Having a photographer's name in a
very prestigious section of the front page offers great visibility, which
increases the likelihood of sales.
There will be three roads to get in the spotlight (notice
that none of these includes being a highly rated member):
- Selling at least one photo within the last three weeks.
- Being a rater who scored the highest number of rating
points within the period of the last 24 hours.
- Having an image with the highest number of views in the
last 24 hours.
The spotlight will get updated multiple times a day, providing
an opportunity for active raters to be showcased on the front page.
2. Consistently active raters will continue to get free
storage space extensions under the rating points reward program. The assignment
of rating points will be changed as follows: 1 point will be awarded for the
comment-only feedback and 3 points for a numeric rating.
3. Two-way anonymous rating process will be implemented to make retribution
impossible (see "Giving Ratings" section below). Retribution has been a long lived problem,
which will finally be addressed with this change.
4. To make the rating process even easier, we will be introducing a new way to
rate a randomly chosen photo right on the home page. There will also be an
ability to change preferences in the Account Settings to enable or disable this behavior and
choose how to organize the sections of the home page.
II Giving Ratings
1. The rating process will be restructured into
two separate interfaces: one for anonymous numeric ratings with required
comments (see "Rating Different Aspects of Photos" section for details on when comments will be optional),
and another for identifiable comment-only feedback. The ability to provide two-way anonymous
ratings will be introduced to avoid biased ratings, ratings inflation, ratings
retribution, and revenge ratings.
- Anonymous ratings
To enter the anonymous ratings
area, a member would go to the
Rate Photos page
(or choose from the four thumbnails on the home page in the section named "Rate
Photos"). All images shown on this page will be completely random with the
default ShutterPoint watermark overriding any custom watermarks. By giving ratings on this page, a
rater will not know who gets rated, and the photographer will not know who is leaving the
rating (hence the term "two-way anonymity"). This will allow members to be fair,
impartial, and honest with their ratings without the fear of retribution or
hate mail, and therefore, constructive criticism will be easier to convey.
- Identifiable, comment-only feedback
To leave any identifiable
comment-only feedback, a member will be able to select a photo by browsing
through the site (same as now) and pick any photo to see it full size. Any
comments left by employing this method will be displayed along with name of the
person who left them (same as now). This will allow photographers to remain
involved in the social aspect of the rating system. It will also be a way for a
rater and a person whose image gets rated to start a dialog if they choose to
do so.
2. The limited ability to remove feedback will be introduced to address a
possible issue with meaningless or offensive feedback that can come as the
result of anonymity. Each photographer will have a "remove" link for
each comment or rating so that this sort of feedback can be deleted. In order
to prevent an unreasonable use of this feature, this functionality will be
limited. The number of removals that any photographer can employ will be set as
10% of the number of all anonymous numeric ratings that a member has received.
For example, if someone received the total of 67 anonymous numeric ratings, a
10% limit will allow to have 7 (6.7 rounded) ratings or comments removed from
any number of photos. Once the limit is used up; there will be no way to remove
ratings other than getting more ratings first. This restriction will limit photographers
from being able to "clean up" criticism excessively.
When any rating/comment is removed as offensive or meaningless, the rater who
originally left this rating/comment will instantly lose 50 rating reward
points. This measure is designed to encourage raters to try to provide
meaningful constructive criticism as opposed to simply rubber-stamping images they
do not like as bad. Any person with an alarming number of ratings/comments
being removed by photographers may temporarily lose the ability to rate.
3. Both "No Revenge" and "No Bias" rules will be abolished,
since anonymity of numeric ratings prevents bias and revenge on its own.
III Rating Different Aspects of Photos
The ability for raters to choose which way they want to rate
each photo will be introduced. A traditional single overall numeric rating
method will remain as is. In addition, a more descriptive rating method
allowing evaluating each aspect of the photo individually will be made
available. When this new method is used, comments will be optional. Below is a
list of photo aspects that will become available for evaluation:
- Composition
- Creativity / Originality
- Depth of Field / Focus
- Color or Tonal Range
- Noise or Grain
- Exposure / Lighting
- Tells a Story
- Dramatic Effect
If the rater chooses to rate by aspects, the system will
automatically convert the ratings into a single overall score on a 1-10 rating
scale. Also, the evaluation of individual photo aspects will be displayed as
comments accompanying the score.
IV Choosing to Receive or Deny Feedback
Each photographer will have these three choices to accept or deny ratings for
each submitted photo:
- Both numeric rating and comments are accepted
- Only comments are accepted
- Neither numeric rating nor comments are accepted
V Presentation of Individual Ratings
1. The actual numeric ratings given via the anonymous rating
interface will be shown next to the comments. The numeric rating will be
presented in a color-coded square to convey how the given rating measures up
against all other ratings given by the same rater to other images (higher,
lower, or nearly the same).
Below are four possible colors the site will display to indicate relative ratings:
|
|
This rater rated most other photos lower than rating "N".
|
|
|
This rater rated most other photos higher than rating "N".
|
|
|
This rater rated most other photos nearly the same as rating "N".
|
|
|
Not enough data to show relative rating for rating "N".
|
2. An ability to view a rating pattern of each rater who
has provided any feedback on any photos will be added. This will be shown in
the "Rater" column of the "Rating Details" section. Rating activity patterns
may help better understand how each rater rates over time. Unlike the simple
average of the given ratings, patterns show how ratings given by raters are
distributed throughout the entire rating scale. The rating pattern icons will
replace currently used rater labels ("Easy rater", etc) and will show
information about given ratings in a more meaningful way.
Below are nine possible rating patterns the site will
display for each rater:
|
Consistently low rater.
|
|
Mostly low rater
|
|
Consistently mid-range rater.
|
|
Mid-range rater.
|
|
Full-scale rater.
|
|
Mostly extremes rater.
|
|
Severe extremes rater.
|
|
Mostly high rater.
|
|
Consistently high rater.
|
3. All existing ratings will be preserved and shown the same way as they are
now, with the exception of the first rater and ratings given by former members.
The "First rater hidden" label will be replaced with the label
"Hidden", and the actual numeric rating will be shown. Ratings given
by former members will also become visible.
VI General
1. Raters will have the ability to edit comments, but not ratings that they
have left previously. Comments will be editable, however, if the rater chooses
to edit the comment, the numeric rating associated with this comment, will be
removed. The edited comment will be shown with the rater's identity regardless
of how it was initially submitted, thus converting this feedback into
identifiable.
2. The ability to begin a two-way forum-based discussion
about a photo's possible flaws and ways to correct them will be implemented. Photographers
will be able to post their photo to the new forums category called "What's
wrong with this photo?" where others can choose to pin-point problems and provide
in-depth suggestions on how to fix them.
3. The link to neglected photos will be removed, as well as
sort options that include neglected photos. The introduction of the two-way
anonymous rating process makes the concept of separating neglected photos
obsolete.
To summarize, the changes listed above are meant to adjust
the culture of the rating process. Ratings will no longer be the same, and getting
adjusted to the new system will require a change in everyone's way of thinking
about them. The new rating system promotes constructive criticism, which will
enable raters to honestly express their opinions about other photographers'
work, which in turn will help photographers sharpen their skills.
|