ophrys's picture

Increasing numbers of high reputations

I really do think that iSpot needs to do something about the reputation system, as it is getting harder to change IDs to the correct one, as more people get the higher reputations. This one today typifies the situation...

http://www.ispot.org.uk/node/322075#comment-106246

Three people have gone for Skylark, where it is a Pipit, and it is frustrating that two who clearly know the birds better are unable to override the ID.

I really mean no offence to those who have gone for Skylark, as I am a geat believer in what iSpot is all about and how it allows people to hone their skills and earn a reputation. That is all great and it is good to see people getting better in their IDs.

However, I find it frustrating as an experienced birder that I can no longer override wrong IDs, as others reach 5 stars too. Should there be another level, now, to allow for the stage where many have reached the higher reputations?

Reply

Comments

Ray Turner's picture

Patience :-)

I think you have a point Ian and have always assumed five stars would someday become a gold ‘expert’ or perhaps a six at least. However I feel the example you have chosen, whilst frustrating, may not be the best. At the time I write this the two alternative IDs only have five agreements between them. One has 1 x 5 star + 2 x 4 star, the other 1 x 5 star + 1 x 4 star - me and thee. Two more reasonably ranked agreements and yo we have a Tree Pipit (assuming they agree with us of course).

I do think the general question of what happens when we are all running round with maximum reputations though is an interesting question.

Ray

Ray

ophrys's picture

Reputation

Thanks, Ray.

I think it is a good example of what I am saying, as the misidentification is such a basic one and I feel that it should be possible to override it quickly. As more and more people have 4 and 5 stars, it is increasingly hard to change such a basic misidentification and that is frustrating. Experts are rarer than hen's teeth, so I think iSpot does need to think about other levels of expertise.

Edit: Just found a hen's tooth! David has come in with an ID of Pipit, which is sensible and has immediately overridden everything else. Doesn't alter my basic point.

Ian
_________________

My Flickr photos...

http://www.flickr.com/photos/52163027@N02/

JoC's picture

headings on posts

I have just arrived at this discussion, so I looked at
http://www.ispot.org.uk/node/322075#comment-106246

A question - you all agree it is a pipit - but who can change the main heading which still says 'Skylark at Arbroath'? Just the original poster?
Jo

Jo

Rachy Ramone's picture

As far as I know...

...yes, just the original poster.

But, as I have learned, there is value to leaving things as they are: it shows that people make mistakes, it shows a "trail" of input, it shows what this specimen might be confused with, and not least, it often prompts people to make helpful comments along the lines of "if it had X then it would be an XXX but as it has Z it must be a ZZZ"

So it's often better to leave the post as it was.

Also, minor point, those of us that tick the filters and follow observations to which we have contributed can get a bit miffed when the poster keeps changing things, as the obs pops up as "core details changed" which makes me rush off to have a look, then I take ages to work out that they have merely changed the spelling, or added the comment "two comments have been made".

Rachy Ramone

How to take close-ups with cheap phone and hand-lens:
http://tree-and-shrub-id.blogspot.co.uk/p/how-to-close-ups.html
Field Guides for Budding Botanists:
http://www.amazon.co.uk/-/e/B01A8YB0WY

ChrisMcA's picture

I (once anyway) changed a

I (once anyway) changed a title to 'not the gt silver diving beetle' & description to "CORRECTION: etcetc" (is that enough to deter google?)

Rachy Ramone's picture

*laughs*

Well, I guess that sort of title is probably worth changing!

I only meant to suggest that it's not necessary to change them, and it's a bit annoying to keep on changing them!

Personally - being a bit geeky - if I feel that I have to change a post, I add a line in the description saying "Edit: spelling corrected" or "extra pic added" or whatever.

Rachy Ramone

How to take close-ups with cheap phone and hand-lens:
http://tree-and-shrub-id.blogspot.co.uk/p/how-to-close-ups.html
Field Guides for Budding Botanists:
http://www.amazon.co.uk/-/e/B01A8YB0WY

JoC's picture

Changing entries

I agree the 'comment trail' needs to be intact, and the change to title I was thinking of was'Initial ID of Skylark... has been challenged/amended'. It won't stop Google locating it, but a visitor the the post could be encouraged to read on and learn for themselves (one of the aims iSpot) how skylarks differ from pipits. Adding the word 'edit' to an edit is, of course, another useful way to keep the trail.

Jo

hydrurga's picture

Re: Two comments have been made

Ah, I would be one of those posters Rachy. When the "Changes" function flags to me that a comment has been made on one of my observations, I go and have a look at it. The relevant flag is automatically reset on my clicking on the link and looking at the comment, so, if I don't reply straight away, I might very well forget it was there.

The only way I have found to address that is to include a note in the observation description along the lines of "X comments". The same goes for the case where the number of identifications is increased. On scanning my observations at a future point, I compare my note in the description with the summary details listed for the observation and can tell straight away whether I need to carry out some action for that observation.

The other thing that I do is, once an observation has in my mind been confirmed as a particular species, I go back and change its title, and those of all the photos, to reflect the common and species names. I do this to keep things organised and make it easier for people viewing my observations.

I wish there were a way in which I could make a change but tick a "minor change" box so that it would not flag my change to anyone else. Perhaps there already is and I just haven't seen it.

To summarise, sorry for being one of those guys!

Pete
http://www.leptonyx.com/nature/

Rachy Ramone's picture

No, Pete...

...you are a perfect example of How To Do It - you add a comment that says "Edit - extra xxx added" or whatever the change is.

Which is great!

Even greater is your truly excellent suggestion that there could be a tick box for "minor change" that would not be reflected in the filtered selections.

That would be AWESOME!! (as the kids say)

If I were being picky, I would query why you keep a note on the Core as to how many comments and/or IDs you have received. I'm intrigued! Does it matter? Like you, I flag up my filters for any new comments, I love to read them, they are often interesting, and often informative. If you look under your MySpot page, and look at the tab for Observations, it presents them in the same layout as the main page, so the number of IDs, agreements and comments are listed next to the photo. So I don't feel any need to add that to the Core description.

But, each to their own! As long as you are happy and organised, that's all that matters, really.

Rachy Ramone

How to take close-ups with cheap phone and hand-lens:
http://tree-and-shrub-id.blogspot.co.uk/p/how-to-close-ups.html
Field Guides for Budding Botanists:
http://www.amazon.co.uk/-/e/B01A8YB0WY

stevegregory's picture

Need more experts to correct errors!

As a relative newcomer to iSpot I'm a bit dismayed by how many clearly incorrect IDs there seem to be. Within the invertebrate section there are a few people with '4' and '5' star reputations (who in my mind should know better) who frequently make id's that are not even close to the mark (in some cases wrong Order, or worse). Fortunately there are many more who are reliably spot on. For the more obscure taxa there are not that many people, and even fewer 'experts' who can over-ride such erroneous IDs and the incorrect id is likely to remain.

I would hope this is less of a problem with birds and plants?

Steve

DavidHowdon's picture

Inverts

There are particular issues with invertebrates I think because it is such a large group. My invert reputation is largely derived from my knoweldge of lepidoptera but is applied when I try to ID, for instance, a beetle.

Subdividing invertebrate into subcategories has been discussed but the iSpot team is reluctant (rightly I think) because it adds more confusion to inxeperienced users who might not be able to correct assign for instance a heteropteran bug to the right order.

I think they are exploring a method to a allow people to agree weakly, so for instance I could suggest or agree a beetle ID but not apply my 'reputation' to it if I do not feel I have knowledge in that area.

Jonathan's picture

Indeed we are exploring a way

Indeed we are exploring a way to narrow reputation, the problem is once you start, where do you stop?! Simplicity is really important.

Jonathan
University of Edinburgh and Biodiversity Observatory (OU)

stevegregory's picture

Not an easy task I

Not an easy task I appreciate.

Steve

jeremyr's picture

subcategories

it works perfectly well on the Danish site (fugleognatur.dk) with a lot less confusion from beginners than you might imagine. Only occasionally does a wasp or bee turn up in the fly section, for example, and when it does it only adds to the interest as it's usually due to mimicry. There's certainly no more confusion than already occurs on ispot, and allowing people to speciallise appears to make each subcategoory more attractive, presumably as it's more cogent. And reputations are built acording to one's area of knowledge

.............................
latest pics and diptera videos

Jonathan's picture

Interesting site for

Interesting site for comparison. Thanks.

Jonathan
University of Edinburgh and Biodiversity Observatory (OU)

DavidHowdon's picture

Stars

There are discussions about the points system elsewhere that probably explain this better but here is my attempt.

The 'stars' (birds, butterflies, plants whathaveyou) are just a representation in pictorial format of the underlying 'score'. Five is the most pictures you can have but your underlying score can continue to rise. You can see this because it does not always take the same number of "four bird" people to over-ride one "five bird" person.

There was a discussion a while back whether to have the additional badges appear (so allowing a six bird person), not sure what happened with that idea.

Another idea was to have the actual score displayed but I think there was a concern that this would encourage to much competition over reputations, not really what iSpot is about.

ophrys's picture

Agreed

Yes, I think that is how it works...thanks, David.

It is a fact, though, that if someone with a recent 5 bird reputation makes a wrong ID, I cannot override that, even if I have made nearly 1000 more IDs than they have. That seems wrong to me...

One of the great things about iSpot is how quickly a correct (or as correct as can be) identification is given to a posting. I think some of the credibility of the site is lost if misidentifications creep in and cannot quickly be changed.

Ian
_________________

My Flickr photos...

http://www.flickr.com/photos/52163027@N02/

chrisbrooks's picture

Correct ID

I think you mentioned it once before Ian on another posting, eventually the "correct ID" will win out.

On the other point I don't think points make any difference once you have reached 5 icons, I noticed it the other day when you could no longer over rule an ID and you had a truckload more points than the other party.

wolvobirder's picture

Experts

I do think that we could do with having more people ranked as experts. There are plenty of people on here that could be called such in my humble opinion. Maybe we could even have a two tier system where experts also gain reputation? Although I guess that would be hard to do as you would need experts to agree with other experts and currently there are just not enough experts.

There are some users on iSpot that I know are very knowledgeable people but they only have 3 "stars" as they are fairly new to the site or infrequent users.

rstewartb's picture

Agreed

I agree. And the opposite is also the case. As a keen photographer of bird behaviour but with very limited knowledge and only recently having started to understand how to distinguish birds I find myself struggling with the four bird reputation that Ispot has given me, I presume because I post quite high volumes of observations and always have a go at ID as advised. I worry that I will unduly influence other IDs so I tend to be reticent about putting IDs against other people's observations. I would support a review of how reputation is gained and ID's are influenced.

In the short term perhaps a mechanism to accelerate the gaining of reputation for recent arrivals recognised as experts in the field by their Ispot 5 Star peers?

Or could we be made able to say how strongly we agree?

Incidentally is this a problem in the other fields?

Ray Turner's picture

Agreement

To some extent the option to express how strongly one agrees exists already. The observer has given a confidence level for the observation and personally this forms an element of the decision to agree or not. So if someone has said ‘I’m as sure...’ and I’m not in agreement to that level I won’t click agree, conversely of course if they suggest ‘It might be...’ and I’m sure then I will. For some reason many people find it strange I do this.

Ray

Ray

Masked Marvel's picture

I think a problem is that

I think a problem is that people can earn lots of reputation by repeatedly identifying the same common species, which earn lots of agreements. One option could be to change the weighting for the amount of reputation that is earned for the species. Now that there is a huge volume of observations it should be possible to work out the average number of agreements for a species and divide the maximum amount of reputation by that.

E.g. I would estimate the average number of agreements for a common toad is 12, therefore the maximum amount of reputation that could be earned is 0.08. Whereas for a midwife toad, the average is 5 so this would earn a maximum of 0.5.

Ray Turner's picture

Good Option

I like this option and think this is the way to go. Really clever programming would allow dynamic weighting so there would not need to be much in the way of admin supervision.

There is however an inherent issue with the reputation system and it is that if you are an active member of the site for long enough you will gain a very high reputation regardless of actual knowledge or ability, I’m not sure how you address that.

Ray

Ray

Masked Marvel's picture

Maybe a cap on the number of

Maybe a cap on the number of times a species can earn reputation would help? i.e. you can only earn reputation for the first 10 times you identify a robin. After that you can still add an ID but it won't earn any reputation. This means that in order to earn high reputation an iSpotter would need to demonstrate breadth of knowledge across a number of species.

DavidHowdon's picture

Perhaps

You would presumably need different limits for different groups - for instance with mammals or amphibians and reptiles there are relatively few in the UK so that cap (per species) would need to be high if anyone is ever to obtain high reputation.

jeremyr's picture

repetition

Masked Marvel identifies precisely the great weakness of the reputation system, that it is based on volume rather than scope, reflecting a fast connection rather than breadth of knowledge. I have been continuously astounded that anyone can take this crude system seriously - I identify two hundred Gooden's Nomad Bees and as many Stretch spiders and I get five stars? Utterly rediculous, and profoundly embarrassing. I know nothing about invertebrates, nor do I have any knowledge that could be described as expertise, and yet would appear to many users as an equal alongside others with genuine expertise. Of course people get to know who the most knowledgeable are, but being ranked as an equal makes it worse. The reputation system has turned out to be much cruder than we all wanted it to be - I'm grateful to MM for voicing it, and if something can actually be done whereby species (and stages, and genders) are graded so that it reflects range/difficulty etc that would be brilliant, though it sounds complicated.. honestly though, I can't believe it's actually taken seriously. If it is then I must downgrade myself by deleting loads of Obs or start again as a new user. Good luck with any revision of the system.

.............................
latest pics and diptera videos

chrisbrooks's picture

Reputations

I think it should be remembered that iSpot is primarily a learning tool and the reputation system is a bi-product of that. It should not be taken as a scale of expertise. As long as the correct ID is reached what does it matter.

Sadly if people are fed up with the system then there is an alternative but that would be iSpots loss.

Mydaea's picture

Well said! A daft system, and

Well said! A daft system, and it is important if the IDs here are to be taken at all seriously, which is one of the aims of the site. After getting some needling from other users, I stated my own position in a thread called 'Bewildered' somewhere in Forums.

lavateraguy's picture

[The reputation system seems

[The reputation system seems to work reasonably well as a means of automatically picking the correct identification from alternatives. [But 3 active experts in plants does tend to override everybody else's opinions.

But it doesn't work as a aide to users in understanding how reliable someone is in identification - you need to know people as individuals.]

Apart from the problem of reputation being based on volume, there is another problem - it is too exponential. You can get to 3 stars with 20 identifications, and 4 stars with about 100, but it takes 800 or thereabouts to get to 5 stars. This leads to a problem with discrimination on the basis of reputation - nearly everybody has 3 or 4 stars.

One the other hand, making the early levels easy to achieve encourages people to participate; and trying to change it now (making 3 and 4 stars harder to achieve) may upset people who lose their hard earned stars in the process.

Chopin's picture

In iSpot ZA

We have a knowledgeable badge. It falls in between expert and the ordinary user. To get a knowledgeable badge you have to be nominated by your peers.A person with a knowledgeable badge can still earn reputation points, similar to an ordinary user. For example, a person with a knowledgeable badge and three plant stars could overrule a person with five plant stars.

Lara

Amadan's picture

I'm a little worried -

That I somehow seem to have amassed 4 stars for Invertebrates.
I will freely admit that I often post the wee beasties partly because of a mild fascination with them, but largely due to the fact that they present themselves more readily than (say) birds or mammals - especially if you have a wife whose idea of a walk is exercise, rather than frequently pausing for wildlife photography.
Frivolity apart, I do think it would help if one could ask for their reputation to be downgraded, because I really don't think my expertise rates my current one. It's one reason I'm reluctant to add agreements, even when I'm pretty sure an identification is right.

dejayM's picture

Don't be..

I would see you, Amadan, as a friend if you contributed to my posts. That you have more 'stars' will mean you have contributed to more posts than me and will be more experienced in CONTRIBUTING.
You see, I believe that anyone can value-contribute, IF what they write is relevant and fairly well researched. I would expect a reasonable Naturalist to be able to do this in ANY field. Join us in the Marines!
http://www.ispot.org.uk/node/330390
Please?

Amadan's picture

That's rather the point, in a way -

I feel that I'm supporting other iSPot users if I contribute to an observation by agreeing or commenting.
One hopes that new users will be encouraged to submit more posts if they receive feedback: certainly for me the most disappointing posts are the ones where no-one either agrees, corrects, or comments. I feel better if someone disagrees - at least it shows interest and provides guidance.
But I don't want to find myself in a position where my opinion is considered excessively influential, given my patchy knowledge base. Hence my concern over an enhanced "star rating".

Rachy Ramone's picture

Modesty is a virtue...

...a rather under-rated one, these "selfie" days!

Amadan, your modesty does you credit, but you have successfully ID'd a large number of observations: you know more than the man in the street, you have "earned" your stars.

I shouldn't worry too much: you seem (to me) to have exactly the right attitude - you are concerned about not agreeing to an ID unless you are quite certain, in case your "higher" icon rating tips the balance. This is a valid concern, but not one which should hold you back from agreeing, if you are pretty certain about the ID.

As has been said, if an ID is wrong, it will eventually be overtaken by a correct one, no matter how many "votes" it has had.

So you carry on contributing!

Rachy Ramone

How to take close-ups with cheap phone and hand-lens:
http://tree-and-shrub-id.blogspot.co.uk/p/how-to-close-ups.html
Field Guides for Budding Botanists:
http://www.amazon.co.uk/-/e/B01A8YB0WY

Jonathan's picture

Thanks for all these thoughts

Thanks for all these thoughts on the reputation system. They are all useful and have been noted. We have various ideas about how to improve it and will be trying some of these out in Version 3 of iSpot that will appear later in the year. Meanwhile, remember:

1. You can withdraw your agreement, so if you are right and someone else is wrong, they can (in theory) be persuaded to change their mind. There is a lot to be said for this approach.

2. No systems is perfect, and the more complicated we make it, the harder it will be to show it is fair.

3. Overall, the rep system does work remarkably well. After all, its just a bit of computer code, isn't it.

Jonathan
University of Edinburgh and Biodiversity Observatory (OU)

dejayM's picture

voyage

Yes
1. this seems a very powerful way to express a revised opinion. It's a pity it leaves no trace in the original (dejayM has removed his agreement)

2. PLEASE don't make it any more complex - I am beginning to hate the 'voyage' into graphic interactions.

3. I REALLY admire this system. I don't want another 'star' until I've earned it properly. I would say, however, that the earliest stages are far too easy (one and two icons)
ðj

Jonathan's picture

Thanks for your feedback.

Thanks for your feedback. Simplicity is our watchword.

What exactly is it that you don't like about interactions, and how could it be improved?

Jonathan
University of Edinburgh and Biodiversity Observatory (OU)

dejayM's picture

getting past..

Well, there are a few things -
This is the simplest
http://www.ispot.org.uk/node/335943
Why, specially....? and what's the point?

And this, for instance,
http://www.ispot.org.uk/node/330914
is over-simplistic (no offence is meant (Bob) of course)

But this -
http://www.ispot.org.uk/node/333098
is getting cluttered. My main point.

This is the final result -
http://www.ispot.org.uk/node/330192
I HOPE it takes your computer as long as mine to download it - remember that such things go through computer protection too.

Even now, few people get as far as comments, which MAY be the most valuable aspect of iSpot (in my opinion).
So now the whole page is destined to be cluttered with, apparently, (only at first glance I suppose) unrelated pictures (three (I meant FOUR) pictures of Oak features in the example above)
They are not, by far, the worst examples, of clutter, I've seen (some have up to five IDs) but each of the above is selected at random just now.

I cannot imagine many 'senior' posters being very interested in Interactions nor can I see very many novice posters getting past 'Similar Observations' or their own worries about getting the basics right.
There, I've said it! Oh dear..
ðerek

ophrys's picture

Disagree!

"I cannot imagine many 'senior' posters being very interested in Interactions"

I agree with much of what you have said above about pointless interactions, like two different birds on a feeder. The comment above is totally wrong, though. Interactions are exactly what interest anyone serious about what they study. I fully intend to do some interactions, as they really interest me. However, I shall only do them once I have two good and different pictures to use, as I also don't like interactions which just repeat the same picture.

Ian
_________________

My Flickr photos...

http://www.flickr.com/photos/52163027@N02/

rstewartb's picture

Youngsters

Lets not forget the youngsters. My grandchildren have started taking an interest in Ispot now that interactions are visible. What the experienced may view as simplistic are often ideal interactions for prompting the interest of children (why foxes eat birds for instance). While Ispot may not see this as its primary purpose, in my view getting some of the next generation interested in nature is invaluable.

Jonathan's picture

Thanks everyone for your

Thanks everyone for your feedback. We shall be improving the display in iSpot version 3 and will take into account all of your comments. Nice to see some positive ones as well!

Jonathan
University of Edinburgh and Biodiversity Observatory (OU)

Peachysteve's picture

Reputation

I'm late to this conversation but thought I would put my two penneth in.
I added an alternative ID to a post today and was quite shocked when that became the "likely ID". I'm sure the person who posted actually knows far more about plants than I do. Of course after a few agreements the correct ID will become clear.
My point is a suggestion about how reputation might be built when considering ID and agreement. I don't know how it does work exactly but I was thinking of a sliding scale of credit, examples would be.
I add an alternate ID to an existing post which is then agreed to by an expert - Highest credit.
I add an ID to an post with no ID which is then agreed to by an expert - High credit.
I add an ID to my own observation which is then agreed to by an expert - Medium credit.
I agree to an ID which has not yet been agreed to by an expert - Low credit.
I agree with an ID that has already been agreed to by an expert - No credit.

I would be quite happy for any change to this system to be implemented retrospectively. I'm learning a lot with the help of this site but I'm never going to have the depth and breadth of knowledge that many other members have.

Jonathan's picture

Thanks for your suggestion.

Thanks for your suggestion.

Jonathan
University of Edinburgh and Biodiversity Observatory (OU)

dejayM's picture

Track

I am VERY interested in what Steve (peachysteve) writes.
I am pretty new to iSpot - 26 weeks.
I am gathering a 'reputation'.
I need input (not JUST agreements) from people with reasonable 'reputations' - it keeps me going, it keeps me interested, it keeps me posting. I can't get enough!
I judge value of input from others by looking at what they say in Comments. I sometimes judge their comment against their iSpot Reputation.
But (here comes my point) I look at their record of interaction with others (Track); this allows me to see a real 'reputation' based on quality input - not just Posting and Agreeing.
There are some, like Steve, who build their iSpot Reputation by Posting and Agreeing. There are others who build their 'reputation' by commenting and interacting. The latter sometimes, possibly often, do not have a strong iSpot Reputation.
I have come to believe that it is preferable to receive Comment from people with a broad interactive 'career' in iSpot than someone who mostly Posts and Agrees.
I would HATE Steve (peachysteve) to see this as a snub. Whilst he seems to be shy about his own iSpot Reputation, this is built on a sound understanding of his numerous and interesting posts - he receives many expert agreements and, of course, Comments.

Jonathan's picture

Comments get you social

Comments get you social points, but do not contribute to your taxonomic reputation.

Jonathan
University of Edinburgh and Biodiversity Observatory (OU)

dejayM's picture

social reputation

Yes Jonathan, I have come to realise that.
I was trying to illustrate that there are TWO sorts of reputations in iSpot - both of great value (to me at least).
I am saying (and of course you know) that both iSpot Reputation and 'reputation' can be checked by anyone coming here - that is what I sometimes do to 'measure' the value of a comment, sometimes perhaps, an Agreement.
Doing that with your Reputation(s) allows me to say that I would GREATLY value your input, both as comment and agreement.
We, you and I, are both 5 star 'socialisers' - what is the figure in brackets (3063)?
Personally I do not think you should change the iSpot Reputation system. I am coming to understand it and, when I get my fourth in Invertebrates and eventually Plants, I will consider I've earned them.
.............
Back then to the original post in this thread by Ian (ophyrs), that people with a high iSpot Reputation, gained through posts and agreements sway the Likelihood. I am three icons, surely quite low in the scale of 'expertise', yet I sway the Likelihood factor by quite a lot; in two or three (maybe more) this has been to the detriment of a correct ID and has not correctly deferred to the proper quarter - someone with a very favourable 'social reputation' (I'll use that expression from now on) but with a low score iSpot Reputation.
For me though and I suspect a lot of middle range, broad-interest iSpot naturalists, it is an EXCELLENT site.