jccurd's picture

Missing Personnel

With humble apologies to Pete Seeger, I'll leave you to figure out the rather obvious tune to these iSpot lyrics:

Where have all the experts gone?
Long time passing
Where have all the experts gone?
Long time ago
Where have all the experts gone?
Spread too thinly every one
How will we ever learn?
How will we ever learn?

OK, that's all tongue-in-cheek but it does convey what feels to me like a bit of a current problem.

Over the past, say, 3 weeks, I've submitted almost 30 observations split between the Invertebrates (~20) and Birds (~10) categories. These have received almost 200 agreements/comments. Just one of my observations attracted a single expert and that was in the Invertebrates category; there appear to be no Bird experts out there.

I know some people may be concerned about points issues but my main concern is the educational aspect of iSpot and the confidence level we may have over the identification of what we might see.

Que pasa?

Reply

Comments

ophrys's picture

MIA

I agree with you about the lack of experts, especially in Birds. I wonder whether they get bored, as most bird IDs on iSpot revolve around Willowchiffs, Trimpits and young Dunnocks!

On a serious note, the current lack of expertise may be due to the fact that this is very much a time of year when any self-respecting naturalist should be out and about observing nature, rather than sitting in front of their c...

jccurd's picture

OK

I've got the Willowchiffs reference but you're going to have to spell out the Trimpits bit for me, ophrys. ;-)

DavidHowdon's picture

Too successful?

I suspect iSpot might be a victim of its own success here. Near as I can tell they did a good job of getting a lot of experts signed up early on, so not that long back the expert to user ratio was quite high.

Now iSpot has been widely popular and the number of users has gone up, but the number of experts probably has not gone up so quickly (because they got most of them signed up early).

So to those who have been around a while it looks like less expert input (and indeed the chance of expert input on any given entry has decline).

--
David Howdon

jccurd's picture

That's my impression, too

I think you are very close to the truth, David. I mentioned the "victim of its own success" issue to Martin Harvey at a keys testing session last year, when the situation wasn't quite as bad as it seems now.

Compared to two years ago, observations from a much larger user community are hitting iSpot so rapidly that they disappear off "the radar" very quickly. I have the feeling, and it is just a feeling, that there may actually be less experts around these days attempting to deal with the flood. Getting one of the few to see one's observation is largely a matter of luck and timing, it seems.

The success is good because it means more folks are taking an active interest in nature but the loss of a reliable identification system is unfortunate. Of course, there are still some reliable old school members such as yourself and ophrys.

Time to switch from Pete Seeger to Winston Churchill:

Never before in the fields of our nature reserves, has so much been owed by so many to so few.

miked's picture

Actually I am not sure this

Actually I am not sure this discussion is going in quite the right direction. As an 'expert' myself I tend not to interact quite as much as I did when the site first started because the intermediate level people are getting the correct ID's so I don't think I need to give the ID myself. I do still look at thousands of observations without giving an agreement or comment, instead mainly concentrate on ones where I think something may have gone wrong or if they might be difficult and think people may be happy with an extra confirmation (if I am able to add anything useful that is).
If I jump in and give ID as soon as something appears then there is not really much incentive for the people who are learning to have a go. Also if I keep adding in ticks to loads of observations there is a danger of the reputation shooting up too quickly for the very easy to ID things and so people can get reputation very quickly just by putting on loads of easy things and IDing them themselves.

Fenwickfield's picture

interesting

I must say I agree with miked that there are some very knowledgeable "None Experts" I have also seen some experts agree or put an identification on which is wrong and it has been corrected by some of the very knowledgeable users I will refrain from mentioning names.I also agree that it is a lot more interesting if you try to identify something yourself and if your really stuck give as much information as possible about the subject.I as a non expert become rather frustrated in certain area's for example fungi when people put on very poor photo's with no information what so ever and expect an identification as you wonder if they really want to learn about the subject I have tried to help and encourage those interested in this subject and can see why the experts fade away,I have always had a good response myself with experts helping me in plants and fungi.I am not that good at birds and inverts but there are a couple of well known ispotters who have always helped and been right in there identifications and I would trust there identifications as much and at times more than some experts.I also think the reputation system is alright if you use the site on a regular basis as it does not take long to realise who is well read on a subject and who is not,Maybe some of the non experts will eventually become the experts I would nominate a few for this post.

Fenwickfield

Masked Marvel's picture

Reputation system

If experts aren't agreeing with IDs to avoid over-inflating reputation points on easy species, surely this is a problem with the reputation system? It must be possible to add a difficulty factor into the reputation points? iSpot now has a huge database of observations and surely an algorithm that incorporates some degree of "difficulty" (e.g. average number of agreements per observation of that species) into the amount of reputation earned for an identification.

miked's picture

Interesting suggestion Masked

Interesting suggestion Masked Marvel, will see how easy it is to investigate this. Have been looking at 'difficulty' since ispot started and thinking how to incorporate it into reputation. It has proved impossible up to now as relying on the different schemes and societies for each group of organisms to develop their own 'degree of difficulty' for each species. Don't want to implement something that has to be frequently changed as each new set of organisms has its degree of difficulty estimated as this could have all sorts of unexpected effects on reputation.

Fenwickfield's picture

Good idea

I think it is a good idea but you may need an expert to put them into different categories from easy,medium,difficult or a point system.

Fenwickfield

jccurd's picture

That's all very well.

As good as a difficulty sounds, I'm not particularly concerned about reputation points, personally. I'm much more concerned about having a high level of confidence about an id, whether mine or someone else's. My confidence level obviously increases when I see an expert agreeing/supplying an alternative id. As such, I find it a little disturbing that experts may be sidestepping an id intentionally, for any reason.

I value an experts opinion and am always interested to receive it. I don't think that's going in the wrong direction at all.

Will Avery's picture

This difficulty can vary for a given species

I just identified a common plant for someone who couldn't recognise the species, almost certainly because the petals had fallen. Similarly ornithologists have told me that identifying juvenile gulls is particularly tricky. I can do common butterflies in their adult form, but would be hard put to identify the larvae.

Jonathan's picture

John, I think that you may be

John,

I think that you may be underestimating the degree of knowledge that iSpotters who do not have an expert badge have. I just took a look at the last observation you made http://www.ispot.org.uk/node/264278 and three people agreed with the name, all of whom had high reputations in the relevant group, but only one of whom is badged as 'expert'. Expertise in iSpot is spread over a much wider group of people than those with an expert badge. You don't get 5 reputation icons easily, as I'm sure you know.

More generally, the whole point of iSpot is to increase expertise, not simply to funnel observations to established experts. The progress of iSpot shows that we have been successful in doing this. >50% of observations get a likely ID within an hour of submission. That's an amazing thing. If we relied exclusively on badged experts, that just couldn't happen.

Jonathan
University of Edinburgh and Biodiversity Observatory (OU)

DavidHowdon's picture

Two minds

I'm sort of in two minds about this. One of the ideas of iSpot as I see it is to find a way of implementing the idea of amateur ability and reputation. On that basis we should seek to get to the point where individuals with high reputations are viewed as being as good as (if not better than) badged 'experts' for identifications.
I note that in the birding world this is largely what happens anyway, the 'top' birders tend to be amateurs and their reputation is an important part of this.

But all that said I'm not sure the iSpot system has got there yet. This is not a critcism of the work iSpot has done on this (which is very good) but currently I don't always find myself comfortable with an ID provided by someone with five 'stars' in all cases.

Perhaps this is mainly because I do invertebrates where I have earned five butterflies mainly from my knowledge of lepidoptera. Even in leps I do not regard myself as close to 'expert' and any suggestion that my IDs of other insect orders should be given any great weight is laughable.

There are of course some people on here I trust as much as any expert, but in part their reputation (with me) is based on what I have seen them do here, and how they have identified things, not just the number of 'stars' they have.

So for now when I post something for ID I will tend to only add it to my records as definitively identified if it gets agreed by a relevant expert or by one of the other iSpot users who I have seen demonstrate a level of expertise I trust.

--
David Howdon

jccurd's picture

Good pick, Jonathan ...

... in that it was a rare observation that actually attracted input from an expert. As I said originally, I had a period where 29 observations attracted no expert at all.

I'm not surprised that >50% of observations get a likely id within an hour since any suggested id by any level of member gets tagged as the "likely id" until overtaken by another made by one with a higher "reputaton". Only observations with no id at all do not have a likely id. (And I have a couple of those, by the way.) The question is, how likely is that "likely id"?

I do have a number of non-experts whom I trust implicitly. Some of those gained my trust initially by being backed-up by an expert. One in particular, technically with only four butterflies like myself, demonstrated his in-depth knowledge by going to great and articulate lengths to explain the reasoning behind his ids time and time again. He is the reason I learned a considerable amount about Odonata. He is unique.

To know that my knowledge has increased, i.e. I've learned something, I need my class work marked. In most cases, unless I have an expert say, "yes, that's correct" or, "no, it's one of these", how am I to know I've progressed? An equivalent would be doing a French translation without having a French speaker check it.

I believed an incorrect butterfly id, made by a non-expert with a high reputation, for a whole year before realizing that it was wrong. [Piedmont Ringlet vs de Prunner's Ringlet. tricky for an amateur but it wouldn't have been for an expert.]

Jonathan's picture

I must correct two points: 1.

I must correct two points:
1. It is NOT the case that "any suggested id by any level of member gets tagged as the "likely id". There is a threshold of reputation points that trigger this. The icons are only a very crude reflection of points and they can mislead.

2. 'Likely ID' is a carefully chosen phrase. 'Likely' does not mean 'certain'for the very good reason that you cannot be certain about most IDs. Anyone, even experts, can get them wrong at times. Being wrong is nothing to be ashamed of. If one is afraid of being wrong, one cannot learn and iSpot is first and foremost about learning.

Jonathan
University of Edinburgh and Biodiversity Observatory (OU)

jccurd's picture

Fair enough, Jonathan

I stand corrected on #1, though I certainly hadn't noticed any cut-off, and of course realize that #2 means likely (probably "most likely compared to other suggestions") rather than certain. It is, after all, just a photograph where examining the genitalia, as often required for invertebrates, is normally not possible.

I still think, though, that to have confidence in progression (of one's knowledge), we need a high degree of likelihood which can best come from an acknowledges expert.

It is also frequently an expert that needs to put the brakes on and say, "you've gone too far - we can go only to genus level from this evidence".

Will Avery's picture

Contrast with plants is important

While I agree with what Jonathan says, I'm still intrigued that, in contrast to the Birds and Invertebrates groups, it seems that scarcely a leaf or petal can fall onto the Plants group without an expert passing an opinion at some point, small though their numbers may be.

I think this difference is worth addressing, because if longstanding, productive and knowledgeable members of this community, such as jccurd, are less than happy about their experience of the site they will use it less. I'd speculate, and I'm sure he can tell me if I'm wrong, that jccurd is at a level of knowledge where he is able to interact with experts to learn things and gain insights he couldn't gain elsewhere. Or at least, not gain without leaving the house. The approbation of acknowledged experts may also be important, alongside the gratitude of less expert users who are helped. The experts, though they may be small in number and may not intervene much, are like the salt in food. And if the four- and five-butterfly people start to drift away then certain parts of the site will be going backwards instead of forwards.

jccurd's picture

Quite right, Will

(Apologies for the delay, by the way - long holiday chasing critters in France)

I have managed to build a small circle of contacts that I can interact with. My particular interests are Odonata and butterflies. There are a handful of de facto experts (i.e. not badged "expert") within iSpot that I trust. However, within the Internet communities that share my interests, I've managed to make contact with other acknowledged specialists which I've increasingly begun to use in preference.

Admittedly, those external folks do not get swamped with the volume of observations that iSpot experts do.

The big problem comes when I'm struggling to find an id for something new - something about which I know next to nothing but am keen toi learn. Moths are a great example - I'm a self-confessed moth numbskull. Here, an expert would be invaluable since I lack the knowledge to assess a non-expert opinion.

Jonathan's picture

If you know experts who are

If you know experts who are not using iSpot, please ask them to join in. If they are recognized as such in their field, we shall be delighted to hear from them via the Contact link (at the bottom of any page).

And by the way, have you seen just how many organizations use iSpot now? http://www.ispot.org.uk/representatives

Experts who belong to any of these can ask their society to nominate them for a badge. We'd also love to hear from societies not yet badged.

Bottom line is, iSpot is only as good as YOU make it.

Jonathan
University of Edinburgh and Biodiversity Observatory (OU)

jccurd's picture

Hmmm ...

Bottom line is, iSpot is only as good as YOU make it.

You should be in politics. :)

Much as I might like to drag in other experts, my contacts are specialists in dragonflies or butterflies, not in invertebrates in general. That's perhaps why my most trusted source, already on iSpot, is not tagged an expert. As David Howden suggested (above), ids in a wider field on inverts. would not be appropriate.

Invertebrates is a huge field and, perhaps, would benefit from subdividing? Now there's a thought.

Jonathan's picture

"Bottom line is, iSpot is

"Bottom line is, iSpot is only as good as YOU make it." In a social network, this is undeniably true. Its not just rhetoric, but thanks for the suggestion for a new career, anyway ;-)

Good point about specialism and this is one that we have considered in depth. I agree of course that 'Invertebrates' is a meaningless category from a biological standpoint, but unless one wants an unworkably large number of categories, you have to have some kind of simple system. Even if we had a category ''flies', there are experts in different families of Diptera. So, where do you stop?

The solution is related to what I said earlier, which is that an expert is someone who knows the limits of their knowledge. Experts already exercise their own judgement in deciding when to help and when to desist. Recognizing this, we intend to alter the system in a few ways that make it easier for experts and others to show the level of their agreement and for all to see where, geographically, these agreements are applied. The first could be done with a traffic-light system, replacing 'I agree' with red/amber/green for: "I disagree", 'It might be this' and 'I agree'. The geographic jurisdiction that iSpotters (not just experts) exercise can be shown in a thumbnail map with dots of the 3 colours.

What do you (I mean anyone reading this), think?

Jonathan
University of Edinburgh and Biodiversity Observatory (OU)

DavidHowdon's picture

Sounds positive

As I've mentioned elsewhere I'm starting to feel reluctant to suggest IDs in invertebrates where I do not basically 'know' the answer because I've got a fairly heavy iSpot reputation in that area. The ability to suggest an ID with an 'amber' rating (or to weakly agree) so as not to use my reputation would also be helpful.

The ability to 'disgaree' is also useful, currently I usually deal with disagreeing with an ID by going up (down?) the taxonomic tree until I reach something I can confident of - so for instance my take a moth I consider mis-dentified as 'lepidoptera'. Ability to disagree would be a better approach.

Do we know how 'disagreement' would interact with the reputation system. Will people lose points for posting an ID that is disagreed with? Personally I think something like that may be desirable (as otherwise reputations are simply ever increasing) but I could see it being unpopular.

--
David Howdon

Jonathan's picture

We have not decided on the

We have not decided on the details yet, which is why we value your opinions on this. Subtracting reputation points is tricky and I think that we shall avoid that if we possibly can. Points accrued by the agreement of non-experts are fractional and so taking them away is not going to make a lot of difference anyway.

Jonathan
University of Edinburgh and Biodiversity Observatory (OU)

Sarah West's picture

Like...

I like the "It might be this" option. For inverts, lichen and fungi especially, it's often really tricky to identify from a photograph, so it would be good to have the option of basically saying "well, from that photo, you can't really tell, but it might be".

On an unrelated note (sorry), is there any way of flagging a first time poster? I worked at the Richmond BioBlitz at the weekend and encouraged someone to use iSPot for the first time. Unfortunately no-one (apart from me!) responded to their posts and I fear they might be discouraged. If we could flag first time posters some way, perhaps others in the iSpot community would be more likely to comment on or ID their photos?

Sarah West
www.OPALexplorenature.org
OPAL Community Scientist
Yorkshire and Humber

Fenwickfield's picture

new topic

Hi Sarah
It may be worth starting a new topic on this than running from this thread.If you know what they posted I will have a look as I normally do look at new ones as you can tell as they have no icons and usually don't put the scientific names on,but I always encourage and try to help as I thought that is what iSpot is about

Fenwickfield

Sarah West's picture

Thanks.

Yes, you are one of the people that makes iSpot great :o)

Sarah West
www.OPALexplorenature.org
OPAL Community Scientist
Yorkshire and Humber

Jonathan's picture

Part of avoiding

Part of avoiding disappointment is making it clear that although comments and IDs may be offered within minutes, it can take longer. It just depends who is online at the time.

Jonathan
University of Edinburgh and Biodiversity Observatory (OU)

Sarah West's picture

Yes, of course

But sometimes what happens is that no-one responds in the first couple of hours, and because iSpot it so well used now the observations don't appear on the first couple of pages, and therefore don't get seen by any experts, or by people like me who aren't experts but try to encourage people. Which I guess goes back to jccurd's original point.

Sarah West
www.OPALexplorenature.org
OPAL Community Scientist
Yorkshire and Humber

Jonathan's picture

Fair point. We will be

Fair point. We will be introducing new filters that will help people find observations more easily.

Jonathan
University of Edinburgh and Biodiversity Observatory (OU)