marksteer's picture

HEY iSpot Team

Is anyone there?

It would be good to get some feedback regarding the slowness of the site. There have been numerous posts about this situation with as far as I can see NO feedback from iSpot.
Today has been particularly abysmal.
I will not post new observations on the site at the moment.
Also I will discontinue trying to confirm id's or post comments.

I, and I'm sure many others, would appreciate your feedback.



Thistle's picture

Wind up the rubber band ...

... and tell us what's going on.

I've accepted iSpot as a slowish site but today has been dreadful. Apparently even the servers between me and iSpot have been giving up waiting according to the error messages(*). I've been reduced to reading Tax Tribunal appeal decisions between pages.

There's obviously a pretty serious problem with the site but applying the mushroom principle of management(**) to the users doesn't help.


* Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /latest_obs_grid_view.
Reason: Error reading from remote server
Apache/2.2.15 (Red Hat) Server at Port 80

** keep them in the dark and feed them on ....

John Bratton's picture

It is unusably slow on the

It is unusably slow on the Bangor library and Menai Bridge library computers, so I shan't be able to take part until this problem is sorted out.

It is surprising that I've seen no response from anyone running iSpot. But then I haven't read all the threads on this subject. They take too long to open.

John Bratton

Thistle's picture

Well, something's been done

Well, something's been done and we seem to be back up to a reasonable speed: thanks to the team. I hope it's a permanent solution. It would still be good if the team would tell us what's happening.


dejayM's picture

Round and round

Yes - speed is fine but it's still only Friday here in Orkney. I think there is a site pressure issue at weekend evenings.
Now - am I the only one to have problems today - cannot see Groups in any recent postings and had to edit several times the Group and Location - round and round I went.

lavateraguy's picture

I'm having problems as well

* Nothing has appeared in the unread or latest obs carousels for 6 hours.

* The observation I just added (370105) hasn't appeared in the latest observations carousel.

* The habitat field appears to have disappeared from the form/

* While iSpot let me enter a location, it's not showing up in the observation.

* And I'm not allowed to start new threads on this forum, so I'm piggybacking here.

landgirl's picture

Yes, me too

My locations aren't showing up, no reputations, no habitats, and the comments box has gone (but I can still call it up). Hope it's just some temporary attempts to fix the ongoing speed problem!
Edit: Although I'm not seeing any new observations, the ones I've just uploaded are being agreed with, so some people must be seeing them!

lavateraguy's picture

My last reply ...

... (mentioning that I found your observations via the track tab on your profile) seems to have vanished into the ether(net).

However, the observations are showing up on the global latest observations carousel, but not on the group-filtered ones.

dejayM's picture

general fault

Yep, stay away from Edit for a while. It's getting difficult to correct a misspelling (I always have a few!) without stating the Location and Group again.
It'll be fixed soon, won't it.

dejayM's picture

Changes or faults

Has anyone noticed that the Comment Trail is now upside-down, most recent at the top?
I don't like that - of course, I'll be the only one.
And when posting a comment, one can now only review at first, then save when happy with appearance. That's an improvement, for me, because I am able to see the proper layout AND if my (too-?) many hyperlinks work.

Amadan's picture


The good news: it is now very fast.
The bad news: When I "Browse Observations", I can see none later than ...node/37008.
I can browse the carousel, but if I open individual pictures, I can't see the location or any comments.
The latter are important to me - I have no particular interest, so I tend to pick out oddities, and any post that has comments - they are likely to be useful id notes, trivia (often great fun) or other fascinating info. In the main Browser, I could easily see which had comments
Thanks for speeding it up, but please, someone, shoot these gremlins quickly!
Also, I can't save this comment without previewing. That might be deliberate, but it's certainly new!
Nor could I add a new topic to the forum.

lavateraguy's picture

Is is faster because it's faster ...

... or because the number of active users has plummeted?

dejayM's picture


I think we've said enough. Like most others, I have stopped posting (but continue to comment, maybe successfully).
Do note, though, the number of duplicates appearing.
I imagine admin are frantic - give 'em some time - midnight's fine.

DavidHowdon's picture

Odd things

There do seem to be some odd things missing - reputations are at least not displaying, and may not be counting towards likely ID (see for example where I know from the names it should now be showing as a likely ID).

In random speculation mode I suspect the reputation thing could well be a cause of a lot of the recent problems. In effect it works something like person A's reputation is determined by the reputations of persons' B,C... etc agreeing with A's IDs. But B's reputation is determined by, amongst other things agreements received from A. So when B agrees with A that updates As reputation which updates Bs which preusmably means updating As etc. I imagine as the number of users grows the size of the calculation grows. No doubt clever alogrithmic things keep this manageable (for instance the capping of the reputation gain from any one ID presumably cuts out a lot of cycles).

lavateraguy's picture

Another problem ...

... is that it's displayed your reply in the "wrong" order - as a later reply it "should" have been displayed after mine.

I don't think that your speculation about reputation is correct. How I imagine it working is that you have a points score, and whenever an agreement is made you get extra points if the agreeer has a higher reputation than all previous agreeers. When you reach a points threshold you get an extra star.

I don't think that getting extra points retrospectively increases the value of your agreements - if it did I think that the system would have broken earlier. Note also that the accuracy of the reputation system has been criticised - it rewards volume as much as accuracy - and applying reputations retrospectively would exacerbate that problem, as users' identifications skill may be presumed to increase with practice. The reputation system is a quick and dirty and good enough means of identifying a likely ID - to use an expensive algorithm to calculate it would defeat the purpose.

That said, there is a potential issue here. To calculate the points to be added one has to look up each agreeers score. The number of users grows approximately linearly with time, and I believe that there has been an uptrend in the average number of agreements to an observation. If the table of users is searched linearly this could be a significant expense (depending on how much time is spent on other, larger, tables). You could improve performance by sorting the table of users and doing a binary chop search, or by using a hash function.

[I wondering how much adding the interactions feature has contributed. If I recall correctly the performance problems started occurring about the same time.]

From observation (and past experience with computer optimisation) I think that there are two problems.

Firstly, there's a threshold depending on the number of active users and the size of the databases at which the system starts thrashing - spending time copying virtual memory between RAM and disc and vice versa. As a quick fix, depending on whether the limit is the physical system memory, or the amount of memory allowed to a process/thread/database instance, this can be addressed by adding memory, or by upping the limit on a process/thread/databast instance.

This would explain the supralinear degradation at busy times.

Secondly, I think that there's a resource leak. Something - memory, or something more abstract - which has a limited availablility, is being allocated, and sometimes not released, so eventually the resource becomes prohibitivelty scarce, and processes end up spending their time waiting for it to become available.

That would explain the behaviour on last Monday and Tuesday, where I doubt that there was heavy usage, but the system was unuseably unresponsive.

Ray Turner's picture

Echo Everything and On Reputation

Echo everything everyone is saying on here.

On reputation; there was a slight change to how it worked a year (or probably more) ago. It used to be the individual score for an observation could grow ad infinitum, it now has a maximum score of one and will be achieved by any expert agreeing with the obs. The rest of us contribute some decimal fraction to the score until unity. It is then just a question of putting in enough obs to cross the thresholds. This does away with the infinitely referencing nightmare postulated by David.

One of the iSpot team (Jonathon?) explained this at the time in a forum topic if anyone fancies finding it.



Jonathan's picture

Hello all, apologies for the

Hello all, apologies for the slowness . We had someone working all last week to fix that, and now I see that there have been all kinds of horrible repercussions.(as mentioned below) These will be fixed ASAP.

Thanks for your patience in the mean time.


University of Edinburgh and Biodiversity Observatory (OU)

Mydaea's picture

I have just noticed that

I have just noticed that several items I placed in my Favourites in the past few days are not showing up there, although the confirmation message appeared when I clicked the 'Add' link.

David Trevan's picture


At last everything seems to be back to normal, so pleased to have our favourite website back, well done iSpot team!

David J Trevan

Jonathan's picture

Thanks. We are now aiming for

Thanks. We are now aiming for 'better than normal' !

University of Edinburgh and Biodiversity Observatory (OU)

Amadan's picture

My thanks also -

Judging from the unusually high activity in the Forum section over this issue, I don't think I'm the only one who is suffering a mild form of addiction to this site.
A good job we didn't have to go Meleagris frigida over it!

dejayM's picture


Was the inversion of the Comment Trail in Posts (most recent at the top) a deliberate move or a result of your recent 'epic'?
I didn't like it and it's unusual for any forum to lay comments this way - we usually progress downwards to gather the next one, rather like here, in Threads.
I later amended this comment because I have just noticed that the old way has been reinstated.

Jonathan's picture

ah! the old ways are the best

ah! the old ways are the best ;-)

University of Edinburgh and Biodiversity Observatory (OU)

Wildlife Ranger's picture

i-spot effort

Noticed I have accrued a non assigned category in My profile counting agreements given and received separately since the i-spot troubles does that mean they willl be lost ??


Whats Happening with Nature ??? Visit the Nature Blog

Supporting FEET Conservation work & Biodiversity Recording

lavateraguy's picture

I see that uncategorised agreements given and recieved ...

... are tallied in the profile, but uncategorised observations and identifications apparently aren't.

I can't predict with surety what the iSpot team will do, but this can be cleaned up with a batch job to fill in the missing categories from the taxonomic data (which they already have the code to do, under Get Recommended), and then rebuild the user profile data from the observations/identifications/agreements data.

Wildlife Ranger's picture

Thanks for that sounds like

Thanks for that sounds like more effort than its worth

Best Wishes


Whats Happening with Nature ??? Visit the Nature Blog

Supporting FEET Conservation work & Biodiversity Recording

landgirl's picture


Just started working properly for me in the last few minutes. Must have been something serious. Let's hope it lasts - I was getting withdrawal symptoms!

David Trevan's picture

Hooray as well!

I was also getting withdrawal symptoms!

David J Trevan

fisharebest's picture

We are working on it - honest!

Hi all,

I am one of the iSpot developers, and am spending this week (and possibly next) looking at performance issues.

I'm not an official spokesperson, but I do appreciate that it's not nice being left in the dark, so here's a brief update.

It's a busy (and growing) site, with new features being added all the time - unsurprisingly, the web/database servers are reaching capacity.

The good news is that there are lots of things that can be done to speed things up, and I've already started with some "easy wins" - server configuration, SQL tuning, etc. These are all small improvements, but are things that can be implemented without lengthy testing/analysis.

Collectivley, they seem to be helping.

After that, I'll look at some more major changes.

So, if you've seen improvements this week (landgirl, Thistle), then thanks for reporting this. You should hopefully see further improvements over the coming days....

Thistle's picture

Many thanks

It's good to know that the problems are recognised and are being worked on: make it a lot less frustrating from this end.

A huge improvement over the last couple of days (and I suspect that this will be reflected in the number of posts being made.)

Thanks for keeping us informed and keep up the good work.


landgirl's picture

Yes, thanks indeed

I'm sure we all appreciate being kept up to date - thanks for taking the time to reply.

marksteer's picture

Thanks for update.Also

Thanks for update - "fisharebest".
Also received answer to email advising response has been posted at:

Certainly things are looking much better this afternoon.

The more I know the more I realise I don't know

lavateraguy's picture

I think that you should roll back (some of) these changes ...

... as what seems to have happened is that we've swapped performance problems for broken functionality, and I think that the cure is worse than the disease.

marksteer's picture

Agree totally! Looks like

Agree totally! Looks like site has gone from bad to worse.
Not posting new observations or id's/ comments until this is sorted.
Doesn't look like new Observations are being included at the moment.
So frustrating!!!!!

The more I know the more I realise I don't know

lavateraguy's picture

What is currently happening ...

... is that new observations are showing up on the global latest observations carousel, but not on the group specific ones. As they're not showing up on my unread carousel, which I've set up to display plants only, I infer that the group field is at the moment not getting into the database.

Observations cycle out of the global carousel rapidly, so they are going to be missed if that is the only way they can be seen. (But presumably one could turn off filtering, but that would ones unread carousel.)

Martin Harvey's picture


Hi everyone, I've posted an update in a new thread at:

Probably best if we stop adding to this older thread as it has become rather difficult to follow.

Entomologist and biological recorder