Changes in data sources - a timely reminder

This morning I awoke to an interesting post from John Bridges on the UK Hoverflies Facebook page; he had been going back through my blog and picked out an article from 2012 in which I was extolling the possible benefits of photographic records to biological recording. To quote John, quoting me:

• For several years, I have been trawling the internet for photographs of hoverflies ----- So far I have extracted almost 9,500 hoverfly records ….............'That's about a weeks worth now.'

• During the mid-summer months, this trawl can take as much as an hour a day...............'I bet he wishes it still did?'

• Over 2700 records of hoverflies have been extracted for 2012 …................. 'Waiting for him after breakfast on a morning now'

• Trawling the internet allows me to undertake biological recording whilst maintaining my presence in the office................'How much spare time now Roger?'

That was just 5 years ago and it is a timely reminder of what has happened in just a few years. It is also something that those who use but don't generate biological records might reflect upon. In 2012 I saw photographic recording as an adjunct to the main source of records and simply a way of increasing my own productivity without disproportionate negative impacts on me! Other recording scheme organisers were more wary (perhaps wisely so) - not wanting their lives to be ruled by the internet and their recording scheme.

John's estimates of the numbers are a little bit out, but the overall thrust of his analysis is correct. For example, 9,500 records equates to about seven weeks worth of data extracted in mid-summer and roughly 25% of the total for 2016. During the summer months I dare not be away from the computer for more than a couple of hours - I do an hour or two first thing in the morning and then log back on at lunch-time (if at home) but definitely by 5pm at the latest - by which time I have a good two hour's of work to do. If I fail to do this the evening is a nightmare so it is a critical marker in my day's work. Then, after supper I often spend the rest of the night extracting data - during mid-summer posts are still coming in at 1am and there are times when they come in like a machine-gun! 2016 was the turning point, and had help not arrived I would probably have been a gibbering wreck by now.

Fortunately, the Cavalry arrived just in time - Ian Andrews and Geoff Wilkinson now help to spread the load, and there are other potential offers of help. The other huge help has been those FB members who now maintain their own spreadsheets. This is a massive improvement because it means that a wider community of members can help with ID (thus growing their skills and spreading the load on me, Ian, Joan and Geoff). I still look at every post, and will still comment on those that need one - I think it is essential that people who have taken the trouble to post get an acknowledgement at least from me (a 'like' at least).

The other thing my 2012 post did not take into account was iRecord. It did not exist at the time, but is also another part of the job today. Last year there were about 6,000 records that took several full days to work through - there is a post somewhere giving times and some basic statistics. This year there are about 10,000 to work through. This form of data submission is growing at a considerable rate and it is likely to require further administrative changes to keep on top of it.

Meanwhile, several recorders now maintain spreadsheets and also send me specimens to check (sorry I am behind on this job folks). I think this is a great way of recording because it broadens the range of species covered, but of course it is another layer of demand on the recording scheme. Meanwhile, spreadsheets arrive at an increasing pace - this year has been the busiest yet. I do the basic checking and formatting before they go to Stuart for importing into the database. Both jobs take a while, but I am pleased to say that we are pretty much up to date (I think there are 5 datasets on the Dropbox awaiting Stuart's attention at the moment).

So, would I go back to the 2012 scenario?

Emphatically not (although I would like my life back)!

The growth in popularity of Facebook Groups (not just Hoverflies) has seen an unprecedented leap in the numbers of people contributing records and acquiring skills. We can produce data for the HRS to show its impact, but I wonder how many other schemes are in a position to do so? That growth will hopefully translate into the development of new people who will ultimately take over the running of the popular schemes (we must think about the very long-term).

Equally importantly, photographic recording has actually started to help to unravel some aspects of species' ecology that we would not otherwise have had. True, it is also creating a skew in the data that I have written about, but we can deal with that provided we still have a pool of traditional recorders who retain specimens for critical checking.

What I would like to see is a further shift towards people managing their own datasets and using the Facebook pages to deal with cases where they are not sure. That will doubtless happen over time and, of course, it needs to because the time of the 'Resident Team' is finite and we really must try to devote it to developing newcomers and embedding skills in the wider community. John, himself, shifted towards managing his own dataset a long while ago - and has contributed a huge number of records this way. In only three years he has become one of the all-time major contributors (there are several others who will do the same). We would not have had this improvement in recording without the advent of digital photography, the internet and social media. So, whilst there has undoubtedly been a massive change to my life, it is largely positive because so many more people get pleasure from hoverflies, are learning new skills and are forming a community that will hopefully last long after I am gone.