This blog is intended as an occasional diary of information to feed back to hoverfly recorders in the UK and elsewhere. Inevitably there will be issues of interest that are in some way relevant to invertebrate ecologists and consequently I intend to use the medium as an opportunity to develop thoughts on pertinent topics.
Friday, 18 November 2016
Has iRecord raised expectations of what recording schemes can achieve?
A thread on the NFBR Facebook page raised the very important
issue of the 'contract' between recording schemes and contributors. If people
place data on iRecord then they expect the data to be checked and verified so
that it forms part of the national dataset. That is not an unreasonable
position to take. Equally, the bodies that funded and continue to support
iRecord have a similar 'contract' with recording schemes. They have stumped up
the cash to create a system that captures a very wide range of ad-hoc records
without the need for recorders to send data to a multitude of recording
schemes. Therefore, if the recording schemes want the data they have an
obligation to attend to the data and to follow the verification process. That
does not mean that all records will be accepted without challenge but the data
should be looked at.
Unfortunately, there is a problem! Vast numbers of records
are being submitted to iRecord but are not being verified. Why is this? Well,
the sad fact is that a large body of recording schemes have yet to sign up
to the verification process, and as they delay the job gets bigger and becomes
an almost insurmountable problem. I will readily confess to being a serial
offender because it can take a long while before I attend to iRecord. This year
people had to wait for about 8 months before I got round to the verification
process. I simply did not have enough hours in the day to deal with iRecord as
well as the wider range of jobs that are associated with running a recording
scheme. I expect there were some who felt let down - I apologise for this. I've
still got about 1,500 records to deal with but have at least cleared about
4,500 records. Hopefully the job will be done by December.
This brings me to the nub of the problem. Running a
recording scheme has changed out of all proportion in the past ten years as a
result of the digital revolution. Fast computers and (relatively) cheap digital
imaging has allowed a huge number of people to participate in recording where
they would not have done so previously. This technological revolution has also
raised the expectations of bodies that draw upon biological records: statutory
agencies, NGOs and, increasingly, the Academic World. The idea that 'Citizen
Scientists' are a vast untapped source of information is now firmly embedded in
the psyche of such bodies as well as politician who see this as a way of
generating much-needed data very cheaply.
Looking to the past
When Stuart Ball and I took on the Hoverfly Recording Scheme
in 1991 it had been moribund for 4 years. Philip Entwistle retired in 1987 and
at the same time gave up running the scheme. We did not start by volunteering:
we both worked for the Invertebrate Site Register (NCC) and our boss was Alan
Stubbs - Alan approached us to see if we would take the job on. It was a
potentially enormous task because the data was substantially card-based and
there were insufficient funds to pay for the data to be digitised and checked.
It would only happen if there were volunteers who were prepared to put in the
time. Stuart concentrated on gathering available machine-readable data, whilst
I took on the 2 cubic metres of record cards.
It took 5 years, with me often working 12 hour shifts at
weekend to plough through the cards. Fortunately, the work could be done in the
winter, so it did not impinge heavily on the summer months when natural
historians like to get out. Figure 1 (below) shows how the data has grown. It paints an important picture: The period up until 1997 saw the backlog cleared. Between 1997 and 2005 the scheme was almost moribund - Stuart and I were involved in many other things and took our foot off the gas. In 2005 we decided we had to push a new project (Atlas 2010) in order to revitalise the scheme - that prompted the arrival of a lot of data from preceding years, hence the jump in records enteres in 2005 and 2006. The biggest spike was in 2011 when we finally drew in all the data and published the second provisional atlas. From 2012 onwards, the incoming data has seen a new spike, which is the impact of the UK Hoverflies Facebook group. I would expect data entered to climb further in late 2016 and early 2017, as the level of recording in 2016 has been phenomenal.
Figure 1. Data seembly and growth of the HRS dataset since 1991.
The composition of the data has also changed. This is illustrated in Figure 2 below. This clearly illustrates how data sources have changed. Traditional record cards have all-but disappeared (we have just one recorder who sends cards I then turn into a spreadsheet for Stuart to upload). The shift was towards machine-readable formats that are relatively easy to verify but do take time uploading into the database, especially when there are lots of grid reference errors. Record Cleaner has helped this process considerably. But the change towards photographic records is also apparent. The graph clearly shows how photography has started to change the data management process. It covers the period to 2015, but will look very different for 2016 because I anticipate about 30,000 records from photographs before we absorb iRecord data! So the shift over 25 years has been from time-consuming data entry in the winter to a substantially bigger effort to secure data in the summer. Of course that leaves the winter for providing feedback to recorders, so there is an upside in the time released from winter activities.
Figure 2. Sources of data
The sheer volume of data being generated by initiatives to
increase public engagement in biological recording is amazing. In the case of
the HRS it has resulted in the data flow doubling in the course of 5 years. I
suspect the same holds for other schemes and some such as moths will have seen
an explosion of data in line with the vast numbers of people who now run a moth
trap in their gardens. Has the number of people who are willing to take on the
task of verification and recording scheme administration also grown? Well,
maybe, but a lag between the increase in activity and the generation of
suitably skilled people is inevitable.
The answer is obviously to try to increase the numbers of
people taking on the administration of biological records. It sounds simple,
but then as the numbers of people involved increase there is a need for
co-ordination and development of an administrative structure to make sure that
data are maintained to a common (high) standard. If there are doubts about data
quality any research outputs that point to problems in environmental policy
will be dissected and easily dismissed by those who might be affected by any
policy change. So, the bigger recording schemes are now grappling with data
management issues. Inevitably, they are having to develop administrative superstructures
that take people away from what they signed up to do and into the role of administrators,
mentors and computer jockeys.
On the face of it, running a recording scheme sounds great
and there will be people who will rise to the challenge, but there will be many
others who are highly competent natural historians but who don't want to become
administrators. The big question is: are there sufficient people to take on
these roles? We must wait and see, but when one hears of County Recorders
retiring with no replacement lined up, there is a serious issue. I have heard
that the YNU now lacks a complete compliment of County Recorders. If that
situation obtains in an organisation that has a longstanding reputation of
commitment to biological recording, one needs to sit up and listen to the
rumblings. Likewise, if large tracts of iRecord are also going unvalidated, or
recorders are retiring without a replacement, they too are hinting at a
There are many initiatives to improve the situation, but in
my experience if you run a training course or mentoring programme you will
probably only secure long-term commitment from about 1 in 10 people who
participate. For some, it is part of a broader journey of discovery, others
will decide it is not for them; there will also be those who are interested but
realise that the requirements of the job are too great for the time they can
Thus, is my glass half full or half empty? I think we must
take comfort from the positive elements of the changing paradigm. Far more
people are engaged and interested enough to contribute to iRecord. BUT, it will
be readily apparent to those more active participants that the role of a
recording scheme organiser is potentially very demanding and perhaps something
best avoided. We must hope that by broadening the net sufficiently, there will
be people who see an opportunity that can be turned into a positive experience.
It is down to the schemes to try to secure that engagement.