A bit of analysis

Yesterday, in a post on the UK Hoverflies Facebook Page I tried to make some sense of what is happening to hoverfly populations this year. It generated several useful comments that got me thinking further. In that post, some basic graphs were presented in which the general differences between the datasets for the period 1 July to 13 August in 2015 and 2016 were illustrated (Figures 1-4). My initial thoughts (anecdotally) were that recorder effort had changed, with much more data coming from a smaller core of very active recorders and that whilst recorder effort had risen, the numbers of species per recorder had dropped. My interpretation of this was that the data lent some credence to the belief that hoverfly numbers were substantially down in 2016.

Figure 1. Basic data for 1 July to 13 August 2015 showing total numbers of daily records, recorders and species recorded.

Figure 2. Basic data for 1 July to 13 August 2016 showing total numbers of daily records, recorders and species recorded.

Figure 3. Comparisons of numbers of records per recorder in the periods 1 July to 13 August in 2015

Figure 4. Comparisons of numbers of records per recorder in the periods 1 July to 13 August in 2016

It was pointed out in comments that this might not be the case and that the data could simply be skewed by the numbers of new recruits who only posted shots of the commonest species. This morning, I therefore took a further look at data for the same period (plus one day). The results were illuminating (Table 1)!

	2014	2015	2016
Total records	2974	4843	5891
Numbers of recorders each year	653	539	456
Top 10 recorder contributions each year	809 = 27.2%	1311 = 27.07%	2049 = 34.78%
Top 20 recorder contributions each year	1053 = 35.4%	1927 = 39.79%	2899 = 49.21%

  Table 1. Breakdown of photographic data for the period 01 July to 14 August in successive years 2014 to 2016

The data suggest that there has actually been a drop in recorder activity, but that those that are active have contributed vastly more records. To some extent I think the latter is true. There is a very nice group of regular contributors who now record almost daily from their garden or wildlife patch. This is the essence of the Garden Hoverfly Recording Scheme that Alan Stubbs pioneered 25 years ago. It is precisely what Stuart and I have been hoping to generate. The data are certainly skewed by some changes in recorder activity. For example, one major recorder in 2014 changed to a slightly different technique and although we have his data on a separate spreadsheet they don't appear in this analysis. Were they to be included, the trend would be greatly more pronounced. In addition, several other very active recorders in 2015 have switched to maintaining their own spreadsheets for submission monthly or at the end of the year. Again, their data would accentuate the change that has happened.

So, what has happened to recorder activity? This is best summed up by looking at the composition of the top 30 recorders within the photographic dataset that I have used. The recruitment/loss process is shown in Table 2. It is greatly encouraging and bodes well for the future of hoverfly recording!

	Number of recorders
Contributing all three years	15
Contributing in 2015 & 2016	12
Contributing in 2014 only	1
Contributing in 2016 only	2

  Table 2. History of the top 30 recorder contributions to the photographic dataset between 2014 and 2014

Presenting records via Facebook

Use of the UK Hoverflies Facebook group as a way of getting photos checked has grown quite remarkably. It is a great way of facilitating interactions between observers and the Recording Scheme and is now generating a phenomenal number of records: there were over 4,000 gathered in July from web-based sources, with the majority coming from this group. Obviously such large numbers mean that data comes in all sorts of formats and there is quite a big job turning FB posts into a spreadsheet, so I thought some guidance on record/photo submission might be helpful.

My spreadsheet for web-based data has the following headings:

Column header	Notes
Species name	I record full identifications and partial identifications in two different spreadsheets. The first comprises those records that can be taken fully to species or to an aggregate name where there has been a split and the Resident Team refer to the aggregated name. For example, if we cannot put a firm name to a specimen that would once have been recorded as Xanthogramma pedissequum I record this as Xanthogramma pedissequum s.l. If we can only get to Genus or Tribe, then these go on a separate sheet.
Date	This is arranged dd/mm/yyyy e.g. 31/07/2016; if we cannot get a full date then it will be recorded as either 'July 2016 or as '2016. The latter two options are far from ideal but can be used.
Grid reference	Ideally a four or six-figure OS grid reference. A lot of grid reference finders take the resolution to the 1 metre level i.e. 10-figure level which in most cases is probably false accuracy if we move around to get shots and records. Nevertheless, if that is what the recorder has used then it is logged.
Location name	If a location name has not been supplied then I look the site up on Streetmap and find a logical name if I can.
Recorder	This is the name of the person who makes the post. Where I know that people use various aliases then I include these too (some people have up to four aliases).
Determiner	If a photo is published then I use my own name unless I feel it appropriate to defer to somebody else - either Ian Andrews or Joan Childs, or occasionally Gerard Pennards. Just occasionally I use the recorder's name when he/she has posted an awkward angled shot of an obvious species and almost certainly knows what they saw. I run a separate spreadsheet for records that come in without a photo to back them up and in these cases the determiner is the name of the recorder unless there are good reasons to use another name.
Stage	Essentially male/female/larva but occasionally it is not possible to determine gender and in these cases the annotation is 'adult'. I use 'adult' for pairs in cop and make a note in the notes section that this was a pair in cop. Likewise where there is a nuptial stack of Eristalis nemorum I note the numbers of adults in the abundance column. I run separate lines for males and females, except where there are mating pairs or nuptial stacks.
Abundance	This is as best as I can do to determine how many individuals of the gender reported or photographed.
Source	This is the URL for the Facebook post or for the iSpot or Flickr post. If several posts depict animals from the same location on the same day then I add additional URLs separated by &
Notes	Here I log observations by recorders and items such as flower visit records. I do not log the leaves an animal was on and make clear that the flower involved (if visited) was a potential pollination event by saying 'at x' or 'visiting  y'. This column is useful for behavioural notes. Where I am unsure of the plant associated with the hoverfly I usually put 'at ???' or 'at ?Sweet Pea' or 'at umbellifer'

What is the best format for a post?

I usually copy and paste details from the FB post, so there are things that can be done to help.

A really nice example of presentation might be along the lines of:

I had an excellent day in the garden with lots of hoverflies. Here are my photos:

Episyrphus balteatus (2 photos, same specimen); Eristalis arbustorum at hogweed; Eristalis pertinax; Eristalis tenax (2 photos of separate animals); Platycheirus albimanus male and female. both visiting vipers bugloss

Not photographed:

Syritta pipiens 1 male; Volucella zonaria 1female at buddleia

Observation details:

Queen Elizabeth II Reservoir, West Molesey

TQ123678

09/08/2016

The above example is fine - I can cut and paste the basic site details straight into the spreadsheet. I usually then copy the species list into Word and run find and replace to turn it into a list with tabs to separate numbers and observations - the above would then look like this:

Episyrphus balteatus	1	adult	2 photos, same specimen
Eristalis arbustorum	1	adult	at hogweed
Eristalis pertinax	1	adult
Eristalis tenax	1	adult	2 photos of separate animals
Platycheirus albimanus	1	male	visiting vipers bugloss
Platycheirus albimanus	1	female	visiting vipers bugloss

I can then cut and paste each column into the spreadsheet

Some hints on what not to do

Species names

It helps not to abbreviate the generic name - I simply have to type this so it is not a huge problem, but it does add to the job.

Grid references:

One of the biggest problems we have is grid references. Stuart has found that the advent of GPS has exacerbated rather than resolved this problem. Somewhere between 5 and 10% of all records have some sort of grid reference problem.

The worst I have to deal with are lat long data, which people submit in all sorts of permutations - they can take a fair while to sort out and make sure that the correct OS reference has been determined.

The other regular problems are:

·         people leaving off the 100km letters (e.g. TQ)

·         people using lower case - I turn these into upper case to make sure that the spreadsheet is uniform and simple to read. It is a small niggle but does add to the time involved in creating a complete record.

·         Separating the grid reference using commas, lines etc e.g. TQ28/68 or TQ28,67 etc - I have to remove these. I regularly run a find and replace to clean all of these sorts of glitches from the data.

·         people who place the location out to sea, either accidentally or deliberately. We do get the occasional marine record from oil rigs and ships, but these are usually noted in the post.

I should also say that it does help to separate records from different sites onto different posts - I am a 'bear of small brain' and having a post with records from several sites does cause confusion and transcription errors.

What do the resulting data look like? 

Example section of photographic spreadsheet for 2016.

 

The challenge of identifying from photographs

After my last post, which generated a bit of additional comment, I thought I really ought to go back and look at the data to try to explain the challenges faced by specialists working on photographic identification. As a starting point I thought it might be helpful to look at the genus Syrphus and what causes the problems:

The first and most significant challenge is that we don't actually know how many species there are in Britain! Some years ago, Syrphus rectus was added to the UK fauna. This is a North American species that may well be holarctic, but we don't know. Males are almost identical to Syrphus vitripennis, especially in the distribution of microtrichia on the wings. Females are an intermediate between S. vitripennis and S. ribesii, in so far that they have the microtrichia of vitripennis, but the hind femur has a dark ring on it. So, maybe the females are doable from photographs? Field photographs never generate the necessary definition to pick up the microtrichia so unless one can detect eye hairs (S. torvus) or the wholly yellow femur of S. ribesii there is considerable uncertainty about the identity of most animals. In theory, there are additional characters of hair colour and extent of yellow on the legs that can help in some circumstances, but these are highly variable features and can be influenced by the angle one views the animal. So, Syrphus is already a problem and that is compounded by uncertainty. Further difficulties arise because some very high quality photographs have shown that even S. ribesii posesses minute hairs on the eyes, and that in some situations these can be mistaken for the hairs of torvus (demonstrated in females with yellow hind femur).

The following photos may help to explain. These are high resolution stackshots taken by John Bridges (North East Wildlife northeastwildlife.co.uk/gallery/index.php). John has been doing some really interesting work this year compiling detailed shots of hovers and here are some of the results (many thanks for permission to use John).

Syrphus torvus female, showing the eye hairs. The hairs of female eyes are far less obvious than those of males, but this shot nicely shows how small they are and how fine the resolution of photos needs to be to resolve identification.

Syrphus ribesii male with the second basal cell showing complete coverage by microtrichia. These minute hairs are only seen when the animal is carefully orientated to the light.

Syrphus vitripennis second basal cell with area devoid of microtrichia highlighted (area above the green line)

So, what can we do? Well, the problem does partially resolve itself if the photos are high quality macro shots such as those taken by Brian Valentine (LordV on Flickr). Even so, getting the right angles is tricky and one has to work with what is presented. The other otion is perhaps to retain specimens, anaethetise them and use stackshot to take more detailed photos. That is quite an undertaking and is likely to be beyond the average photographer. So, we must accept that at least some species are rarely likely to be taken to a full ID unless one captures all the salient features. This is demonstrated by the table I have attached later in this account where the problem areas are highlighted.

Challenges such as this can be found in many genera, but even so, there are plenty of hoverflies that can be identified from photographs providing the photo is of sufficiently good resolution and is sharp. The smaller the animal is in the frame when photographed, the less likely it is that it will be possible to achieve a firm identification. This is illustrated in the table below. I have compared the data for 2015 and 2016 so that as a big a sample as possible is considered. Clearly there are year on year differences in success rates. In many cases there are very minor differences between years, but for some the differences are quite substantial - something I will have to look at when I have time.

Success rate for identification of photographs in 2015 and 2015 at generic level. Genera where major problems arise are highlighted in orange.

It should not be assumed that 100% success rates actually imply that identification is easy. In many cases, the numbers of species in a genus are small, or the frequency with which a species/genus is encountered is very low and the sample size is too small to make any realistic assessment. Furthermore, there may be records within the uncertain genus category that might still not be taken to species.