Saturday, 29 April 2017

Verification of records

Over the last couple of days, there have been busy threads on both the BWARS and NFBR Facebook groups concerning record verification. It all started with a question concerning the distribution of Anthophora plumipes, a relatively easily recognised solitary bee that flies in early spring and whose distribution is well-known.

The map that was produced on the NBN (Figure 1) showed vastly wider distribution than is shown in reliable data compiled by BWARS (Figure 2) but not currently available through the NBN. The difference is obvious! The data in question was compiled by the ‘Great British Bee Count’ run by ‘Friends of the Earth’. In essence, the data are junk as they stand! Sadly, an awful lot of well-meaning people have been encouraged to participate in what seems to me to be little more than a publicity stunt. No thought seems to have been given to data verification or to the impact poor data can have on the work and outputs of long-established biological recording schemes.

Figure 1. Distribution of Anthophora plumipes according to data collected by the 'Great British Bee Count'

Figure 2. Distribution of Anthophora plumipes based on verified data compiled by BWARS.

In fairness to FoE, they do seem to have recognised the problem and I believe have linked up with Buglife to do something about it. I was recently contacted by someone at Buglife to seek my views on whether the project should extend into hoverflies and whether I would be willing to verify the data. I said NO on both counts. Why? Surely I should be getting involved?

My rationale is simple. The Great British Bee Count swamped BWARS with utterly unreliable data and they were neither able nor willing to take on the job of verification; I don’t blame them as it is not the simple job people sometimes think. It is not just a question of getting a specialist to sit down and check a few photos; it is weeks or possibly months of work that is tedious and frustrating. Also, is it making best use of skills developed over several years or, in most cases, tens of years? My answer is emphatically NO.

I seem to recall that FoE’s rationale for starting the Great British Bee Count was that there was inadequate data on bee distribution and that it needed more effort from the general public. That was pretty naive. The issue should not start with data availability, although it is fair to say that coverage of most invertebrate taxa is much poorer than for vascular plants, birds or mammals. The big issue starts with the complexities of identification and the skills needed to become competent with exceedingly difficult identification. Acquiring these skills take time and patience. I spent maybe a decade doing aculeate Hymenoptera, and still do the odd few specimens. I don’t consider myself an ‘expert’ but can make a reasonable job of separating out the majority of regularly encountered species when I sit down for a couple of days and work through a block of specimens within a single genus. Likewise, I now feel I can cope with Hoverfly identification from photos, but it has taken me ten years to reach that point (and I am still learning).

If we want bigger datasets, the starting point has therefore GOT to be growing skills. It is a very slow process but is the best use of specialist time if one looks at a long-term strategy to improve our knowledge. That is why the HRS has been running training courses for a decade or more. Thanks to OPAL grants we can take the courses to the places where they are needed, and we do so regularly. Even so, I reckon that at best we convert 5% of the people who do our courses into serious recorders; and of that cohort, probably only 10% will go on to have the necessary expertise to take on the task of data verification. For most it is a hobby and one that has to fit in amongst a plethora of other activities and responsibilities. It is the rare individual who can devote time to developing the skills that are needed to take on the challenging taks of ID from photographs and data verification.

So, the message is clear. If we want more data we have got to engage constructively with people who want to learn. FoE’s initiative will generate a lot of interest, and hopefully it will get a new generation of young people sufficiently enthused to take up the net and pooter, microscope and keys. That is the real benefit of the initiative; not the generation of a block of data that has already wasted a certain amount of specialist time dealing with the ensuing kerfuffle when the data they publish are so obviously inaccurate.

On a broader theme, it is also worth reflecting that there is a great deal of naivety about the potential of 'Citizen Science' to solve shortfalls in data. The Great British Bee Count has been helpful in showing the pitfalls of badly designed initiatives and  the need for researchers to be very careful about the datasets that can be relied upon. It also shows why recording scheme organisers have to be vigilant in evaluating those records that they receive. There are lots of tests that can help to verify a record, but in the end the only sure way is to examine a specimen on a pin and place that specimen in a suitable repository for re-evaluation as and when necessary!

Monday, 17 April 2017

An early spring?

Each spring, observers often remark upon whether plants and animals are emerging earlier or later than previous years. In the last 30 years, the general impression has been that springs are getting earlier and this impression is reinforced by the data. Amongst the hoverflies, there are several whose emergence has shifted by as much as two or three weeks, with some emerging at crazily early dates. But, does the first reported emergence actually tell us very much?
In reality, a one-off event is inconsequential; it is far more important to look at the overall phenology of a species or a group of organisms. And, when one looks at phenology, it is not the first and last dates that are important, it is the degree to which peak numbers shift that tells the full story. Thus, analysts get very frustrated when recorders say ‘I’ll give you first and last dates but I cannot be bothered with anything else. Without the supporting context, first and last dates are utterly meaningless.

Making sense of the data

Monitoring photographic data compiled by recorders who are relatively unselective is a great way of developing data on readily recognisable and useful indicators of seasonal change. The Hoverfly Recording Scheme has been doing this for around ten years, but it is only since the advent of the UK Hoverflies Facebook group that the volume of data has reached a level where the data are sufficiently robust to look at differences early in the year. In the past, one would have had to wait for a year or more for relevant data to arrive. Now, we have the data almost immediately to hand and can start to interpret the impressions of observers almost on a ‘real-time’ basis.
This year, the overall impression has been that spring got going very early. Was that really the case? I thought it was worth looking at a suite of indicator species to find out. Initially I compiled a long list of species that looked to have emerged earlier than usual. This was rapidly whittled down to just three species because many of the potentially early species are reported in relatively low numbers. They are not really very useful because the reports depend entirely upon chance. Records of widespread and abundant species provide a much more solid basis for analysis because many more people will see and report them.


For this analysis I took three species that fit my criteria of being abundant, easily recognisable and widely reported. They are: Epistrophe eligans, Leucozona lucorum and Dasysyrphus albostriatus.
As 2017 has only just started, the median date for these species cannot be calculated. The median date for early emergence can, however. So, I compiled a table of the first three dates for each species in 2014 to 2017. From this, I calculated the median date for each year for each species and then ranked them according to date of median emergence (Figure 1). This initial analysis shows that the early emergence dates for 2017 are indeed earlier than in previous years, with two out of three species ranking first and the other lying second in the rankings.
Figure 1. Tabulation of first two third dates for three early hoverflies

Can one get any other ideas on the degree to which this year is early when compared with previous years? My answer to this was to look at the spread of dates for each year, taking the median dates for the three species and creating a second median. This is probably statistically wrong, but as a crude analysis it helps to paint a picture (Figure 2). Taking these dates, a further median can be created between the earliest and latest medians of the combination of three species. This date is 31 March. One can look at the degree to which each combined median varies from the central date (Figure 3) which suggests that 2017 is possibly as much as 7 days earlier than the median for the previous 3 years.
Figure 2. Median dates for the combination of species over the period 2014 to 2017

Figure 3. Variation in median dates from the median for the period 2014 to 2017


Is this believable? Time will tell, but my general impression is that the species lists for 2017 contain animals that would not have been seen for at least two weeks further into the season even within the last ten years. If one compares with 20 years ago, the evidence is very strong that hoverfly emergence has advanced by several weeks and that the field season is getting longer. In some cases, it is likely that unless you get out early, some short-lived species will have come and gone before you have mobilised!

Tuesday, 3 January 2017

Hoverfly identification: getting started with the WILDGuide

When I was young and got a new identification guide (in my day the Wayside and Woodland series and the Observer's books) I would avidly read the entire contents and would then seek out the animals I was particularly interested in. I well-remember making my first discovery of Puss Moth eggs when I was nine or ten years old - just as the book said, laid in pairs on the underside of poplar leaves! I have always assumed that other field naturalists did much the same, but I suspect not. My suspicions were reinforced a year or so ago, when after a lecture in London, one of the questions that came from the floor was (slightly paraphrased):

'I have the WILDGuide and it is rubbish; can you suggest a book that can actually get me to species - one with keys'?

My reaction was that the WILDGuide did contain keys and that the questioner should 'read the b....y book'! It got me thinking, however: we went to a lot of trouble to try to design a key to get readers to the Tribes, but do they use it? I guess some will and some will not. The important point is that the guide was not designed like the classic bird guide with plates of pictures that you simply match to the bird you have seen. I say 'simply', but from experience making a diagnosis of birds is far from easy without a respectable knowledge of the critical features and what to look for. The same holds for hoverflies and, indeed, most animals and plants. If you simply start at the beginning of the pictures and try to make a match, it is possible to go wildly wrong!

So, where should one start?

The obvious place is 'at the beginning'. Arriving at a firm identification for any group of organisms depends upon one's knowledge of their anatomy and morphology. The anatomical elements can be learned from text books, but it does have to be applied. There are subtle differences in the way anatomical features are expressed in different animals, even within the same family, so it take a bit of time. Which is where comparative anatomy and morphology come into the equation. Taxonomists rely on comparative anatomy to make their diagnoses and tend to rely on interpretations of anatomical evolution to ensure that they are describing the same feature in a range of animals. In flies that is horribly complicated because the evolution of the wing veins is complex and somewhat uncertain in places (try working out the original origins of the veins in the Phoridae for example!).

When we run training courses, we put a lot of emphasis on making sure that the group understands critical anatomical features. If you don't understand these, the keys are a nightmare. Similarly, if you don't understand what separates the Syrphini from the Bacchini, or the Cheilosini from the Chrysogastrini, life can be pretty complicated. So, if you don't have a chance to go on a course, you can teach yourself to some extent and that will make life a lot easier. Some features (many) are microscopic, and thus it is a lot easier learning from microscope and specimens, but modern photography is remarkable and it is amazing what can be done from a good set of photos. So, for those just starting out this winter, I suggest bringing up photos from the Facebook Group and seeing if you can run them through the key to Tribes.

It is all about markers

Using a key gets easier with time. Once you know how to find a character, it is a whole lot easier to establish its form. Does it have eye hairs, is it peculiarly shaped or is it a particular colour pattern? But, beware, there is always the exception that causes a major headache. The Hoverfly WILDGuide  is a half-way house because it was intended as an introductory guide that compliments the more comprehensive monographs. Although there are keys, it is true that the book is not based on keys. I often find it useful to go back to Stubbs & Falk and actially verbally run through the sequence of characters to help FB members find their way around difficult genera. That said, one has to be careful not to do this too much otherwise sales of the main monograph may be compromised.

In hoverflies, the biggest problems I think are:

  • Teneral specimens in which the colours are not fully formed;
  • Ina ccessibility of some critical features such as elements within the genital capsule;
  • Specimens that develop under exceptionally warm or cold conditions, whose patterns are greatly affected by temperature;
  • Species with spring and summer broods and exhibit brood dimorphism; and
  • Misinterpretation of dusting because of glare or rubbing.

All of these factors combine to make life difficult, so there is no real substitute for experience. The more you see, the more likely you are to have seen some of the problems and know what they mean. There is no rule that says every specimen can be taken to species. Even the most experienced taxonomist will have a small selection of specimens that they are unsure about. As time goes by I find myself getting increasingly uncertain about some genera! If in doubt, put it to one side and try again later, or seek an opinion from a recognised specialist. Beware, there are people who will give a diagnosis based on spurious characters. It is best to stick with reliable techniques because taxonomists don't deliberately set out to make things complicated;  things get complicated as more and more specimens emerge that don't fit the conceptual model of the characters that define the species.

Nevertheless, as time goes by, you will start to know the critical markers. For example, one of the most useful ones is the loop in vein R4+5 that defines the Eristalini and the genus Merodon. Episyrphus balteatus should become familiar quite rapidly (but dark and pale forms may still confuse). Fortunately, if you start recording in the early spring, you will be eased into the bewildering array of forms and will hopefully acquire markers from those early species.

It does not happen with the click of a finger!

When I started working on hoverflies I spent innumerable evenings poring over specimens under the microscope, muttering 'these are impossible'. Of course they were not impossible, and I have overcome the worst of the early challenges. I think it takes about 3 years to acquire a reasonable grounding in hoverfly identification, at which point you realise that there are innumerable species that you have not seen. At that point, it is time to go back to the texts and learn about the ecology of species that you have not found. In this way, you should get to see more species because you know what to look for and where to look.

Sunday, 25 December 2016

Developing a County Atlas

A County Atlas project is a very good way of generating local interest in any group of organisms, especially if there is an active group and a dyamic organiser. To my mind, the best model is that of the Surrey Wildlife Trust's County Atlases, which now form quite a remarkable canon of work. As a result, Surrey is probably the best documented county for its invertebrate fauna. A lot of credit should go to Martin Newman, the Chief Executive of the Trust who was responsible for turning into reality the basic model suggested by yours-truly and Graham Collins over lunch one day in Purbright in spring 1994 (I think).

This series of books has far-surpassed what I think any of us foresaw at the time. I hoped that we might manage to publish the work of several active recorders at the time – David Baldock, Roger Hawkins, Graham Collins and me. But, twenty years later the range of titles far exceeds the original aspirations of Butterflies, Moths, Dragonflies, Hoverflies, Shieldbugs and Orthoptera. There has even been a second edition of Butterflies that provides a chilling oversight of the decline of some of the county's iconic species. These are lasting records that form an important baseline for future generations upon which to base analysis of newer data.

I always hoped that Surrey would be in the vanguard and that its series would stimulate other counties to do something similar. There have been occasional volumes for a few groups in other counties, but to the best of my knowledge no other County Trust has attempted anything on the same scale; yet we have shown that it can be done!

How is it done?

In short, hard work! But the scale of the job depends upon the numbers of recorders. For some insect groups there should be no problem generating sufficient records. There are often good numbers of capable butterfly and dragonfly enthusiasts. Things become a little more difficult to achieve good coverage of moths but, again, there are far more people running moth traps that in the 1980s and 1990s. In theory, the same applies for hoverflies: the numbers of active recorders have grown substantially, but here we see that the available capacity is thinly spread. I dare say a similar situation obtains for bumblebees and some other Hymenoptera. So, for these groups, why not give it a go? Equally, there may be other relatively popular families – Longhorn Beetles, for example.

When Graham and I started on our atlases we set out to cover Surrey at 5km resolution, but over time that refined to tetrad level, which I think is about right. Any coarser resolution makes it difficult to relate species' distribution to solid and drift geology, or to urbanisation. Each weekend, in suitable weather, we would choose an area of Surrey and visit as many squares as possible, often stopping for just sufficient time to achieve the '80%' rule. In other words, you generally assemble 80% of the list relatively quickly, but can spend innumerable hours adding the occasional additional species. That is not terribly efficient, and so it is best to move on and cover several more squares and in doing so you increase the overall species list but make sure that those that are abundant on the day are recorded at the majority of sites. Repeat visits help to fill in the gaps and can be targetted to fill in obvious shortfalls. As I recall, it took about 10 years before we were convinced we had adequate data to produce an atlas. I would guess that about 75% of the data for hoverflies was generated by Graham and me.


When we were active in Surrey, it was much more difficult to distribute provisional maps. Now it can be done electronically and pdfs can be made available quite readily. Once working maps are distributed, people are often encouraged to fill in the gaps in their area. So, the obvious lesson is to produce a set of working maps and make them available to anyone who is interested.

Most people will not travel very far from home, but if they can be encouraged to fill in the squares in a radius of 5km from home you will soon start to see the benefits of developing a network. Then it is up to the project organiser to visit under-recorded areas and fill in the gaps. I used to check out particular geological formations to look for species that I suspected might be found. Obviously, a different assemblage is likely to be found in different situations, but there will also be surprises, so one should not be too presumptive about what might turn up. I also found that when certain species turned up, it was worth visiting similar habitats across the county over the next couple of days, as some species are very short-lived. This seemed to be the case with Brachyopa and with Myolepta as well as some Cheilosia.


The Surrey Atlas Project was structured to attract sponsorship for the first couple of titles, after which we were able to use income from preceding titles to fund later ones. Our design was pretty up-market, but simple soft covers using 'print on demand' models could be a lot cheaper and easier to update. One can design to fit the budget. It striikes me that maybe today there could be support from sources such as Lottery and Aggregates Levy funds. The important point about this part of the project is to try to do a bit more than just a set of maps. Interpretation is the critical issue. For younger readers, this part of the job is a great way of improving your ability as an ecologist if you have aspirations for a career in conservation and ecology.


My big regret about the Surrey Atalas series is that we did not have numbers on the spine. I did suggest this but it was not pursued. Had we had numbers I feel sure there would have been more sales as the series gained popularity and people started to collect the books. In this respect, I think it is important to remember that printing is expensive and that unit costs can be reduced by increasing the volume sold. But, if you print more you must sell more – storage is expensive and it is dead money, so the length of the print run is always a balancing act and marketing techniques are essential to make the project viable.

Saturday, 24 December 2016

An exercise in parataxonomy

Earlier this month, I discussed a new paradigm in parataxonomy. In this post I describe some of the data developed by John Bridges (NE Wildlife) who has combined his amazing photographic skills with equal dedication in trying to make sure that the subjects captured as photographs are correctly identified. John's approach is one that might be of interest to other photographers who want to build up a more complete picture of the hoverflies that inhabit their favoured wildlife site(s).

John's technique is to settle at a chosen location and to wait for the subject matter to come to him. It can be extremely successful, as John has shown that there is a constant turnover of flies at suitable lures. His daily species lists are remarkably close to the level of coverage I would expect to achieve using net and pooter.

In addition to photographing everything that attends the chosen lure (a flower or possibly sugar-sprayed leaves), John captures a selection of specimens and retains them for photo-stacking at home. Each specimen is logged against the number of the photograph that recorded it. In this way, it has been possible to develop a quite comprehensive set of photographs with a firm identification of the corresponding specimen.

Once the specimen has been run through the keys, it is bagged up and retained for me to check later in the year. I have just completed this year's sample and thought it worth sharing with a wider audience because it is highly instructive of what can be achieved. His species-list for the year is just shy of 100 species, which is impressive when you bear in mind that north-east England is relatively species-poor and that John records from quite a restricted range of habitats. The list also contains a number of noteworthy species, including one of fewer than 20 confirmed records of Eristalis similis and one of Platycheirus aurolateralis, which is rarely seen and is extremely difficult to separate from the commoner P. splendidus. The list of species represented in the collected sample is represented in Table 1.

Table 1. Species represented in samples retained after photographing.

In addition to generating reliable records, John is helping to build up a valuable portfolio of live animal photographs that should help taxonomists better understand how to describe live animals rather than using specimens. This may, in the longer-term, help to improve our ability to provide reliable diagnoses from photographs.

Storing specimens

John's system involves labelled polythene bags that take up relatively little space and can be stored until the autumn when they are sent to me for verification (figures 1 & 2).

Figure 1. Samples awaiting identification

Figure 2. Sample checking in progress

In theory, it is possible to identify some of the specimens through the bag, but in practice I find it easier to remove them from the bag. I dispose of the majority of specimens on the compost heap, so I remove all from their bags anyway. A few are retained where they are important vouchers - they will find their way into my storage system. Others are retained to become part of the teaching pack that Stuart and I use in training courses.

This system has obvious advantages in providing a clear way of retaining specimens without having to pin them and then label them. There are drawbacks, however. The main problem is mould and decomposition. Last year, John stored the bags in the freezer, which meant that once defrosted they were still relatively fresh and could be pinned. Unfortunately, a small proportion went mouldy for some reason. This year, the specimens were simply stored at room temperature. Again, mould was a problem in about 10% of cases. The problem is more acute where an additional (un-photographed) sample is retained in the same bag. More condensation develops and mould is therefore a problem, as is specimen decomposition. In almost all cases I was able to verify the determinations made by John, but it was more tricky in some cases.

One answer would be to place specimens in the bags and leave them unsealed for a few days until the specimens have dried out. If left for 24 hours in this state, I think there will be lower risk. The system will hopefully be refined further in 2017.

Working through specimens stored in this manner is a little more time-consuming than working from pinned specimens, but it is a more practical way of working with recorders who don't have the facilities to pin and store specimens.

Sunday, 18 December 2016

Animations of range change

Having recently seen BWARS animations of range change in the Tree Bumblebee Bombus hypnorum I brought this to Stuart's attention. Stuart has produced several animations and I did try to post them on the UK Hoverflies FB page. Unfortunately they did not work! So, instead, I have embedded three into this post.

Stuart tells me that 'These animations attempt to show the relative frequency with which the species was recorded (i.e. attempting to correct for recording effort) over a five year period, finishing in the year shown. It uses the neighbourhoods and weights from Frescalo, i.e. a combination of the proximity and environmental similarity of a neighbouring square relative to the target square. So, for a given five year period and 10km square - the weighted total number of records in a neighbourhood is accumulated and also the weighted numbers of records of the targeted species. That leads to a calculation of the weighted proportion of the number of records of the targeted species in the neighbourhood. These calculated proportions form a 3D surface which is represented as shades of blue - darker the blue, higher the relative proportion of the species. Although the proportions are not scaled the same between species, they are between years within a given species. So if there are years when the species was recorded less frequently, the overall colour will be paler. Of course the opposite is also true - a spike in the proportion will peg the darkest colour. So I think there must be such a spike early in the run for V. zonaria, and that leads to everything being pale later in the run! It does seem to work OK for the other two species.'

Rhingia rostrata
When I first started recording hoverflies, this was a very rare animal that was confined to a ver few locations. It was even given Red Data Book 2 status is the first Insect Red Data Book. Today, it is remarkably widespread!

Rhingia rostrata
Volucella inanis
This was formerly a largely southern species and a classic example of one associated with urban heat islands. Today, it has escaped that envelope and has spread far into Yorkshire, but quite a bit of its northern rage is within the largely urban environment; possibly because its favoured wasps nest in houses?

Volucella inanis

Volucella zonaria
Until the 1940s, this was a rare vagrant but it became established at several locations on the south coast before gaining a foothold in London where it was a classic associate of urban heat islands. It has spread a long way northwards - with a slightly more northerly range than the (assumed) native V. inanis. It is, however, much more coastal in the north.
Volucella zonaria

Thursday, 15 December 2016

HRS training courses - some basic facts

In a recent thread on UK Hoverflies, an observer made a comment that strongly suggested that the HRS was in the pockets of the Wildlife Trusts.

I would like to make clear a few basic facts about the courses that we run:

1. Courses are run on a non-profit basis. We charge the costs of: fuel, accommodation, basic subsistence and any other charges we incur (e.g. Dartford crossing). We also make a charge for course materials - to cover re-printing costs. We do not charge for mileage, but in the course of any year I would expect to cover between 2,000 and 5,000 miles in support of such courses . That means that over five years I provide a set of tyres free of charge to training events - at about £250 for  a set of boots.

2.  If we did not have Wildlife Trusts and Local Records Centres to act as local organisers, these courses would not happen. Courses are open to all, not just to WLT members, so anybody who participates is getting a subsidy from those who support the Trusts. The WLT are an essential part of the network that provides training to field naturalists. If they were not there, we would NOT be running courses.

3. We do not charge directly to the course organisers. I cover all of the costs. An invoice is then sent from Dipterists Forum and monies are paid to the Forum. I subsequently invoice the Forum with receipts for all costs incurred. Consequently, at all times there is a strict audit trail of the costs incurred. I must stress that we make absolutely NO income from these courses. Thus, I pay out anything up to £400 in advance and am subsequently reimbursed. I wonder how many people would be prepared to pay costs up front in this manner!

4. I subsidise the courses by covering the costs of collecting material for the courses - in any one year I prepare somewhere around 250 specimens for use by students. I have also provided all of the store boxes (10) and provide the storage for the material - that is two metres of shelf space. Stuart provides accommodation for the 13 microscopes - about a cubic metre of storage.

5. A typical two day weekend course involves me leaving Stamford at 13.30  on the Friday and collecting Stuart and the relevant equipment from Peterborough before travelling on to our venue. We rarely get back to Peterborough much before 8.00 on Sunday, and often later still. I then have a further 30 minutes drive to Stamford. In total, I reckon we provide between 50 and 60 hours of our time for each course, to which we should add pre-course preparation, which is sometimes considerable.

6. We have travelled to very diverse locations - from Lerwick in the Shetlands and Kirkwall in the Orkneys, to Bangor, Gateshead, Glasgow, Exeter, Studland, Norwich and various other locations in SE England. Overall, I wonder how many people would be willing to donate anything up to 8 weekends a year running such courses?

7. Over the years, we have donated earnings from more commercial courses (e.g. FSc) to Dipterists Forum - we make nothing from them. Often, we don't charge at all, and simply take the view that without such venues it would not be possible to provide training to those who want it. We have also donated all proceeds from the Hoverfly WILDGuide to Diptersists Forum. This income has been used to subsidise a variety of people to attend training courses.

So, I must protest that anybody that thinks that we are profiteering from an association with the Wildlife Trusts is grossly mis-representing the true situation.