Full data - why does it matter?

In the early days of biological recording, all that mattered was the creation of dots on maps. Nobody quite knew what occurred where, so a record comprising a four-figure or even two-figure grid reference and a year date was sufficient to create such a dot - job done!

In those early years the information was, at best, sparse. There was a certain amount of interest in first and last dates for species, so there were plenty of recorders who would say 'I'll give you first and last dates but cannot be bothered with the rest'. Again, the early data are not great for complete runs of records throughout the year and, even now, we get a proportion of recorders who don't see a lot of point in recording common species throughout the season. I think this partly stems from a lack of understanding of how data can be used and why it is desirable to have the most comprehensive information possible.

So, what do we need, ideally?

This is a complex question because one could collect all manner of data but it has to be stored and retrieved and then used. Even the very best databases are not well suited to retention of every form of information. For example, habitat information is often very difficult to capture because different recorders interpret habitat differently. Instead, I think we need to look at how the data are currently used and work to that requirement.

From the HRS perspective, we use data to create maps - so grid reference is essential. Most requests for data are for mapping to 1km resolution but may be more refined on occasions so a minimum of a four-figure grid reference is desirable. Higher resolution (e.g. 100 metres, 10 metres or 1 metre) may be possible if a single record but when one moves about whilst recording 100 metre resolution is probably the best that can be achieved. That is normally enough to locate a record within a polygon forming the outline of a site, so will probably work OK with GIS investigations. Rot holes in individual trees may well be recorded at much finer resolution if you have a GPS.

The full date is also absolutely essential. Unless it is for a given date, the record cannot readily be used in any analysis to look at phenological changes or relationships with local or national weather patterns. So, data that comprise a date range (e.g. 5-7 July 2018) cannot be used in such analyses. They are also very tricky to store in the database. We do accept data from Malaise traps and other trapping systems but the information has much more limited uses.

Giving first and last dates actually distorts the dataset, so this approach is not helpful. When we look at phenological change and any links to climate change, we look at the deviation from the historic median, so we need to calculate the new median. That is dependent upon as refined data as possible. So, no matter how common a species is, we want all records and not just first and last dates.

Likewise, we are interested in all records for a given site on a given date. Full lists convey much more information, and the coverage of all species is a critical part of modern occupancy models that I have written about in the past (Data requirements for occupancy modelling 23/05/2018). Common species form the constant background for understanding what might or might not be present using occupancy models.

I like detail on the gender of the animal seen. Until I started extracting this data from photographs we had very little information on the differences in male and female phenology but we now have quite a lot. Not all species behave in the same ways, as I have also shown in my posts. We continue to benefit from improvements in this aspect of data collection and can now look at how males and females respond to changing weather patterns.

There is a huge amount of interest in flower visit information related to pollinator studies. Demand can only be expected to grow. So, records of flowers visited are very useful. Only last year I was asked for HRS data on visitors to ivy and was able to confidently supply some 5,000 records. What we need though is records where the animal is actually visiting the flower and showing signs of nectaring or taking pollen. Unfortunately, historic data has all manner of information that might or might not be correct, with lots of records of say 'on sycamore' which might mean visiting sycamore flowers or could mean sitting on a sycamore leaf. So notes need to be accurate - I note as 'at x or y' - with the at denoting that a flower visit is involved.

Behavioural notes are also very useful e.g. seen in copula or defending a territory. Other observations can also be helpful as they start to build up a picture of the life of the animal. Such notes are especially useful where larvae have been recorded. If you do record larvae, do make sure that this is noted - we have a lot of records of species such as Cheilosia grossa that clearly don't occur in July as adults, but the data give no hint that it is a larval record. Such records are flagged as doubtful and don't get used in analysis or mapping!

Hopefully this short discussion will help to explain why I can be such a pain in pressing for full dates or proper grid references. The critical issue is that datasets such as those compiled by the HRS are a key tool in understanding what is happening to wildlife and may in some small way influence policy-makers to do the right thing!

Looking for a career in conservation?

There is fierce competition for jobs in conservation, so the big question is ‘how to make yourself stand out from the crowd?’ The answer is to put yourself in the shoes of the team tasked with sifting all those applications. I used to do this job as part of recruiting new staff, and it was incredibly difficult. Imagine that pile of anything upwards of 500 applicants for a job (there was one in the 1990s where I was told there were over 1,000 applicants! You must be clear why you have rejected people with very similar degrees, so you also must have a set of criteria against which to judge applicants. One of my criteria was ‘does the applicant actually show a real interest in wildlife? And ‘will this person bring something special into the team?’

As an applicant for jobs, have you ever turned the role around and then asked – would I stand out from the crowd? If not, how can I make myself more employable? I guess a MSc might help, but then an awful lot of people have a MSc. So, maybe a PhD? I am certainly aware of one job where I was eminently qualified but did not have a PhD and therefore did not make the shortlist. BUT, a PhD does not necessarily mean that the person will have what it really takes to work with people or to look at issues from a practical angle. Equally, there is a danger that a PhD will look at a job in a way that is too research orientated instead of being something that has to be delivered within a clear timescale. So, as an employer I would not leap to the conclusion that a PhD is essential. Where is the evidence of enthusiasm and personal drive?

Remember too, that although you will start your career in a non-managerial role, you may want to ascend the greasy pole! Getting suitable organising experience from an early age is often very helpful. You don’t have to be in work to get such experience. Participating in the running of a club or society at Uni is a good start. Then, there comes the post-Uni experience – clubs and societies are crying out for new, young and dynamic members. Their ranks may seem ‘crusty’ but that will change if you join, get involved and encourage other youngsters to join in. Why are those societies lacking young people? It is a vicious circle – the lack of younger members puts young people off joining, and so the society gets older still. You could break that mould and, in the process, put something positive on your CV both for today and for the future.

There are also skills that are in short supply. My former employer was stuffed full of birders (and no it was not RSPB), but there were precious few entomologists beyond those interested in dragonflies and Lepidoptera. Think about acquiring skills in a less well-frequented discipline. Yes, they are harder to break into because you need to think about keys, microscopes and maybe retaining specimens. BUT, in the process, you will acquire taxonomic skills and will also learn a lot more about ecology. The best ecologists I know are amongst Coleopterists, Hemipterists and (I would say that) especially Dipterists!

It will take years to become known as a top-ranking birder – there are so many also-rans! It will take a lot less time to become known in the circles of Dipterists, Coleopterists, Hymenopterists and Hemipterists. So, get involved with those groups. Everybody is getting worried that we don’t have a vibrant new youth base. We could have if there were new young leaders. Why not get involved with one (or more) of the Recording Schemes. Scheme organisers are on the lookout for bright young replacements – eventually we will retire and there will be vacancies for new ‘names’. It could be you, but you do need to put in the effort.

Remember too, that most societies have newsletters and journals that like short notes. Such notes and observations all count as ‘publications’ and although not top-stream peer-review, they do help you hone your writing skills and experience of ‘peer-review’ which may stand you in good stead should you think about a PhD. I’ve been hugely frustrated to see some really nice MSc projects that have never been written up but would have formed a very nice paper in the journals of one of the major non-vocational societies such as the British Journal of Entomology & Natural History and Dipterists Digest.

Does this resonate with you? If so, the world is your oyster. You don’t have to go abroad to develop the experience that can set you apart from the crowd; and you don’t have to wait. The sooner you get started the sooner you will be developing a name and reputation.

I don’t think I had a particularly outstanding career, but I have gained immensely from my engagement in the non-vocational world. As a 16-year old I joined the Committee of Mitcham Camera Club. By 25 I was on Cons Committee at Surrey Wildlife Trust and by 33 I was scheme organiser for the Hoverfly Recording Scheme. I’ve done a lot more besides and have gained hugely from the experience both professionally and socially. You could do the same or a lot better!

Species status depends upon accurate information

Following up on a thread on UK Hoverflies concerning wildlife tourism, I thought I ought to go back to the status review of hoverflies to illustrate the state of knowledge difference between hoverflies and butterflies.

With butterflies, we see a continual growth in the numbers of species creeping towards the red lists.That creep exemplifies the problem of a lot of Britain's wildlife: habitat is being lost or it is changing as a result of atmospheric nitrification, lack of management or inappropriate management, or basic loss of habitat.

In the case of hoverflies, however, the reviews point in the other direction with species being knocked off the lists on a fairly regular basis. This is because our state of knowledge is slowly improving and it turns out that we got the status wrong in the first instance. As with all things, early versions are often troubled by lack of data. This is illustrated by the changes in the conservation statuses currently used and those from the previous generation of reviews.

Figure 1. Conservation statuses of hoverflies from three generations of status reviews. Note that the system for evaluating red-list statuses has changed but the one for determining 'Nationally Scarce' has not.

The more data that we have, the better chance we have of identifying robust and meaningful conservation statuses. If we lack the data then the hand of those who wish to undermine the importance of less well-known taxa is strengthened! You can see the riducule coming up in the Public Hearing:

..... 'and how many people actually look for this 1cm long fly?' 'You say maybe 20!' 'That suggests to me that the fly is not rare, but is overlooked and of little consequence'. 'Your Honour, I submit that this evidence is false evidence because it is incomplete'.

It is an issue that Stuart and I have grappled with for a lot of our working lives. We have long argued that the problem for invertebrate conservation is that it is a Cinderella subject. So few people take the time to develop real skills and therefore the volume of information available is small. Those who wish to destroy important invertebrate sites can rubbish the conservation argument with ease, and the press can destroy it with a few choice headlines - e.g. 

100 new jobs put at risk by tiny fly!

When we prepared the status review for hoverflies we built in some resilience by looking at the levels of recorder effort and adjusting the statuses to take this into account. Our approach will hopefully mean that the statuses will remain largely stable in coming years, but we must expect there to be some movement in lesser-known species. But we still need more people to tackle the difficult groups that require specimens and microscopy. Unless we encourage the development of detailed skills and the greater use of microscopy, much of invertebrate conservation will continue to be a losing battle and we will be unable to reliably track the fates of the more specialist species.

The start of Autum?

The data for hoverfly records extracted from the UK Hoverflies Facebook page seem to be showing that this week is the start of the autumn slowdown in hoverfly abundance. I suspect that at least some of this is weather-related with poor conditions in several areas as I gather from posts.

So, I wondered whether there were regional differences in the trends and sorted the data according to three basic regions - Sorth of a line between the Humber and the Mersey (appx), Midlands - roughly down to a line between the Severn and the Thames, and South. For this analysis I used a centred 5 day running mean to smooth the inevitable spikiness of the graphs.

What do the respective graphs tell us? Well, I have been a bit surprised by the data. In previous years, the smallest graph has always been the North where we normally have far fewer recorders. It looks as though there have been more records from the north this year (Figure 1), which is a welcome change. Northern England and Scotland are often much less well-represented in a wides spectrum of recording schemes.

Figure 1. Daily records for 3 regions in 2018 presented as a 5-day centred running mean

Southern England records normally far-outstrip those of either the North or the Midlands, mainly because the vast majority of recorders seem to be based in southern England. It is difficult to be sure what has happened this year but there seems not to have been the same level of activity in the south. Perhaps that is because a larger number of people moving to spreadsheets came from the South? I think that is unlikely because we would expect recruitment of newer recorders to mirror past recruitment to a very large extent. So, my suspicion is that the effects have been far more pronounced in the South and that this has had an impact on recorder activity too.

I therefore looked at the proportions of daily records from the three regions and was surprised to see a significant difference at or around the point where the heatwave struck.In southern England it hit around 20 June and records plummeted by 25 June. In the north, however, records stayed high until around 10 July before dropping quickly (Figure 2). It has to be borne in mind that Figure 2 represents the proportion of all records, so as records from one area drop, those in other areas will effectively rise. So, the drop in the proportion of northern records really reflects a rise in the numbers of records in the south as it recovers from the harshest impact of the drought and heatwave. There may, nevertheless also be a somewhat delayed effect on the northern fauna

 

Figure 2. Daily records for 2018 from the middle of April represented as proportions for each region on a 5-day centred running mean

I think that some of the more erratic peaks from late July onwards reflect differing weather patterns such that the three regions have presented rather different recording opportunities. So, there is a bit of interesting work to be done linking these patterns to the respective local weather variation.

Interpreting data - effects of the heatwave

In my last post on the effects of the heatwave, I demonstrated the short-term impact of the heatwave on the numbers of records and numbers of species recorded. To do this, I used the data assembled from social media; primarily Facebook, but also to a much smaller degree Flickr. The big advantage of this dataset is that people almost invariably post their observations within a day or two of the observation; thus giving as near to a 'real-time' picture of what is going on.

It is certainly true that a small number of records will arrive over the following weeks, but approximately 98% arrive within two days of the record data. Thus, any small difference between data secured two days after the event and two weeks after the event is around 2%. When you bear in mind that the numbers of records for each day can be in the range of 120 to 180 per day in mid-summer, any slight lag in receiving records is of comparatively little consequence. I should add that all data posted are extracted within 24 hours of the post, so there is a nominal lag there too. Furthermore, the data are aggregated into weekly blocks, so there would need to be a huge influx of data for any one week to make a significant change to the shape of the graphs.

In the case of the graphs I presented on 16 August, the dip in record numbers between weeks 25 and 26 is 191 records (611 down to 420) (31.2%)  and the dip in the numbers of species recorded at the same time is 21 (75 down to 54) or 28%. Thus, there would have to have been a huge volume of records withheld for those dates but no records withheld for other dates. That seems unlikely and, indeed it is clearly not the case as the graphs look much the same now using updated data (figures 1 & 2).

Thus, I think we can be fairly safe in saying that there was a significant drop in both the abundance and diversity of hoverflies recorded during the most influential part of the drought. We can also see that as the season has progressed, the numbers of records arriving has revived to approximately the same levels as might be expected from the average of the preceding 3 years.

Figure 1. Weekly numbers of records in 2018 compared with the avaerage for the preceding 3 years.

Figure 2. Numbers of species recorded each week in 2018 compared with the average for the preceding 3 years.