Testing peatland testate amoeba transfer functions: appropriate methods for clustered training-sets

Richard J. Payne; Richard J. Telford; Jeffrey J. Blackford; Antony Blundell; Robert K. Booth; Dan J. Charman; Łukasz Lamentowicz; Mariusz Lamentowicz; Edward A.D. Mitchell; Genevieve Potts; Graeme T. Swindles; Barry G. Warner; Wendy Woodland

doi:10.1177/0959683611430412

Testing peatland testate amoeba transfer functions: appropriate methods for clustered training-sets

Richard J. Payne (Lead / Corresponding author)
, Richard J. Telford (Lead / Corresponding author)
, Jeffrey J. Blackford
, Antony Blundell
, Robert K. Booth
, Dan J. Charman
, Łukasz Lamentowicz
, Mariusz Lamentowicz
, Edward A.D. Mitchell
, Genevieve Potts
, Graeme T. Swindles
, Barry G. Warner
, Wendy Woodland

Research output: Contribution to journal › Article › peer-review

62 Citations (Scopus)

Abstract

Transfer functions are widely used in palaeoecology to infer past environmental conditions from fossil remains of many groups of organisms. In contrast to traditional training-set design with one observation per site, some training-sets, including those for peatland testate amoeba-hydrology transfer functions, have a clustered structure with many observations from each site. Here we show that this clustered design causes standard performance statistics to be overly optimistic. Model performance when applied to independent data sets is considerably weaker than suggested by statistical cross-validation. We discuss the reasons for these problems and describe leave-one-site-out cross-validation and the cluster bootstrap as appropriate methods for clustered training-sets. Using these methods we show that the performance of most testate amoeba-hydrology transfer functions is worse than previously assumed and reconstructions are more uncertain.

Original language	English
Pages (from-to)	819-825
Number of pages	7
Journal	Holocene
Volume	22
Issue number	7
Early online date	14 Dec 2011
DOIs	https://doi.org/10.1177/0959683611430412
Publication status	Published - Jul 2012

Keywords

cluster bootstrap
clustered data
leave-one-site-out cross-validation
palaeoclimate
transfer functions

ASJC Scopus subject areas

Global and Planetary Change
Archaeology
Ecology
Earth-Surface Processes
Palaeontology

Access to Document

10.1177/0959683611430412

Cite this

Payne, R. J., Telford, R. J., Blackford, J. J., Blundell, A., Booth, R. K., Charman, D. J., Lamentowicz, Ł., Lamentowicz, M., Mitchell, E. A. D., Potts, G., Swindles, G. T., Warner, B. G., & Woodland, W. (2012). Testing peatland testate amoeba transfer functions: appropriate methods for clustered training-sets. Holocene, 22(7), 819-825. https://doi.org/10.1177/0959683611430412

@article{3fb3489daac1481b89577dc400791305,

title = "Testing peatland testate amoeba transfer functions: appropriate methods for clustered training-sets",

abstract = "Transfer functions are widely used in palaeoecology to infer past environmental conditions from fossil remains of many groups of organisms. In contrast to traditional training-set design with one observation per site, some training-sets, including those for peatland testate amoeba-hydrology transfer functions, have a clustered structure with many observations from each site. Here we show that this clustered design causes standard performance statistics to be overly optimistic. Model performance when applied to independent data sets is considerably weaker than suggested by statistical cross-validation. We discuss the reasons for these problems and describe leave-one-site-out cross-validation and the cluster bootstrap as appropriate methods for clustered training-sets. Using these methods we show that the performance of most testate amoeba-hydrology transfer functions is worse than previously assumed and reconstructions are more uncertain.",

keywords = "cluster bootstrap, clustered data, leave-one-site-out cross-validation, palaeoclimate, transfer functions",

author = "Payne, \{Richard J.\} and Telford, \{Richard J.\} and Blackford, \{Jeffrey J.\} and Antony Blundell and Booth, \{Robert K.\} and Charman, \{Dan J.\} and {\L}ukasz Lamentowicz and Mariusz Lamentowicz and Mitchell, \{Edward A.D.\} and Genevieve Potts and Swindles, \{Graeme T.\} and Warner, \{Barry G.\} and Wendy Woodland",

note = "Funding Information: RJP was supported by a Humanities Research Fellowship from the University of Manchester and a Study Grant from the British Institute at Ankara. Norwegian Research Council projects ARCTREC and PES helped support RJT.",

year = "2012",

month = jul,

doi = "10.1177/0959683611430412",

language = "English",

volume = "22",

pages = "819--825",

journal = "Holocene",

issn = "0959-6836",

publisher = "SAGE Publications",

number = "7",

}

TY - JOUR

T1 - Testing peatland testate amoeba transfer functions

T2 - appropriate methods for clustered training-sets

AU - Payne, Richard J.

AU - Telford, Richard J.

AU - Blackford, Jeffrey J.

AU - Blundell, Antony

AU - Booth, Robert K.

AU - Charman, Dan J.

AU - Lamentowicz, Łukasz

AU - Lamentowicz, Mariusz

AU - Mitchell, Edward A.D.

AU - Potts, Genevieve

AU - Swindles, Graeme T.

AU - Warner, Barry G.

AU - Woodland, Wendy

N1 - Funding Information: RJP was supported by a Humanities Research Fellowship from the University of Manchester and a Study Grant from the British Institute at Ankara. Norwegian Research Council projects ARCTREC and PES helped support RJT.

PY - 2012/7

Y1 - 2012/7

N2 - Transfer functions are widely used in palaeoecology to infer past environmental conditions from fossil remains of many groups of organisms. In contrast to traditional training-set design with one observation per site, some training-sets, including those for peatland testate amoeba-hydrology transfer functions, have a clustered structure with many observations from each site. Here we show that this clustered design causes standard performance statistics to be overly optimistic. Model performance when applied to independent data sets is considerably weaker than suggested by statistical cross-validation. We discuss the reasons for these problems and describe leave-one-site-out cross-validation and the cluster bootstrap as appropriate methods for clustered training-sets. Using these methods we show that the performance of most testate amoeba-hydrology transfer functions is worse than previously assumed and reconstructions are more uncertain.

AB - Transfer functions are widely used in palaeoecology to infer past environmental conditions from fossil remains of many groups of organisms. In contrast to traditional training-set design with one observation per site, some training-sets, including those for peatland testate amoeba-hydrology transfer functions, have a clustered structure with many observations from each site. Here we show that this clustered design causes standard performance statistics to be overly optimistic. Model performance when applied to independent data sets is considerably weaker than suggested by statistical cross-validation. We discuss the reasons for these problems and describe leave-one-site-out cross-validation and the cluster bootstrap as appropriate methods for clustered training-sets. Using these methods we show that the performance of most testate amoeba-hydrology transfer functions is worse than previously assumed and reconstructions are more uncertain.

KW - cluster bootstrap

KW - clustered data

KW - leave-one-site-out cross-validation

KW - palaeoclimate

KW - transfer functions

UR - https://www.scopus.com/pages/publications/84861723909

U2 - 10.1177/0959683611430412

DO - 10.1177/0959683611430412

M3 - Article

AN - SCOPUS:84861723909

SN - 0959-6836

VL - 22

SP - 819

EP - 825

JO - Holocene

JF - Holocene

IS - 7

ER -

Testing peatland testate amoeba transfer functions: appropriate methods for clustered training-sets

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this