The DARC Toolbox: automated, flexible, and efficient delayed and risky choice experiments using Bayesian adaptive design

Benjamin Vincent, Tom Rainforth

Research output: Working paper

Abstract

Delayed and risky choice (DARC) experiments are a cornerstone of research in psychology, behavioural economics and neuroeconomics. By collecting an agent's preferences between pairs of prospects we can characterise their preferences, investigate what affects them, and probe the underlying decision making mechanisms. We present a state-of-the-art approach and software toolbox allowing such DARC experiments to be run in a highly efficient way. Data collection is costly, so our toolbox automatically and adaptively generates pairs of prospects in real time to maximise the information gathered about the participant's behaviours. We demonstrate that this leads to improvements over alternative experimental paradigms. The key to releasing our real time and automatic performance is a number of advances over current Bayesian adaptive design methodology. In particular, we derive an improved estimator for discrete output problems and design a novel algorithm for automating sequential adaptive design. We provide a number of pre-prepared DARC tools for researchers to use, but a key contribution is an adaptive experiment toolbox that can be extended to virtually any 2-alternative-choice tasks. In particular, to carry out custom adaptive experiments using our toolbox, the user need only encode their behavioural model and design space - both the subsequent inference and sequential design optimisation are automated for arbitrary models the user might write.
Original languageEnglish
PublisherPsyArXiv
Number of pages49
DOIs
Publication statusE-pub ahead of print - 20 Oct 2017

Fingerprint

Experiments
Decision making
Economics
Design optimization

Cite this

@techreport{dbf574b1d9f0441a93d7b00fa0da7821,
title = "The DARC Toolbox: automated, flexible, and efficient delayed and risky choice experiments using Bayesian adaptive design",
abstract = "Delayed and risky choice (DARC) experiments are a cornerstone of research in psychology, behavioural economics and neuroeconomics. By collecting an agent's preferences between pairs of prospects we can characterise their preferences, investigate what affects them, and probe the underlying decision making mechanisms. We present a state-of-the-art approach and software toolbox allowing such DARC experiments to be run in a highly efficient way. Data collection is costly, so our toolbox automatically and adaptively generates pairs of prospects in real time to maximise the information gathered about the participant's behaviours. We demonstrate that this leads to improvements over alternative experimental paradigms. The key to releasing our real time and automatic performance is a number of advances over current Bayesian adaptive design methodology. In particular, we derive an improved estimator for discrete output problems and design a novel algorithm for automating sequential adaptive design. We provide a number of pre-prepared DARC tools for researchers to use, but a key contribution is an adaptive experiment toolbox that can be extended to virtually any 2-alternative-choice tasks. In particular, to carry out custom adaptive experiments using our toolbox, the user need only encode their behavioural model and design space - both the subsequent inference and sequential design optimisation are automated for arbitrary models the user might write.",
author = "Benjamin Vincent and Tom Rainforth",
year = "2017",
month = "10",
day = "20",
doi = "10.17605/OSF.IO/YEHJB",
language = "English",
publisher = "PsyArXiv",
address = "United States",
type = "WorkingPaper",
institution = "PsyArXiv",

}

TY - UNPB

T1 - The DARC Toolbox: automated, flexible, and efficient delayed and risky choice experiments using Bayesian adaptive design

AU - Vincent, Benjamin

AU - Rainforth, Tom

PY - 2017/10/20

Y1 - 2017/10/20

N2 - Delayed and risky choice (DARC) experiments are a cornerstone of research in psychology, behavioural economics and neuroeconomics. By collecting an agent's preferences between pairs of prospects we can characterise their preferences, investigate what affects them, and probe the underlying decision making mechanisms. We present a state-of-the-art approach and software toolbox allowing such DARC experiments to be run in a highly efficient way. Data collection is costly, so our toolbox automatically and adaptively generates pairs of prospects in real time to maximise the information gathered about the participant's behaviours. We demonstrate that this leads to improvements over alternative experimental paradigms. The key to releasing our real time and automatic performance is a number of advances over current Bayesian adaptive design methodology. In particular, we derive an improved estimator for discrete output problems and design a novel algorithm for automating sequential adaptive design. We provide a number of pre-prepared DARC tools for researchers to use, but a key contribution is an adaptive experiment toolbox that can be extended to virtually any 2-alternative-choice tasks. In particular, to carry out custom adaptive experiments using our toolbox, the user need only encode their behavioural model and design space - both the subsequent inference and sequential design optimisation are automated for arbitrary models the user might write.

AB - Delayed and risky choice (DARC) experiments are a cornerstone of research in psychology, behavioural economics and neuroeconomics. By collecting an agent's preferences between pairs of prospects we can characterise their preferences, investigate what affects them, and probe the underlying decision making mechanisms. We present a state-of-the-art approach and software toolbox allowing such DARC experiments to be run in a highly efficient way. Data collection is costly, so our toolbox automatically and adaptively generates pairs of prospects in real time to maximise the information gathered about the participant's behaviours. We demonstrate that this leads to improvements over alternative experimental paradigms. The key to releasing our real time and automatic performance is a number of advances over current Bayesian adaptive design methodology. In particular, we derive an improved estimator for discrete output problems and design a novel algorithm for automating sequential adaptive design. We provide a number of pre-prepared DARC tools for researchers to use, but a key contribution is an adaptive experiment toolbox that can be extended to virtually any 2-alternative-choice tasks. In particular, to carry out custom adaptive experiments using our toolbox, the user need only encode their behavioural model and design space - both the subsequent inference and sequential design optimisation are automated for arbitrary models the user might write.

U2 - 10.17605/OSF.IO/YEHJB

DO - 10.17605/OSF.IO/YEHJB

M3 - Working paper

BT - The DARC Toolbox: automated, flexible, and efficient delayed and risky choice experiments using Bayesian adaptive design

PB - PsyArXiv

ER -