Near real-time processing of proteomics data using Hadoop.

Chris Hillman; Yasmeen Ahmad; Mark Whitehorn; Andy Cobley

doi:10.1089/big.2013.0036

Near real-time processing of proteomics data using Hadoop.

Chris Hillman (Lead / Corresponding author), Yasmeen Ahmad, Mark Whitehorn, Andy Cobley

Computing

Research output: Contribution to journal › Article › peer-review

7 Citations (Scopus)

420 Downloads (Pure)

Abstract

This article presents a near real-time processing solution using MapReduce and Hadoop. The solution is aimed at some of the data management and processing challenges facing the life sciences community. Research into genes and their product proteins generates huge volumes of data that must be extensively preprocessed before any biological insight can be gained. In order to carry out this processing in a timely manner, we have investigated the use of techniques from the big data field. These are applied specifically to process data resulting from mass spectrometers in the course of proteomic experiments. Here we present methods of handling the raw data in Hadoop, and then we investigate a process for preprocessing the data using Java code and the MapReduce framework to identify 2D and 3D peaks.

Original language	English
Pages (from-to)	44-49
Number of pages	5
Journal	Big Data
Volume	2
Issue number	1
Early online date	14 Feb 2014
DOIs	https://doi.org/10.1089/big.2013.0036
Publication status	Published - 14 Mar 2014

Access to Document

10.1089/big.2013.0036Licence: No Licence / Unknown

final published versionFinal published version, 1.09 MBLicence: CC BY

Cite this

@article{83999479b34d43ba87374b98688caa45,

title = "Near real-time processing of proteomics data using Hadoop.",

abstract = "This article presents a near real-time processing solution using MapReduce and Hadoop. The solution is aimed at some of the data management and processing challenges facing the life sciences community. Research into genes and their product proteins generates huge volumes of data that must be extensively preprocessed before any biological insight can be gained. In order to carry out this processing in a timely manner, we have investigated the use of techniques from the big data field. These are applied specifically to process data resulting from mass spectrometers in the course of proteomic experiments. Here we present methods of handling the raw data in Hadoop, and then we investigate a process for preprocessing the data using Java code and the MapReduce framework to identify 2D and 3D peaks.",

author = "Chris Hillman and Yasmeen Ahmad and Mark Whitehorn and Andy Cobley",

year = "2014",

month = mar,

day = "14",

doi = "10.1089/big.2013.0036",

language = "English",

volume = "2",

pages = "44--49",

journal = "Big Data",

issn = "2167-6461",

publisher = "Mary Ann Liebert Inc.",

number = "1",

}

TY - JOUR

T1 - Near real-time processing of proteomics data using Hadoop.

AU - Hillman, Chris

AU - Ahmad, Yasmeen

AU - Whitehorn, Mark

AU - Cobley, Andy

PY - 2014/3/14

Y1 - 2014/3/14

N2 - This article presents a near real-time processing solution using MapReduce and Hadoop. The solution is aimed at some of the data management and processing challenges facing the life sciences community. Research into genes and their product proteins generates huge volumes of data that must be extensively preprocessed before any biological insight can be gained. In order to carry out this processing in a timely manner, we have investigated the use of techniques from the big data field. These are applied specifically to process data resulting from mass spectrometers in the course of proteomic experiments. Here we present methods of handling the raw data in Hadoop, and then we investigate a process for preprocessing the data using Java code and the MapReduce framework to identify 2D and 3D peaks.

AB - This article presents a near real-time processing solution using MapReduce and Hadoop. The solution is aimed at some of the data management and processing challenges facing the life sciences community. Research into genes and their product proteins generates huge volumes of data that must be extensively preprocessed before any biological insight can be gained. In order to carry out this processing in a timely manner, we have investigated the use of techniques from the big data field. These are applied specifically to process data resulting from mass spectrometers in the course of proteomic experiments. Here we present methods of handling the raw data in Hadoop, and then we investigate a process for preprocessing the data using Java code and the MapReduce framework to identify 2D and 3D peaks.

U2 - 10.1089/big.2013.0036

DO - 10.1089/big.2013.0036

M3 - Article

SN - 2167-6461

VL - 2

SP - 44

EP - 49

JO - Big Data

JF - Big Data

IS - 1

ER -

Near real-time processing of proteomics data using Hadoop.

Abstract

Access to Document

Fingerprint

Cite this