Visual feature learning with application to medical image classification

  • Siyamalan Manivannan

    Student thesis: Doctoral ThesisDoctor of Philosophy


    Various hand-crafted features have been explored for medical image classification, which include SIFT and Local Binary Patterns (LBP). However, hand-crafted features may not be optimally discriminative for classifying images from particular domains (e.g. colonoscopy), as not necessarily tuned to the domain’s characteristics.

    In this work, I give emphasis on learning highly discriminative local features and image representations to achieve the best possible classification performance for medical images, particularly for colonoscopy and histology (cell) images. I propose approaches to learn local features using unsupervised and weakly-supervised methods, and an approach to improve the feature encoding methods such as bag-of-words. Unlike the existing work, the proposed weakly-supervised approach uses image-level labels to learn the local features. Requiring image-labels instead of region-level labels makes annotations less expensive, and closer to the data normally available from normal clinical practice, hence more feasible in practice.

    In this thesis, first, I propose a generalised version of the LBP descriptor called the Generalised Local Ternary Patterns (gLTP), which is inspired by the success of LBP and its variants for colonoscopy image classification. gLTP is robust to both noise and illumination changes, and I demonstrate its competitive performance compared to the best performing LBP-based descriptors on two different datasets (colonoscopy and histology). However LBP-based descriptors (including gLTP) lose information due to the binarisation step involved in their construction. Therefore, I then propose a descriptor called the Extended Multi-Resolution Local Patterns (xMRLP), which is real-valued and reduces information loss. I propose unsupervised and weakly-supervised learning approaches to learn the set of parameters in xMRLP. I show that the learned descriptors give competitive or better performance compared to other descriptors such as root-SIFT and Random Projections. Finally, I propose an approach to improve feature encoding methods. The approach captures inter-cluster features, providing context information in the feature as well as in the image spaces, in addition to the intra-cluster features often captured by conventional feature encoding approaches.

    The proposed approaches have been evaluated on three datasets, 2-class colonoscopy (2, 100 images), 3-class colonoscopy (2, 800 images) and histology (public dataset, containing 13, 596 images). Some experiments on radiology images (IRMA dataset, public) also were given. I show state-of-the-art or superior classification performance on colonoscopy and histology datasets.
    Date of Award2015
    Original languageEnglish
    SponsorsEuropean Research Council
    SupervisorManuel Trucco (Supervisor)


    • Medical image classification
    • Feature learning
    • Colonoscopy
    • Histology
    • Radiology
    • Local binary patterns
    • Machine learning

    Cite this