Neural Inference Search for Multiloss Segmentation Models

Sam Slade, Li Zhang, Haoqian Huang, Houshyar Asadi, Chee Peng Lim, Yonghong Yu, Dezong Zhao, Hanhe Lin, Rong Gao

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)
43 Downloads (Pure)


Semantic segmentation is vital for many emerging surveillance applications, but current models cannot be relied upon to meet the required tolerance, particularly in complex tasks that involve multiple classes and varied environments. To improve performance, we propose a novel algorithm, neural inference search (NIS), for hyperparameter optimization pertaining to established deep learning segmentation models in conjunction with a new multiloss function. It incorporates three novel search behaviors, i.e., Maximized Standard Deviation Velocity Prediction, Local Best Velocity Prediction, and n -dimensional Whirlpool Search. The first two behaviors are exploratory, leveraging long short-term memory (LSTM)-convolutional neural network (CNN)-based velocity predictions, while the third employs n -dimensional matrix rotation for local exploitation. A scheduling mechanism is also introduced in NIS to manage the contributions of these three novel search behaviors in stages. NIS optimizes learning and multiloss parameters simultaneously. Compared with state-of-the-art segmentation methods and those optimized with other well-known search algorithms, NIS-optimized models show significant improvements across multiple performance metrics on five segmentation datasets. NIS also reliably yields better solutions as compared with a variety of search methods for solving numerical benchmark functions.

Original languageEnglish
Number of pages15
JournalIEEE Transactions on Neural Networks and Learning Systems
Publication statusE-pub ahead of print - 16 Jun 2023


  • Behavioral sciences
  • Convolutional neural network (CNN)
  • Decoding
  • Loss measurement
  • Semantic segmentation
  • Standards
  • Task analysis
  • Transformers
  • hyperparameter optimization
  • multiloss function
  • semantic segmentation

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications


Dive into the research topics of 'Neural Inference Search for Multiloss Segmentation Models'. Together they form a unique fingerprint.

Cite this