Semantic-Preserving Feature Partitioning for Multi-View Ensemble
  Learning
Mohammad Sadegh Khorshidi Navid Yazdanjue Hassan Gharoun Danial Yazdani Mohammad Reza Nikoo Fang Chen Amir H. Gandomi
In machine learning, the exponential growth of data and the associated
``curse of dimensionality'' pose significant challenges, particularly with
expansive yet sparse datasets. Addressing these challenges, multi-view ensemble
learning (MEL) has emerged as a transformative approach, with feature
partitioning (FP) playing a pivotal role in constructing artificial views for
MEL. Our study introduces the Semantic-Preserving Feature Partitioning (SPFP)
algorithm, a novel method grounded in information theory. The SPFP algorithm
effectively partitions datasets into multiple semantically consistent views,
enhancing the MEL process. Through extensive experiments on eight real-world
datasets, ranging from high-dimensional with limited instances to
low-dimensional with high instances, our method demonstrates notable efficacy.
It maintains model accuracy while significantly improving uncertainty measures
in scenarios where high generalization performance is achievable. Conversely,
it retains uncertainty metrics while enhancing accuracy where high
generalization accuracy is less attainable. An effect size analysis further
reveals that the SPFP algorithm outperforms benchmark models by large effect
size and reduces computational demands through effective dimensionality
reduction. The substantial effect sizes observed in most experiments underscore
the algorithm's significant improvements in model performance.