Meta Episodic learning with Dynamic Task Sampling for CLIP-based Point
  Cloud Classification
Shuvozit Ghose Yang Wang
Point cloud classification refers to the process of assigning semantic labels
or categories to individual points within a point cloud data structure. Recent
works have explored the extension of pre-trained CLIP to 3D recognition. In
this direction, CLIP-based point cloud models like PointCLIP, CLIP2Point have
become state-of-the-art methods in the few-shot setup. Although these methods
show promising performance for some classes like airplanes, desks, guitars,
etc, the performance for some classes like the cup, flower pot, sink,
nightstand, etc is still far from satisfactory. This is due to the fact that
the adapter of CLIP-based models is trained using randomly sampled N-way K-shot
data in the standard supervised learning setup. In this paper, we propose a
novel meta-episodic learning framework for CLIP-based point cloud
classification, addressing the challenges of limited training examples and
sampling unknown classes. Additionally, we introduce dynamic task sampling
within the episode based on performance memory. This sampling strategy
effectively addresses the challenge of sampling unknown classes, ensuring that
the model learns from a diverse range of classes and promotes the exploration
of underrepresented categories. By dynamically updating the performance memory,
we adaptively prioritize the sampling of classes based on their performance,
enhancing the model's ability to handle challenging and real-world scenarios.
Experiments show an average performance gain of 3-6\% on ModelNet40 and
ScanobjectNN datasets in a few-shot setup.