Efficient Active Learning Halfspaces with Tsybakov Noise: A Non-convex
  Optimization Approach
Yinan Li Chicheng Zhang
We study the problem of computationally and label efficient PAC active
learning $d$-dimensional halfspaces with Tsybakov
Noise~\citep{tsybakov2004optimal} under structured unlabeled data
distributions. Inspired by~\cite{diakonikolas2020learning}, we prove that any
approximate first-order stationary point of a smooth nonconvex loss function
yields a halfspace with a low excess error guarantee. In light of the above
structural result, we design a nonconvex optimization-based algorithm with a
label complexity of $\tilde{O}(d
(\frac{1}{\epsilon})^{\frac{8-6\alpha}{3\alpha-1}})$\footnote{In the main body
of this work, we use $\tilde{O}(\cdot), \tilde{\Theta}(\cdot)$ to hide factors
of the form $\polylog(d, \frac{1}{\epsilon}, \frac{1}{\delta})$}, under the
assumption that the Tsybakov noise parameter $\alpha \in (\frac13, 1]$, which
narrows down the gap between the label complexities of the previously known
efficient passive or active
algorithms~\citep{diakonikolas2020polynomial,zhang2021improved} and the
information-theoretic lower bound in this setting.