DCP-NAS: Discrepant Child-Parent Neural Architecture Search for 1-bit
  CNNs
Yanjing Li Sheng Xu Xianbin Cao Li'an Zhuo Baochang Zhang Tian Wang Guodong Guo
Neural architecture search (NAS) proves to be among the effective approaches
for many tasks by generating an application-adaptive neural architecture, which
is still challenged by high computational cost and memory consumption. At the
same time, 1-bit convolutional neural networks (CNNs) with binary weights and
activations show their potential for resource-limited embedded devices. One
natural approach is to use 1-bit CNNs to reduce the computation and memory cost
of NAS by taking advantage of the strengths of each in a unified framework,
while searching the 1-bit CNNs is more challenging due to the more complicated
processes involved. In this paper, we introduce Discrepant Child-Parent Neural
Architecture Search (DCP-NAS) to efficiently search 1-bit CNNs, based on a new
framework of searching the 1-bit model (Child) under the supervision of a
real-valued model (Parent). Particularly, we first utilize a Parent model to
calculate a tangent direction, based on which the tangent propagation method is
introduced to search the optimized 1-bit Child. We further observe a coupling
relationship between the weights and architecture parameters existing in such
differentiable frameworks. To address the issue, we propose a decoupled
optimization method to search an optimized architecture. Extensive experiments
demonstrate that our DCP-NAS achieves much better results than prior arts on
both CIFAR-10 and ImageNet datasets. In particular, the backbones achieved by
our DCP-NAS achieve strong generalization performance on person
re-identification and object detection.