Enhance Sketch Recognition's Explainability via Semantic Component-Level
  Parsing
Guangming Zhu Siyuan Wang Tianci Wu Liang Zhang
Free-hand sketches are appealing for humans as a universal tool to depict the
visual world. Humans can recognize varied sketches of a category easily by
identifying the concurrence and layout of the intrinsic semantic components of
the category, since humans draw free-hand sketches based a common consensus
that which types of semantic components constitute each sketch category. For
example, an airplane should at least have a fuselage and wings. Based on this
analysis, a semantic component-level memory module is constructed and embedded
in the proposed structured sketch recognition network in this paper. The memory
keys representing semantic components of each sketch category can be
self-learned and enhance the recognition network's explainability. Our proposed
networks can deal with different situations of sketch recognition, i.e., with
or without semantic components labels of strokes. Experiments on the SPG and
SketchIME datasets demonstrate the memory module's flexibility and the
recognition network's explainability. The code and data are available at
https://github.com/GuangmingZhu/SketchESC.