With the enrichment and improvement of gesture language, sign Language becomes more and more important for communication between hearing impaired and ordinary people, and American Sign Language(ASL) fingerspelling recognition using an uncalibrated visual camera is a meaningful attempt. This paper uses the low-cost Kinect depth camera to obtain RGBD images of sign language. Through the proposed image segmentation algorithm SD-Segment, the pixels of RGB and depth images are aligned when the camera__ internal parameters are unknown, and the image is accurately segmented at the same time. A dual-path feature blending attention network (DFANet) is developed to obtain the fine-grained distinguishing features related to gestures. In order to take advantage of the complementarity between depth and RGB image, a depth-pixel aware module (DPAM) is developed to utilize the pixel relationship in the RGBD feature maps. According to the experimental results, compared with the state-of-the-art methods on the publicly available ASL fingerspelling dataset, the accuracy rate(+2.40%) of the network is greatly improved, and the highest accuracy rate of 98.16% is obtained on the self-built 8-person dataset. DPAM helps the network highlight distinctive hand regions in RGBD images, and DPAM does not significantly increase the number of parameters and computational overhead of the network.