Aspect-based sentiment analysis (ABSA) is a granular-level sentiment analysis task that aims to detect the sentiment polarities of a specified aspect in the text. This research shows excessive curiosity in modelling target and context through attention networks to attain effective feature representations for sentiment detection works. We have proposed a synthetic attention in bidirectional encoder representations from transformers (SA-BERT) with an extreme gradient boosting (XGBoost) classifier to classify sentiment polarity in the review dataset. The proposed model generates dynamic word vector encoding of the aspect and corresponding context of the reviews. Then, the aspect and context of the reviews are meaningfully represented by a transformer that can input the vector word in parallel. After that, the model uses the synthetic attention mechanism to learn essential parts of context and aspects in reviews. Finally, the model places overall representation in the sentiment classification layer to predict sentiment polarity. Both proposed SA-BERT and SA-BERT-XGBoost models achieved the highest accuracy (92.02 and 93.71%) on the restaurant16 and highest F-1 scores (81.19 and 81.64%) on the restaurant14 dataset, respectively. The average accuracy and F1 scores are approximately 2 and 3.04% higher than the baseline models (DLCF-DCA-CDM, R-GAT+BERT, ASGCN-DG, AEN-BERT and BERT-PT). Therefore, proposed models outperform in comparison with baseline models.