Unveiling the Power of Self-Attention for Shipping Cost Prediction: The
  Rate Card Transformer
P Aditya Sreekar Sahil Verma Varun Madhavan Abhishek Persad
Amazon ships billions of packages to its customers annually within the United
States. Shipping cost of these packages are used on the day of shipping (day 0)
to estimate profitability of sales. Downstream systems utilize these days 0
profitability estimates to make financial decisions, such as pricing strategies
and delisting loss-making products. However, obtaining accurate shipping cost
estimates on day 0 is complex for reasons like delay in carrier invoicing or
fixed cost components getting recorded at monthly cadence. Inaccurate shipping
cost estimates can lead to bad decision, such as pricing items too low or high,
or promoting the wrong product to the customers. Current solutions for
estimating shipping costs on day 0 rely on tree-based models that require
extensive manual engineering efforts. In this study, we propose a novel
architecture called the Rate Card Transformer (RCT) that uses self-attention to
encode all package shipping information such as package attributes, carrier
information and route plan. Unlike other transformer-based tabular models, RCT
has the ability to encode a variable list of one-to-many relations of a
shipment, allowing it to capture more information about a shipment. For
example, RCT can encode properties of all products in a package. Our results
demonstrate that cost predictions made by the RCT have 28.82% less error
compared to tree-based GBDT model. Moreover, the RCT outperforms the
state-of-the-art transformer-based tabular model, FTTransformer, by 6.08%. We
also illustrate that the RCT learns a generalized manifold of the rate card
that can improve the performance of tree-based models.