why org.apache.spark.sql.types.DecimalType's max precision value is 38 in SparkSQL?
<p>I am working with Apache Spark's SQL to process structural Bigdata. I came across usage of Spark SQL's datatypes specially DecimalType that support largest number to store than any other datatype from sparkSQL, but up to 38 precision even though according to documentation: <a href="http://spark.apache.org/docs/latest/sql-programming-guide.html#data-types" rel="nofollow noreferrer">http://spark.apache.org/docs/latest/sql-programming-guide.html#data-types</a> in that you can find: <a href="https://i.stack.imgur.com/wkiCp.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/wkiCp.png" alt="enter image description here"></a> and It internally use BigDecimal of Scala language which allow precision approx. 2^32. why it is so?
<br/><br/>I need to achieve same functionality that provided by scala's BigDecimal with sparkSQL. can i get to know how i can achieve solution to this or any other way I can try out?   </p>