5. Solution: From $2 \cos 2 B-8 \cos B+5=0$, we get $(2 \cos B-3)(2 \cos \beta-1)=0$.
Solving, we get $\dot{B}=60^{\circ}$. Also, from $-\sqrt{3}=\tan (A+C)=$ $\frac{3+\sqrt{3}}{1-\tan A \tan C}$, we get
$\tan A \tan C=2+\sqrt{3}$, so $\tan A=2+\sqrt{3}, \tan C=1$ (since $\tan A, \tan C>0$. and $A>C$ implies $\tan A>\tan C$), which means $A=75^{\circ}$, thus $S_{\triangle A B C}=\frac{1}{2} \cdot C D \cdot C D(\cot A+\cot B)$ $=12-4 \sqrt{3}$.
5. In $\triangle A B C$, $A>B>C$, if $2 \cos 2 B-$ $8 \cos B+5=0, \tan A+\tan C=3+\sqrt{3}$, and the altitude $C D$ from $C$ to $A B$ is $2 \sqrt{3}$, then the area of $\triangle A B C$ is