Minecraft wiki entry for Defeating_a_Nether_fortress, associated text: Seen any magma cubes around? Kill one to get 2-3 magma cream. Be prepared to deal with a pack of them as they split up like slimes.
If you have tons of spare gold and are playing in a version pre-1.16.2, you can barter with piglins which have a chance of dropping 2-6 magma cream for you.
In loot chests in bastions you can occasionally find magma cream, but especially post-1.16.1 those are better saved for later.
Combining a slimeball from a wandering trader or killing a slime with a blaze powder will make magma cream, but slimes are relatively rare.