Can't decrypt RSA data with open SSL
<p>I try to encrypt some data in matlab using a public key I created with openssl</p>

<p>I created the keys using:</p>

<pre><code>openssl genrsa -des3 -out private.pem 1024
openssl rsa -in private.pem -pubout -outform DER -out public.der
</code></pre>

<p>I encrypt my data using this matlab code (with Java libraries):</p>

<pre><code>import java.security.spec.RSAPublicKeySpec
import javax.crypto.Cipher;
import java.security.KeyFactory
import java.math.BigInteger

fid = fopen('public.der');
a = fread(fid);
key = java.security.spec.X509EncodedKeySpec(a);
kf = KeyFactory.getInstance('RSA');
KEY = kf.generatePublic(key);

cipher = Cipher.getInstance('RSA/ECB/PKCS1Padding');
cipher.init(Cipher.ENCRYPT_MODE, KEY)

plaintextBytes = [24];
ciphertext = cipher.doFinal(plaintextBytes)' ;

fid2 = fopen('msg.txt','w');
fwrite(fid2,ciphertext);
fclose(fid2);
</code></pre>

<p>I try to decrypt it using:</p>

<pre><code>openssl rsautl -decrypt -inkey private.pem  -in msg.txt -keyform PEM -pkcs
</code></pre>

<p>Then I get this error:</p>

<pre><code>RSA operation error
80305:error:0407109F:rsa routines:RSA_padding_check_PKCS1_type_2:pkcs decoding error:/BuildRoot/Library/Caches/com.apple.xbs/Sources/OpenSSL098/OpenSSL098-59.40.2/src/crypto/rsa/rsa_pk1.c:267:
80305:error:04065072:rsa routines:RSA_EAY_PRIVATE_DECRYPT:padding check failed:/BuildRoot/Library/Caches/com.apple.xbs/Sources/OpenSSL098/OpenSSL098-59.40.2/src/crypto/rsa/rsa_eay.c:614:
</code></pre>