Signing a PDF with adbe.pkcs7.detached
<p>I have a PDF generator written in C, and now I want to add digital signatures to it. I started with a minimal PDF, signed it with <a href="http://jsignpdf.sourceforge.net/" rel="nofollow">JSignPdf</a>, and am now trying to get my own program to produce a file that Adobe Acrobat Reader will interpret identically. I've checked the <a href="https://stackoverflow.com/questions/378247/digitally-sign-pdf-files">Digitally sign PDF files</a> question, but the comments there seemed to conclude that one should use iText instead of trying to do it yourself. I don't want that.</p>

<p><em>Update:</em>
I have indeed also read both the PDF Reference for 1.7 and the "32000" specification linked below, but sometimes I get a bit lost by the number of references. Starting with a working example is often the easiest way for me to understand how everything fits together. Sorry for not making this clear in my initial post.</p>

<p>I've gotten Acrobat Reader to acknowledge that there is a signature in the file, but something is still wrong. In the Signature Panel it says "Signed by Unknown" instead of using the correct name from the key. When opening "Signature Properties" it says "This signature is invalid because there are errors in the formatting or information contained in this signature". On the "Advanced Signature Properties", the Hash Algorithm is "Not available".</p>

<p>The PDF from JSignPdf is correct, according to Acrobat Reader. After telling it to accept my self signed certificate, it displays a nice green checkbox for the signature. To find the minimal additions needed, I've cleared one PDF tag after another, carefully not changing the offset for the remaining tags. This gives the same "This signature is invalid..." error message as above, but it still shows "The signer's identity is valid", as well as showing the Hash Algorithm as "SHA1".</p>

<p>The question is what the reason is for this difference, and if there are any tools that can give a more detailed explanation of what is wrong?</p>

<p>In the /Type/Catalog dictionary, I have an /AcroForm. I've tried putting it both in place and as a reference, but that makes no difference. The /AcroForm contains /SigFlags 3 and /Fields [ x 0 R ], where x is the id of a /Type/Annot with /Subtype/Widget. (The "endobj" is moved to the ">>" line to save some space here.)</p>

<p><em>Update:</em> There are some dictionaries, even though I don't remember their name right now, where the "in place" vs "reference" is significant. Especially the implementation notes in the 1.7 spec has a few of these, as well as a few "the spec says this field is optional, but actually it's required".</p>

<pre><code>2 0 obj &lt;&lt;
 /Type /Catalog
 /Pages 3 0 R
 /AcroForm &lt;&lt;
  /Fields [ 8 0 R ]
  /SigFlags 3
  &gt;&gt;
 &gt;&gt; endobj
</code></pre>

<p>In the /Type/Page object, I have /Annots [ x 0 R ], which seemed to be required to get Acrobat Reader to accept that there was any signature here at all.</p>

<p><em>Update:</em> With a working signature, things change a bit. Without this reference, Acrobat Reader does indeed say the signature is valid, but doesn't show any details about it. With it, the "Signature Properties" menu item is enabled again.</p>

<pre><code>4 0 obj &lt;&lt;
 /Type /Page
 /Parent 3 0 R
 /Resources &lt;&lt;
  /ProcSet [/PDF /Text]
  /Font &lt;&lt; /F1 6 0 R &gt;&gt;
  &gt;&gt;
 /MediaBox [0 0 595 842]
 /Contents 5 0 R
 /Annots [ 8 0 R ]
&gt;&gt; endobj
</code></pre>

<p>The /Annot dictionary contains /T(Signature1), /FT/Sig, /Rect[0 0 0 0], and /V y 0 R, where y is a /Type/Sig object. The JSignPdf version also contains "/F 132" and "/P 4 0 R", but I can't find them in the <a href="http://www.adobe.com/content/dam/Adobe/en/devnet/acrobat/pdfs/PDF32000_2008.pdf" rel="nofollow">PDF Specification</a>. They don't seem to be required anyway.</p>

<p><em>Update:</em> Ah, I had missed the link from section 12.7.1 to 12.5.2.</p>

<pre><code>8 0 obj &lt;&lt;
 /Subtype/Widget
 /T(Signature1)
 /V 7 0 R
 /Type/Annot
 /FT/Sig
 /Rect [ 0 0 0 0 ]
&gt;&gt; endobj
</code></pre>

<p>The /Type/Sig object contains /Filter/Adobe.PPKLite, /SubFilter/adbe.pkcs7.detached, /M(D:20160907094326+02'00'), a /ByteRange array and a /Contents string.</p>

<p><em>Update:</em> I'm using this combination as it was recommended for PDF/A.</p>

<pre><code>7 0 obj &lt;&lt;
 /Contents &lt;3082031f...&gt;
 /Filter/Adobe.PPKLite
 /Type/Sig
 /ByteRange [ 0 904 2907 527 ]
 /SubFilter/adbe.pkcs7.detached
 /M(D:20160907094326+02'00')
&gt;&gt; endobj 
</code></pre>

<p>The /ByteArray has for values: 0, offset-of-last-byte-before-"&lt;"-in-Contents, offset-of-first-byte-after-">", and the length of the remainder of the file. If I take the file from JSignPdf, run this (where buf contains the file data):</p>

<pre><code>SHA1_Init(ctx);
SHA1_Update(ctx, buf + offset1, len1);
SHA1_Update(ctx, buf + offset2, len2);
SHA1_Final(digest, ctx);
</code></pre>

<p>I get the exact same data as in the PKCS7 data for the ":messageDigest" tag. The same is true for my own file. So, I trust those values to be correct.</p>

<p>Using the same cert and key I get the exact same PKCS7 data, except of course the messageDigest and rsaEncryption hex dumps. However, copying the JSignPdf PKCS7 data to my file (as they are exactly the same length) doesn't work, it still complains about not finding the Hash Algorithm. My PKCS7 data in the JSignPdf works, but of course gives the wrong checksum. So, everything OpenSSL-related is most likely correct, the problem must be in the PDF tags somewhere. Is there a reference I've missed, or some tag or object ordering that must be followed?</p>

<p><strong>Solved:</strong> The only remaining thing to play with at this point was the values for the ByteRange tag. The first length was actually ok. However, the second offset was off by one in the implementation, being 1 too small. Adjusting this, I got a green checkbox for the signature!</p>