Windriver VxWorks Simulator Self modifying code
<p>Good morning.</p>

<p>I have a program that is Self-Modifying-Code.  </p>

<p>Really, it build the binaries, which then are changed by <code>ELFPatch</code> and changes some function's prologues.</p>

<p>I am working with <strong>Windriver WorkBench 3.3 &amp; VxWorks 6.9 Update3</strong>.</p>

<p>I created a standard simulator (PENTIUM),
when i run my code on the simulator:</p>

<pre><code>void replace_prolog(void* func_ptr) {
    char* p = (char*)func_ptr;
    for (int i=0; i &lt; PROLOGUE_SIZE; ++i)
        p[i]=m_prologue[i]; // &lt;&lt; prologue is a member array. 
    ...
}
</code></pre>

<p>Let's call the Real Prologue : Original Prologue;<br>
The Changed Prologue : Changed Prologue;<br>
The One that is placed at Run-Time : Replacement Prologue;  </p>

<p>I get an Exception (signal 11 - Segmentation Fault).<br>
!! I realized it is VxWorks's <code>.text</code> Segment Protection.</p>

<p>So, I created a <code>SimPC based VIP</code> to be my simulator BSP, and <strong>excluded <code>INCLUDE_PROTECT_TEXT</code></strong> (and all it's relevant kernel components)<br>
and run the simulator:<br>
Now, there is no exception! </p>

<h2>Facts</h2>

<ol>
<li>Looking at Memory Browser I see the Changed Prologue Bytes (memory didn't change)!</li>
<li>Printing the buffer to console, prints the Replacement Prologue Bytes values! (Weird)</li>
<li>looking at assembly view (Mega Weird): shows the <strong>Changed Prologue</strong> Hex values but the Original Prologue asm commands (push bp;...) even though the byte value does not match them.</li>
</ol>

<h2>My Questions</h2>

<ul>
<li>Anyone had any experience with modifying <code>.text</code> segment?</li>
<li>Anyone encountered memory that would not change (without an exception/signal) on simulator, which is not a memory mapped port/volatile ?  </li>
</ul>

<h2>Long Shot Assumption</h2>

<p>I have an assumption it is about caching, hinting that vxWorks know this region shouldn't change, so it doesn't write_through, but don't know how i can check it...</p>

<p><strong>EDIT 2</strong>: tried setting my pointers to be <code>volatile</code> => same behavior!</p>

<p>Please Help.</p>