How does the sysenter instruction work? What instruction is called after sysenter? Is the next instruction saved in a register?

mov     edx,esp
  • it goes into ring 0 typically with edx pointing to the parameters and eax being the 'id' of what to do – evlncrn8 Sep 23 '14 at 6:51
  • possible duplicate of How does sysenter work under Linux? – perror Sep 23 '14 at 7:02
  • I am suspecting that sysenter works the same for Linux and Windows... But, I didn't try with Windows, so you need to try out. – perror Sep 23 '14 at 7:04
  • @perror: the linked question explains how to use sysenter to call Linux syscalls. It does not explain how systenter works on the CPU level. – Igor Skochinsky Sep 23 '14 at 10:27
  • @Igor : Right, I probably misunderstood the question. Sorry. – perror Sep 23 '14 at 11:11

Here is a perfect explanation how sysenter works: http://www.osronline.com/article.cfm?id=257

In essence: All native API calls from User Mode have a body that simply loads an index into EAX, executes SystemCallStub, and returns






| improve this answer | |
  • exactly can you say where save address eiP after call sysenter in returning to user mode?in stack or register? – user2522767 Sep 24 '14 at 12:16
  • a trap frame is used and setup for EIP after returning to user-mode. read about how trap frames work exactly – pHiL Sep 24 '14 at 12:33
  • Did you mean Nowhere saving? can you show me a reference about trap frames work? tanx – user2522767 Sep 24 '14 at 12:49
  • the trap frame is set in kernel-mode so you cant modify it from user-mode. If you want to learn more, search for "interrupts, traps, exceptions" – pHiL Sep 25 '14 at 9:38

The SYSENTER instruction affects many registers, but most importantly it sets EIP to the value of the SYSENTER_EIP_MSR register (in IA-32 terminology). At boot-time, Windows sets SYSENTER_EIP_MSR to the address of ntoskrnl!KiFastCallEntry().

So at a high-level, SYSENTER "jumps" to ntoskrnl!KiFastCallEntry().

For more details, see http://www.codeguru.com/cpp/misc/misc/system/article.php/c8223/%22System-Call-Optimization-with-the-SYSENTER-Instruction.htm and http://trapframe.org/just-enough-kernel-to-get-by-2/.

| improve this answer | |

Your Answer

