

Join them; it only takes a minute:

Sign up
Join the Stack Overflow community to:
  1. Ask programming questions
  2. Answer and help your peers
  3. Get recognized for your expertise

I'm trying to understand the python compiler/interpreter process more clearly. Unfortunately, I have not taken a class in interpreters nor have I read much about them.

Basically, what I understand right now is that Python code from .py files is first compiled into python bytecode (which i assume are the .pyc files i see occasionally?). Next, the bytecode is compiled into machine code, a language the processor actually understands. Pretty much, I've read this thread Why python compile the source to bytecode before interpreting?

Could somebody give me a good explanation of the whole process keeping in mind that my knowledge of compilers/interpreters is almost non-existent? Or, if that's not possible, maybe give me some resources that give quick overviews of compilers/interpreters?


share|improve this question
You do not "interpret into machine code" — that's what compilers do. Python interpreter just executes the bytecode. (And it's .pyc for bytecode.) – Piotr Kalinowski Jul 21 '10 at 13:26
On a side note, you might find helpful to know that the last modification time of the original .py file is encoded in the .pyc file. This allows Python to figure out if a new .pyc file needs to be created or not. The purpose of .pyc files is, of course, to avoid parsing the whole script each time the script is invoked. A Python program will not run faster if the .pyc is used. Only the loading time changes. – s.m. Jul 21 '10 at 13:46
up vote 33 down vote accepted

The bytecode is not actually interpreted to machine code, unless you are using some exotic implementation such as pypy.

Other than that, you have the description correct. The bytecode is loaded into the Python runtime and interpreted by a virtual machine, which is a piece of code that reads each instruction in the bytecode and executes whatever operation is indicated. You can see this bytecode with the dis module, as follows:

>>> def fib(n): return n if n < 2 else fib(n - 2) + fib(n - 1)
>>> fib(10)
>>> import dis
>>> dis.dis(fib)
  1           0 LOAD_FAST                0 (n)
              3 LOAD_CONST               1 (2)
              6 COMPARE_OP               0 (<)
              9 JUMP_IF_FALSE            5 (to 17)
             12 POP_TOP             
             13 LOAD_FAST                0 (n)
             16 RETURN_VALUE        
        >>   17 POP_TOP             
             18 LOAD_GLOBAL              0 (fib)
             21 LOAD_FAST                0 (n)
             24 LOAD_CONST               1 (2)
             27 BINARY_SUBTRACT     
             28 CALL_FUNCTION            1
             31 LOAD_GLOBAL              0 (fib)
             34 LOAD_FAST                0 (n)
             37 LOAD_CONST               2 (1)
             40 BINARY_SUBTRACT     
             41 CALL_FUNCTION            1
             44 BINARY_ADD          
             45 RETURN_VALUE        

Detailed explanation

It is quite important to understand that the above code is never executed by your CPU; nor is it ever converted into something that is (at least, not on the official C implementation of Python). The CPU executes the virtual machine code, which performs the work indicated by the bytecode instructions. When the interpreter wants to execute the fib function, it reads the instructions one at a time, and does what they tell it to do. It looks at the first instruction, LOAD_FAST 0, and thus grabs parameter 0 (the n passed to fib) from wherever parameters are held and pushes it onto the interpreter's stack (Python's interpreter is a stack machine). On reading the next instruction, LOAD_CONST 1, it grabs constant number 1 from a collection of constants owned by the function, which happens to be the number 2 in this case, and pushes that onto the stack. You can actually see these constants:

>>> fib.func_code.co_consts
(None, 2, 1)

The next instruction, COMPARE_OP 0, tells the interpreter to pop the two topmost stack elements and perform an inequality comparison between them, pushing the Boolean result back onto the stack. The fourth instruction determines, based on the Boolean value, whether to jump forward five instructions or continue on with the next instruction. All that verbiage explains the if n < 2 part of the conditional expression in fib. It will be a highly instructive exercise for you to tease out the meaning and behaviour of the rest of the fib bytecode. The only one, I'm not sure about is POP_TOP; I'm guessing JUMP_IF_FALSE is defined to leave its Boolean argument on the stack rather than popping it, so it has to be popped explicitly.

Even more instructive is to inspect the raw bytecode for fib thus:

>>> code = fib.func_code.co_code
>>> code
>>> import opcode
>>> op = code[0]
>>> op
>>> op = ord(op)
>>> op
>>> opcode.opname[op]

Thus you can see that the first byte of the bytecode is the LOAD_FAST instruction. The next pair of bytes, '\x00\x00' (the number 0 in 16 bits) is the argument to LOAD_FAST, and tells the bytecode interpreter to load parameter 0 onto the stack.

share|improve this answer
So if it's not turned into machine code... how is it finally executed by my x86 procesor? I was under the impression that everything that is happening on my computer could eventually be broken down into 1's and 0's that are being read by my processor or some other hardware. – NickHalden Jul 21 '10 at 15:45
@JGord: I've extended my answer to address your comment. – Marcelo Cantos Jul 21 '10 at 22:46
Ok, so the part that is still blowing my mind is that my processor does not understand the LOAD_FAST opcode correct? That is bytecode for the python virtual machine. So somehow the virtual machine was written in a language that could be assembled into x86. Sure. However, how does the virtual machine actually do the operations it interprets from the byte code on my hardware? Let's take an example. My script does some calculation and it needs to send the result over the bus to my graphics card. The python virtual machine cannot do that am I right? Somehow the physical proc is doing something? – NickHalden Jul 22 '10 at 13:13
The interpreter/VM is in C. It is (to oversimplify somewhat) a loop that uses the current byte to choose one of many cases in a huge switch statement. Somewhere in the middle of the switch, there is a case LOAD_FAST: followed by code that reads the next two bytes, looks up the specified parameter in some "parameters" collection, and pushes it onto a stack object. To interact with the outside world, Python allows calls to extension modules, which act like Python code and objects, but are really compiled code and can thus talk to graphics cards, etc., directly, on behalf of your script(s). – Marcelo Cantos Jul 22 '10 at 13:23
To be a bit more explicit about your last question: there is no Python opcode for "talk to the graphics card". There is an opcode for "call this function in this module", and if the module is a graphics programming extension module, the interpreter will call the library's entry point for the requested function, passing it some parameters. The C library (assuming it's C) teases out the parameters, converting them from Python objects into C values and structs, and forwards the call onto a bona-fide graphics library, which then plonks a colorful triangle on your screen, or whatever. – Marcelo Cantos Jul 22 '10 at 13:34

Your Answer


