Take the 2-minute tour ×

I want to make a little project and I want to use neural networks with python. I found that pybrain is the best solution. But until now, all the examples and questions I have found, cannot help me.

I have a sequence of numbers. Hundreds of rows. Some values are missing and instead of a number, there are a "x".

For example


and so on. This is just an example. Not my sequence.

I thought to read one by one the values and train my neural net and when I find one 'x' I will predict the number and I will continue training it with the following numbers.

What I have found until now are training like this one


with some inputs and some outputs.

Any advice how can I continue with it?

Edit: I figure something and I would like to receive feedback, because I don't know if it is right.

I still have the string for above. I split it in list so I have a list where each entity is a number.

for ind in range(len(myList)):
   if not myList[ind] == "x" and not myList[ind+1]=="x":

net = FeedForwardNetwork() 
inp = LinearLayer(1) 
h1 = SigmoidLayer(1) 
outp = LinearLayer(1)


net.addConnection(FullConnection(inp, h1))  
net.addConnection(FullConnection(h1, outp))


trainer = BackpropTrainer(net, ds)

lis[ind+1] = net.activate((ind,))

GO to the beggining and continue from the last "x" which replaced from the net.activate()

What do you think? Do you believe that something like this will work?

share|improve this question
If you are interested in neural networks, you may consider joining the machine-learning site: area51.stackexchange.com/proposals/41738/machine-learning –  trav1s Jun 6 '13 at 5:30

3 Answers 3

up vote 2 down vote accepted

In general, if you are training your ANN using back propagation, you are basically training an input-output map. This means that your training set has to comprise known input-output relations (none of your unknown values included in the training set). The ANN then becomes an approximation of the actual relationship between your inputs and outputs.

You can then call x = net.activate([seq]) where seq is the input sequence associated with the unknown value x.

If x is an unknown input sequence for a known result, then you have to call the inverse of the ANN. I do not think there is a simple way of inverting an ANN in pybrain, but you could just train an ANN with the inverse of your original training data. In other words, use your known results as the training inputs, and their associated sequences as the training results.

The main thing to consider is the appropriateness of the tool and the training data for what you are trying to do. If you just want to predict x as a function of the previous number, then I think you are training correctly. I am guessing x is going to be a function of the previous n numbers though, in which case you want to update your data set as:

n = 10
for ind in range(len(myList)):
    # Don't overrun our bounds
    if ind == len(myList)-1:

    # Check that our sequence is valid
    for i in range(ind-n, ind+1):
        if i >= 0 and myList[i] == "x":
            # we have an invalid sequence
            ind += i   # start next seq after invalid entry

    # Add valid training sequence to data set
share|improve this answer
Hello @Engineero. The problem with your answer is that each time the input will have different lenght. And I think that when I set my NN like this ds = SupervisedDataSet(1, 1), I have to fix the number of input sequence. I don't know if I can add a sample with lenght 10 and on the other iteration another sample with lenght 20 for example. –  Tasos Jun 7 '13 at 20:23
@Tasos, you could begin the above loop with for ind in range(n, len(myList)): and that way you should only end up with length = 10 input training sequences. –  Engineero Jun 7 '13 at 20:29
So your opinion is to add my sample with a fix number which I choose? A number bigger than 1 is better to have better predictions. Right? –  Tasos Jun 7 '13 at 20:45
I would imagine so, but I do no know the nature of the data set. If you are trying to predict a completely random process then you are not going to get any help from more samples. If your process is such that one number depends on the numbers that came before it, then you might get somewhere. –  Engineero Jun 7 '13 at 21:18
I vote your answer. If I won't get any better, I will choose yours as the best. Thank you. –  Tasos Jun 7 '13 at 21:26

I can give you not a specific answer for that python library, but as I see it, you have a neural net and you give it samples of the form

    [ i0 i1 ... i n ] --> [ o0 o1 ... on ]
    (input vector)        (output vector)

Now you train the the net with sample vectors of length 1. Your net does not know about the sequence of the numbers presented to it, that sequence is only interesting for the outcome of the trained net.

To get a network, that knows about the sequence you could present vectors of consecutive numbers as input and the single number you want, as output. You leave ot the sequences containing the X Example:

    Sequence: 1 2 3 4 X 2 3 4 5 6 7 8
    Training with input length 3, output length 1:
    [1 2 3] -> 4
    [2 3 4] -> 5 (the second one, as the first one is not available)
    [3 4 5] -> 6
    [4 5 6] -> 7
    [5 6 7] -> 8

I think using this, your net can adapt a little to the input sequence. The "how" to extract the right training sequences as input, I have to leave to the domain expert (you).

share|improve this answer

What you are describing is a statistical application called Imputation: substituting missing values in your data. The traditional approach does not involve neural networks, but there has certainly been some research in this direction. This is not my area, but I recommend you check the literature.

share|improve this answer

Your Answer


