Click here to Skip to main content
12,823,954 members (43,350 online)
Click here to Skip to main content
Add your own
alternative version

Tagged as


23 bookmarked
Posted 17 Jan 2010

An Introduction to Encog Neural Networks for Java

, 17 Jan 2010 LGPL3
Rate this:
Please Sign up or sign in to vote.
An introduction to creating neural networks with the Encog Framework for Java.



This article provides a basic introduction to neural networks and neural network programming using the Encog Artificial Intelligence Framework. Encog is an AI framework that is available for both Java and Microsoft .NET. In this article, I will show you how to create and train a very basic neural network with Encog. I will likely follow this up with more complex neural network examples in the future. However, for now, this article shows you how to get started with Encog.


Encog is an advanced Artificial Intelligence Framework. Using Encog, you can create advanced neural network applications. Though Encog supports other aspects of Artificial Intelligence programming, this article will focus on neural network programming. Neural Network programming is the primary focus of Encog, as of version 2.3. Encog is released under the Lesser GNU Public License (LGPL). Encog can be downloaded from the following URL:

This is a simple introductory example that shows how to create a neural network that recognizes the XOR operator. The XOR operator is essentially the "Hello World" of the neural network world. It is often used to demonstrate a new neural network.

Before I show you how to create a neural network in Encog, it is important to understand how a neural network works. Nearly all neural networks contain layers. A layer is a group of neurons that behave similarly. There are many different layer types used by the different types of neural networks that are supported by Encog. However, there are two very important layers that nearly every neural network will have. These are the input and output layers. The input layer is how you feed data to the neural network. The output layer is how you get the response back from the neural network.

The input and output from the neural network are both Java double values, ordinary floating-point numbers. Both the input and output layers will have some number of neurons. This determines how many floating-point numbers the layer will deal with. The input and output layers will typically have a different number of neurons. Deciding the structure of the input and output layer is how you define the "problem" that you are trying to solve to the neural network.

Let's see how we could create a neural network to perform as an XOR operator. The XOR operator requires that the two inputs be different, for the output to be true. The XOR operator's truth table can be expressed as follows:

0 XOR 0 = 0
1 XOR 0 = 1
0 XOR 1 = 1
1 XOR 1 = 0

I would like to now create a neural network that can perform this operator. Such a network will have two input neurons and one output neuron. It will accept the two operands and return the result.

Using the Code

We will now take a look at how the code for this example is constructed. This example was created with Encog v2.3, which is the current version of Encog at the time of this writing. The Encog 2.3 JAR file is provided with the download for this example. However, you may wish to grab the latest Encog JAR from the URL provided earlier in this article.

Neural networks must be trained before they are of any use. To train this neural network, me must provide training data. The training data is the truth table for the XOR operator. The XOR has the following inputs:

public static double XOR_INPUT[][] = { 
  { 0.0, 0.0 }, 
  { 1.0, 0.0 },
  { 0.0, 1.0 }, 
  { 1.0, 1.0 } };

These are all of the possible inputs to the XOR operator. Likewise, the expected outputs are also stored as an array.

public static double XOR_IDEAL[][] = { 
  { 0.0 }, 
  { 1.0 }, 
  { 1.0 }, 
  { 0.0 } };

These two arrays will be combined to create the training data for the XOR operator. The following line of code combines these two arrays to create training data:

NeuralDataSet trainingSet = new BasicNeuralDataSet(XOR_INPUT, XOR_IDEAL);

We must now create a neural network. The following lines of code do this:

BasicNetwork network = new BasicNetwork();

network.addLayer(new BasicLayer(new ActivationSigmoid(), true,2));
network.addLayer(new BasicLayer(new ActivationSigmoid(), true,4));
network.addLayer(new BasicLayer(new ActivationSigmoid(), true,1));

You can see the three layers being created. The three addLayer methods above create the three layers. The first layer is the input layer, it has two neurons. The final layer created is the output layer, it has a single neuron. The layer created in the middle is called the hidden layer. The hidden layer helps the rest of the neural network learn. More hidden neurons must be added to allow the neural network to learn more complex patterns. Four hidden neurons is more than enough for the XOR operator. Picking the number of hidden neurons is usually a process of trial and error.

The call to the reset method above randomizes the weights between all of the neurons in the network. These random weights will not allow the neural network to function as an XOR operator. However, the training process, that we will see shortly, will refine these weights and encourage the neural network to produce output similar to the XOR operator.

The ActivationSigmoid class specifies the type of activation that is to be used. The sigmoid function is a good choice here because we use no negative values on the XOR operator. If the network needed to recognize negative, the hyperbolic tangent activation function would be more appropriate. Encog supports a number of different activation functions, all of which have their unique uses.

A training object must be created to train the neural network. Encog supports a number of different training methods. Some are used for specific neural network types. Others perform better on different types of data. The best 'general purpose' trainer for this type of neural network is called resilient propagation (RPROP). The following code creates an RPROP trainer:

final Train train = new ResilientPropagation(network, trainingSet);

Once the trainer has been created, we should loop through a series of iterations. Each iteration will take the neural network closer to being able to function as an XOR operator.

int epoch = 1;

do {

  System.out.println("Epoch #" + epoch + 
                     " Error:" + train.getError());

} while(train.getError() > 0.01);

At each iteration, or epoch, we check the error. We continue until the error is less than 1%. The error defines how far the output of the neural network is from the ideal output defined earlier.

Once the neural network has been trained satisfactorily, we will make use of it. We will feed each of the inputs into the neural network and observe the output. The following code does this:

System.out.println("Neural Network Results:");

for(NeuralDataPair pair: trainingSet ) {

  final NeuralData output = network.compute(pair.getInput());
  System.out.println(pair.getInput().getData(0) + 
         "," + pair.getInput().getData(1)  + 
         ", actual=" + output.getData(0) + 


We loop through each of the training data items and present the output from the neural network as well as the actual output. The output from the program is shown here:

Epoch #1 Error:0.5494587158070631
Epoch #2 Error:0.5494587158070631
Epoch #3 Error:0.5142230676643447
Epoch #4 Error:0.5005001549387983
Epoch #5 Error:0.5004690225170658
Epoch #6 Error:0.5004317073796488
Epoch #40 Error:0.026928155758252966
Epoch #41 Error:0.014605778809936418
Epoch #42 Error:0.007180166468750018
Neural Network Results:
0.0,0.0, actual=0.003348914814615583,ideal=0.0
1.0,0.0, actual=0.995118794356202,ideal=1.0
0.0,1.0, actual=0.9999200785659956,ideal=1.0
1.0,1.0, actual=8.741041885168623E-4,ideal=0.0

As you can see, the neural network loops through a number of iterations before the error percent drops below 1%. After 42 iterations, the network is trained. When you run the application, it may take more or less, as the neural network starts with random weights.

Once it is done, you see the actual output from the neural network. The output from the neural network does not exactly match the ideal output. This is expected. However, the value 0.99 is very close to 1.0.

Points of Interest

This article demonstrated a very simple neural network application. It is massive overkill to use a neural network to create an XOR operator. There are many more complex examples provided with Encog. I will likely write further articles here demonstrating some of the more advanced features of Encog.

One of the interesting features of a neural network is that it is created completely differently from a traditional program. For a traditional program, you would think about how to implement the XOR operator and create all of the necessary programming logic to do it. For a neural network, you just provide input examples and the expected outputs. It is up to the neural network to learn how to provide these expected outputs. Usually, you really have no idea how it actually learns to provide its output. This is especially true with large neural networks that may have hundreds of neurons.




About the Author

United States United States
Jeff Heaton is a data scientist, PhD student and indy publisher. Jeff works primarily in the programming languages Python, R, C#, Java and C/C++. He is an active technology blogger, open source contributor, and author of more than ten books. His areas of expertise include Data Science, Predictive Modeling, Data Mining, Big Data, Business Intelligence and Artificial Intelligence. He is the lead developer for the Encog Machine Learning Framework open source project. Jeff holds a Masters Degree in Information Management from Washington University, is a Senior Member of the IEEE, a Sun Certified Java Programmer and a Fellow of the Life Management Institute.

You may also be interested in...


Comments and Discussions

GeneralLooking forward to a more real world example Pin
Ashwin Jayaprakash21-Jan-10 15:05
memberAshwin Jayaprakash21-Jan-10 15:05 
GeneralRe: Looking forward to a more real world example Pin
JeffHeaton22-Jan-10 4:19
memberJeffHeaton22-Jan-10 4:19 



Permalink | Advertise | Privacy | Terms of Use | Mobile
Web02 | 2.8.170308.1 | Last Updated 17 Jan 2010
Article Copyright 2010 by JeffHeaton
Everything else Copyright © CodeProject, 1999-2017
Layout: fixed | fluid