Please note that the content of the website has not bee updated for over many years. Currently the site mainly contains some of the work I did in undergrad and graduate studies. A lot of the old tutorials/notes have been moved to the Wiki.

Subjects:

Skills:

Type:

Study Project

Scope:

Artificial Intelligence Assignment2 - Programming

Date:

June, 2002

Status:

Completed

Link:

A comprehensive report is available.

Artificial neural network can classify groups. Multi-layer perceptron (back propagation) is suitable and widely used to solve complex classification problems. The aim of the project was to "illustrate the learning behavior of back-propagation neural network for pattern classification". In particular, the task was to write a MATLAB program to implement the back propagation algorithm to classify two classes defined by the following conditional probability density functions.

**Procedure**

First, a training data was generated using MATLAB function randn and variances. Second, a feed forward network of one input layer (input themselves), two hidden layers, and one output layer was constructed. Then, the neural network was optimized by appropriately selecting the number of neurons on each layer, the learning rate, and the momentum rate by trial and error.

**Implimentation Code**

%Lab2 format long % intialize variables data_size = 11000; % variance and mean vectors for classes variance_class_1 = 1 ; mean_vector_class_1 = [0;0] ; variance_class_2 = 4 ; mean_vector_class_2 = [2;0] ; % random data x for clases 1 & 2 C1 = zeros(2,data_size); C2 = zeros(2,data_size); for k=1:data_size C1(1,k) = mean_vector_class_1(1,1) ; C1(2,k) = mean_vector_class_1(2,1) ; C2(1,k) = mean_vector_class_2(1,1) ; C2(2,k) = mean_vector_class_2(2,1) ; end C1_x = C1 + sqrt(1)*randn(2, data_size) ; C2_x = C2 + sqrt(2)*randn(2, data_size) ; % Back Propagation Neural Network net = newff([-10 10; -10 10], [8, 4, 1], {'tansig','tansig','purelin'}); net.trainParam.show = 50; % Show results every 50 iterations net.trainParam.epochs = 200; % Max number of iterations net.trainParam.goal = 1e-2; % Error tolerance stopping criterion net.trainParam.mu = 0.9; % Momentum net.trainParam.lr = 0.01; % Learning rate % Train network '0.5' output for class C1 and '0.95' for class C2 for i=1:1000 targets_C1(i) = 0.05; targets_C2(i) = 0.95; end targets = [targets_C1 targets_C2]; train_data = [C1_x(1,1:1000) C2_x(1,1:1000); C1_x(2,1:1000) C2_x(2,1:1000)]; net = train(net, train_data, targets); classification_boundary = 0.5; % Accuracy calculation for test data network_output_C1 = sim(net, [C1_x(1,1000:11000); C1_x(2,1000:11000)]); network_output_C2 = sim(net, [C2_x(1,1000:11000); C2_x(2,1000:11000)]); correct_C1 = 0; correct_C2 = 0; for i=1:10000 if (network_output_C1(i) < classification_boundary ) correct_C1 = correct_C1 + 1; end end for i=1:10000 if (network_output_C2(i) > classification_boundary ) correct_C2 = correct_C2 + 1; end end accuracy_for_C1 = (correct_C1 / 10000)* 100 accuracy_for_C2 = (correct_C2 / 10000)* 100 % The two nets are equally probable accuracy_net = 0.5*accuracy_for_C1 + 0.5*accuracy_for_C2

**Conclusion**

The final network can be characterized as shown in the graph. The optimum learning rate was 0.01 and momentum rate was 0.5. The best average accuracy of classification for both classes was 78%.

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer