Dark logo

Artificial Neural Network Regression and Classification

Published August 9, 2021
Doug Rose
Author | Agility | Artificial Intelligence | Data Ethics

Unlike human beings who often learn for the intrinsic value of knowing something, machine-learning is almost always purpose-driven. Your job as the machine's developer is to determine what that purpose is before you start development. With neural network regression and classification, you then need to decide on the capability that will serve that purpose best. Ask yourself, “Am I looking at a classification problem, a regression problem, or a clustering problem?” Those are the three things artificial neural networks do best: classification, regression, and clustering. Here’s how you choose:

1. Classification is best when you need to assign inputs to known (labeled) categories. There are two types of classification:

  • Binary classification is used when you have only two categories; for example, a machine learning algorithm may create a line graph that distinguishes between dogs and cats based on their size and type, as shown below.
  • Multi-class classification is used when you have three or more categories; for example, if you need a system that can place customers in four categories — very dissatisfied, dissatisfied, satisfied, and delighted.

2. Regression is best when you need to predict a continuous response value — a variable that can take on any value between its minimum and maximum value; for example, if you need a system that can predict the value of a home based on certain criteria, such as square footage, location, number of bedrooms and bathrooms, and so on.

3. Clustering is the right choice when you want to identify patterns in the data and have no idea what those patterns may be; for example, if you want to identify patterns among loyal, somewhat loyal, and un-loyal customers.

 In this article, you gain a deeper understanding of classification and regression. In my next article, I focus on clustering problems. But first, let's take a look at how the approach to machine learning differs based on the type of problem you are trying to solve.

Supervised Versus Unsupervised Learning

Classification and regression problems involve supervised learning — using training data, to teach the machine (the artificial neural network) how to associate inputs with outputs. For example, you may feed the machine a picture of a cat and tell it, "This is a cat." You feed it a picture of a dog and tell it, "This is a dog." Then, you feed the machine test data; for example, a picture of a cat without telling the machine what the animal in the picture is, and the machine should be able to tell you it's a cat. If the machine gives the incorrect answer, you correct it, and the machine makes adjustments to improve its accuracy.

Clustering problems are in the realm of unsupervised learning. You feed the machine data inputs without labels, and the machine identifies common patterns among the inputs without labeling those patterns. 

For more about supervised and unsupervised learning, see my previous post Supervised Versus Unsupervised Learning.

Solving Classification Problems

Classification is one of the most common ways to use an artificial neural network. For example, credit card companies use classification to detect and prevent fraudulent transactions. The human trainer will feed the machine an example of a fraudulent transaction and tell the machine, "This is fraud." The trainer then feeds the machine an example of an honest transaction and tells the machine, "This is not fraud." As the trainer feeds more and more labeled data into the machine, it learns the patterns in the data that distinguish fraudulent transactions from honest transactions.

The machine may be set up with three output nodes (one for each class). If a transaction is highly characteristic of fraud, the Fraud neuron fires to cancel the transaction and suspend the card. If a transaction is less characteristic of fraud, the Maybe Fraud neuron fires to notify the cardholder of suspicious activity. If the transaction is even less characteristic of fraud, the Not Fraud neuron fires and the transaction is processed. 

Solving Regression Problems

In regression problems, the machine tries to come up with an approximation based on the data input. During the training session, instead of showing the machine how inputs are connected to labels, you show the machine the connection between a known outcome and the variables that impact that outcome. For example, the amount of time it takes to drive home from work varies depending on weather conditions, traffic conditions, and the time of day, as shown below.

A stock price predictor would be another example of machine learning used to solve a regression problem. The stock price would be the dependent variable and would be driven by a host of independent variables, including earnings, profits, future estimated earnings, a change of management, accounting errors or scandals, and so forth.

One way to look at the difference between classification and regression is that with classification the output requires a class label, whereas with regression the output is an approximation or likelihood.

In my next article, I examine an entirely different type of problem — those that can be solved not by classification or regression but by clustering.

Related Posts
August 9, 2021
The Neural Network Chain Rule

The neural network chain rule works with backpropagation to help calculate the cost of the error in the gradient descent.

Read More
August 9, 2021
The Neural Network Cost Function

The artificial neural network cost function helps machine learning system learn from their mistakes.

Read More
August 9, 2021
Neural Network Backpropagation

Artificial neural network backpropagation uses backward propagation to adjust the network weights to correct errors in its predictions.

Read More
1 2 3 5
9450 SW Gemini Drive #32865
Beaverton, Oregon, 97008-7105
Dark logo
© 2022 Doug Enterprises, LLC All Rights Reserved
linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram