Different types of classification

The power of the classification methods is due to the quality of its algorithms, which have been improved and updated over the years. These are divided into several main types, depending on the nature of the signal used for learning or the type of feedback adopted by the system.

They are as follows:

• Supervised learning: The algorithm generates a function that links input values to a desired output through the observation of a set of examples, in which each piece of data that's input has its relative output data. This is used to construct predictive models.
• Unsupervised learning: The algorithm tries to derive knowledge from a general input, without the help of a set of preclassified examples, which are used to build descriptive models. Typical examples of the application of these algorithms are search engines.

The following diagram shows examples of supervised learning (on the left) and unsupervised learning (on the right):

In summary, the classification is called supervised if the classes are known prior and the training set patterns are labeled, and unsupervised when classes are unknown and must be derived from the data itself.

It is possible to make a further distinction—parametric and non-parametric methods, which can be described as follows:

• Parametric: This makes assumptions about class distributions and derives related parameters from a training set. This is based on the statistical probability distribution of each class. An example would be linear discriminant analysis (LDA).
• Non-parametric: This only takes into account the distances among objects in space, regardless of their distribution, which is then used to estimate the probability density function. An example would be k-nearest neighbors (KNN).

In parametric methods, we look for a good data model. In non-parametric methods, a good discriminating surface is searched. In the following diagram, several classification methods are displayed, grouped by their types:

Finally, a classification or recognition system has the task of providing a user with an assessment of the observed physical reality. This evaluation uses a subdivision of reality (consisting of objects called samples or patterns) in sets, having homogeneous characteristics, and the said classes.