The AdaBoost algorithm of Freund and Schapire was the first practical boosting algorithm, and remains one of the most widely used and studied, with applications in numerous fields.

How to set αt? The process of selecting αt and ht(x) can be interpreted as a single optimization step minimising the upper bound on the empirical error. Improvement of the bound is guaranteed, …

AdaBoost stands for Adaptive Boosting. [Literally, boosting here means to arrange a set of weak classifiers in a sequence in which each weak classifier is the best choice for a classifier at that point …

The AdaBoost algorithm can be viewed as an algorithm that searches for hypotheses of the form of Equation 1 in order to minimize the empirical loss under the exponential loss function:

This interpretation generalizes AdaBoost to other convex surrogates of the zero-one loss (e.g., hinge or logistic). It also hints at a deeper connection between gradient descent and ensemble learning.

Here we discuss the loss function interpretation of AdaBoost. As was shown (decades after AdaBoost was first invented), AdaBoost can be viewed as greedy optimization of a particular loss function.

In this chapter, we are considering AdaBoost algorithm for the two class classi cation problem. AdaBoost (Adaptive Boosting) generates a sequence of hypothesis and combines them with weights. That is. T …