drwnML

Class List for drwnML

• drwnBoostedClassifier : Implements a mult-class boosted decision-tree classifier. See Zhu et al., Multi-class AdaBoost, 2006.
• drwnClassifier : Implements the interface for a generic machine learning classifier.
• drwnClassificationResults : Encapsulates summary of classifier output from which various curves can be generated (e.g., precision-recall curves).
• drwnCompositeClassifier : Implements a multi-class classifier by combining binary classifiers.
• drwnConditionalGaussian : Utility class for generating conditonal gaussian distribution.
• drwnCondSuffStats : Implements a class for accumulating conditional first- and second-order sufficient statistics.
• drwnConfusionMatrix : Utility class for computing and printing confusion matrices.
• drwnDataset : Implements a cacheable dataset containing feature vectors, labels and optional weights.
• drwnDecisionTree : Implements a (binary-split) decision tree classifier of arbitrary depth.
• drwnDisjointSets : Implements a forest of disjoint sets abstract data type.
• drwnFeatureMap : Defines the interface for a feature mapping .
• drwnFeatureTransform : Implements the interface for a generic feature transforms possibly with learned parameters, e.g., PCA (unsupervised) or LDA (supervised).
• drwnFeatureWhitener : Whitens (zero mean, unit variance) feature vector (see also drwnPCA).
• drwnFisherLDA : Fisher's linear discriminant analysis (LDA).
• drwnGaussian : Implements a multi-variate gaussian distribution.
• drwnGaussianMixture : Implements a multi-variant Gaussian mixture model.
• drwnHistogram : Implements a simple templated histogram class.
• drwnJointFeatureMap : Defines the interface for a joint feature mapping .
• drwnKMeans : Implements k-means clustering. Outputs the squared-distance to each of the cluster centroids. The nearest cluster can be found by passing the output to the drwn::argmin function. Supports weighted training examples.
• drwnTLinearRegressor : Implements linear regression optimization templated on a drwnFeatureMap.
• drwnLinearTransform : Implements a linear feature transform with externally settable parameters.
• drwnLPSolver : Solves equality constrained linear programs with positivity constraints via the log-barrier method.
• drwnTMultiClassLogistic : Implements a multi-class logistic classifier templated on a drwnJointFeatureMap.
• drwnOptimizer : Interface for solving large-scale unconstrained optimization problems using L-BFGS.
• drwnPCA : Principal component analysis feature transformation.
• drwnPRCurve : Precision-recall curve.
• drwnQPSolver : Quadratic program solver.
• drwnRandomForest : Implements a Random forest ensemble of decision trees classifier. See L. Breiman, "Random Forests", Machine Learning, 2001.
• drwnRegression : Implements the interface for a generic machine learning regression, e.g. see drwnLinearRegressor.
• drwnSparseLPSolver : Solves linear programs with sparse equality constraints.
• drwnSparseVec : Quick-and-dirty sparse vector class as a plugin replacement for std::vector.
• drwnSuffStats : Implements a class for accumulating first- and second-order sufficient statistics (moments).

Disjoint Sets

The drwnDisjointSets class implements a forest of disjoint sets abstract data type. The elements are numbered consequtively from 0 to N - 1. Each element belongs to exactly one set, where the sets have unique integer identifiers in . To get the number of elements of each set, call size(id) with a valid set identifier. The total number of elements (N) is given by size. The following code snippet provides an example usage for the class.

int main(int argc, char *argv[])
{
  // create disjoint sets
  const int n = 100;
  drwnDisjointSets sets(n);

  // randomly merge some sets
  for (int i = 0; i < n; i++) {
    int s1 = sets.find(rand() % n);
    int s2 = sets.find(rand() % n);
    if (s1 != s2) {
      sets.join(s1, s2);
    }
  }

  // show sets
  vector<int> s = sets.getSetIds();
  for (vector<int>::const_iterator it = s.begin(); it != s.end(); it++) {
    cout << toString(sets.getMembers(*s)) << endl;
  }

  return 0;
}

Feature Space Mappings

Objects that provides utility functions for mapping from an input feature space (possibly joint) to an output feature space.

Formally, the mapping is defined as for class-less feature mappings or for joint feature mappings where and . For example, we can define multi-class logistic regression as

The standard joint feature map, in this case, is (see drwnIdentityJointFeatureMap). Note that an assignment to the last class label results in a zero feature vector.

Note

drwnTMultiClassLogistic and drwnTLinearRegressor can make use of feature maps to transform an input feature space to an output feature space on-the-fly. The feature mapping is performed each time the output feature vector is needed, which can be very efficient for simple feature transforms but expensive for complicated transformations. In such cases it may be better to transform the features first (see, for example, drwnTFeatureMapTransform).

The following code snipet shows an example of a joint feature map being used to train a multi-class logistic classifier:

// load a dataset
drwnClassifierDataset dataset("trainingSet.bin");
int numFeatures = dataset.numFeatures();
int numClasses = dataset.maxTarget() + 1;

// train the classifier
drwnTMultiClassLogistic<drwnSquareJointFeatureMap> classifier;
classifier.initialize(numFeatures, numClasses);
classifier.train(dataset.features, dataset.targets);

// test accuracy
dataset.read("testingSet.bin", false);
vector<int> predictedLabels;
classifier.getClassifications(dataset.features, predictedLabels);

drwnConfusionMatrix confusion(numClasses);
confusion.accumulate(dataset.targets, predictedLabels);
DRWN_LOG_MESSAGE("Accuracy: " << confusion.accuracy());    

See Also

drwnFeatureMap

drwnJointFeatureMap

drwnTMultiClassLogistic, drwnMultiClassLogistic

drwnTLinearRegressor, drwnLinearRegressor