Homepage of Aditya Krishna Menon

Fellow

Research School of Computer Science

The Australian National University (ANU)

Canberra, Australia

Email: aditya.menon followed by anu.edu.au

Biography

I'm a Fellow at the ANU Research School of Computer Science. I work on machine learning and its applications.

I completed my honours in Computer Science from the University of Sydney in 2006 under the supervision of Sanjay Chawla. I completed my PhD in Computer Science from the University of California, San Diego in 2013 under the supervision of Charles Elkan. From May 2013 -- June 2016, I was a researcher in the machine learning group at NICTA. From July 2016 -- December 2017, I was a senior research scientist at CSIRO Data61.

Starting January 2018, I am a Fellow at the Australian National University working with Lexing Xie and Bob Williamson.

Here is a copy of my CV.

Research interests

I am broadly interested in the design and analysis of machine learning algorithms for (weakly) supervised learning problems occurring in practice. Specific areas of interest include:

Weakly-supervised learning (e.g. learning from label noise, positive and unlabelled learning)
Classification with real-world constraints (e.g. class imbalance, fairness)
Matrix factorisation and applications (e.g. collaborative filtering, link prediction)
Temporal point processes and their inference
Relations amongst foundational problems (e.g. class-probability estimation, bipartite ranking)

Selected publications

Below are a few representative publications. For a full list, see here.

Inspired by Bernard Chazelle's wonderful idea of "liner notes" for his papers, I've included some of my own for a few papers. (Liner notes are not peer-, or even coauthor-reviewed.)

Learning from binary labels with instance-dependent corruption.
Aditya Krishna Menon, Brendan van Rooyen, and Nagarajan Natarajan.
To appear in Machine Learning, 2018.
[pdf]
Low-rank linear cold-start recommendation from social data.
Suvash Sedhain, Aditya Krishna Menon, Scott Sanner, Lexing Xie, and Darius Braziunas.
In AAAI Conference on Artificial Intelligence (AAAI), San Francisco, 2017.
[pdf]
Bipartite ranking: a risk-theoretic perspective.
Aditya Krishna Menon and Robert C. Williamson.
In Journal of Machine Learning Research (JMLR), Volume 17, Issue 195. 2016.
[pdf] [liner notes]

Back in 2011, months of exhaustive experimentation with AUC maximisation in the context of link prediction taught me one thing: it's hard to do better than logistic regression. I was excited when later that year, Kotłowski et al. gave a compelling theoretical explanation for why this might be.

When I joined NICTA in 2013, I followed a rite of passage and read Mark Reid and Bob Williamson's tour de force on information, divergences, and risks. Tucked away in that paper was a mention of how the AUC related to the concepts presented. I innocently mentioned to Bob that perhaps there was more to be said here. He agreed, and suggested that I start by spelling out what was in my mind.

We never expected it would take quite that long to spell. [hide]
Linking losses for density ratio and class-probability estimation.
Aditya Krishna Menon and Cheng Soon Ong.
In International Conference on Machine Learning (ICML), New York City, 2016.
[pdf] [slides] [poster] [code] [liner notes]

Long before being introduced to proper losses, I'd known about, and completely accepted, the basic premise as to why density ratio was different to class-probability estimation: the later only indirectly modelled the ratio, which is generally a sub-optimal strategy. Sometime in late 2015, I re-read the cool work on LSIF. This time, with link functions and partial losses fresh in my mind, I noticed that one could think of this as involving a particular proper loss plus link function. This was mildly interesting, but the point remained: using anything but the link that directly yields the density ratio must be a bad idea.

This was something of a defeat for my running theory at the time, that most everything can be attacked with some version of logistic regression. To better understand this failure mode, I was looking at Sugiyama's elegant unified Bregman view of density ratios. I noticed that logistic regression was mentioned, but didn't pay much attention to it; after all, the Bregman view of regret under proper losses immediately led to the KL minimisation view of logistic regression.

It was only on a re-read that I realised there was a bit more to this innocuous result: it was a statement of Bregman minimisation not to the true probability, but to the true density ratio. Now that I didn't know was true for logistic regression. Studying the surprisingly simple proof, I saw it could be viewed as a surprising equality between the KL divergence on probabilities and on ratios.

I was sure this couldn't be true in general; but why? I worked through the case of a general divergence, trying to find out the line where the proof would break.

Half an hour later, after staring at my working multiple times, I was still incredulous that everything seemed to work in general, too. And so was Lemma 2 born. [hide]
Learning from corrupted binary labels via class-probability estimation.
Aditya Krishna Menon, Brendan van Rooyen, Cheng Soon Ong and Robert C. Williamson.
In International Conference on Machine Learning (ICML), Lille, 2015.
[pdf] [slides] [poster] [code] [liner notes]

Ever since I read Elkan and Noto's 2008 paper on PU learning, I was fascinated by the topic and their approach; I also felt like I didn't fully grok either, since their sample-selection bias viewpoint was unfamiliar to me. I was inspired to rectify this deficiency on my part when reading one of du Plessis and Sugiyama's elegant papers that extended this work, and crisply stated the problem in terms of the distributions the underlying samples are drawn from.

Around the same time, I also stumbled upon some beautiful works on label noise, notably those of Scott et al. and Natarajan et al. I was struck by the apparent similarity of the two problems, but still couldn't quite decipher the apparently different approaches taken to solve them.

At this stage, I sought to clarify in my mind how precisely these different problems relate. Which, as is my wont, involved thinking about them in terms of class-probabilities. On discussing ideas with colleagues at NICTA (serendipitously interested in very similar problems), soon enough we ended up writing this paper. [hide]