AutoML

State-of-the-art machine learning/AI systems consist of complex pipelines with choices of hyperparameters, models and configuration details that need to be tuned for optimal performance. The resulting optimization space can be too complex and high-dimensional for researchers and engineers to explore manually. When automated systems are used, the high costs of running a single experiment (e.g. training a deep neural network) and the high sample complexity (i.e. large number of experiments required) together make naïve approaches impractical.

Many of the problems we are interested in can be cast as high-dimensional combinatorial optimization tasks.  Broadly speaking, we tackle these problems by designing probabilistic machine learning models to guide (automated) experimental decisions and meta-learning to reduce the sample complexity and transfer knowledge across related datasets or problems.

Specific problems that the Microsoft Research AutoML team focuses on include:

	Neural architecture search
	Model selection
	Feature engineering
	Hyperparameter tuning
	Model compression

Core AutoML technology is already live in Azure Machine Learning (opens in new tab), Power BI (opens in new tab) and other Microsoft products.