Build 2020: How Microsoft aims to build trust in artificial intelligence

During its annual Build 2020 developer conference, Microsoft outlined plans for responsible development processes for machine learning. The aim is to make these repeatable, reliable and accountable.

Azure Machine Learning automatically tracks the lineage of datasets, so customers can maintain an audit trail of their machine learning assets, including history, training and model explanations, in a central registry. This gives data scientists, machine learning engineers and developers improved visibility and auditability in their workflows.

During the conference, Microsoft provided guidance on how custom tags in Azure Machine Learning could be used to implement datasheets for machine learning models, enabling customers to improve the documentation and metadata of their datasets. Custom tags are based on Microsoft’s work with the Partnership on AI and its Annotation and Benchmarking on Understanding and Transparency of Machine Learning Lifecycles (About ML) project, which aims to increase transparency and accountability in the documentation of machine learning systems.

Another area of focus for Microsoft is the tools that build a better understanding of machine learning models and assess and mitigate unfairness in data. It has been investing heavily in tools for model unfairness and explainability over the past few months. These areas are of particular importance to practitioners of machine learning at the moment.

In 2019, Microsoft released Fairlearn, an open source toolkit that assesses the fairness of machine learning models. At this year’s Build, Microsoft announced that it would integrate the toolkit natively into Azure Machine Learning in June.

The Fairlearn toolkit offers up to 15 “fairness metrics” against which to assess and retrain models. It also offers visual dashboards to show practitioners how the model is performing against certain groups selected by the customer, such as age, gender or race. Microsoft plans to add to these capabilities as research in the field progresses.

It is also addressing explainable artificial intelligence (AI) with a tool called InterpretML, which offers a set of interactive dashboards that use various techniques to deliver model explainability. For different types of model, InterpretML helps practitioners better understand the most important features in determining the model’s output, perform “what if” analyses and explore trends in the data.

Microsoft also announced that it was adding a new user interface equipped with a set of visualisations for interpretability, support for text-based classification and counter-factual example analysis to the toolset.