Inside the AI developer’s toolbox

To abstract or not to abstract

Jos Martin, senior engineering manager at data analysis and simulation software company MathWorks, defines an artificial intelligence framework as something for programmers that abstracts elements of complexity.

In a post on the Computer Weekly Developer Network (CWDN) blog, Martin says: “As a user of deep learning you don’t usually need to go and write training algorithms.”

For instance, Google’s Brain team has developed TensorFlow, Facebook has created PyTorch, Microsoft provides its Cognitive Toolkit, while Amazon Web Services (AWS) offers the MXNet deep learning framework as a service.

Martin says there are many different intelligence layer types already implemented in openly available libraries, and as such, developers should really view any single layer of intelligence as a whole set of computing neurons connected together.

What these frameworks have in common is that they aim to make it easier for programmers to create AI-powered applications. “Instead of having to spend mental effort on backpropagation formulas, AI engineers can work with abstract, high-level building blocks. This makes the engineering feedback loop much faster and it is comparable to how writing computer code in a high-level programming language is easier than writing code in an assembly language,” Gareth Dwyer, an AI and Python developer for Codementor, writes on the CWDN blog.

A lot of these AI frameworks are being developed under open source licences, with strong inputs from a global community of developers, which creates a vibrant and positive aspect of the IT industry today. But Dwyer warns that tooling in any form and under any type of licence can also be a distraction, or even have a net negative impact.

“Frameworks can suffer from depreciation over time. As Google pushes out new versions of TensorFlow, AI models that were written [the year before] are already obsolete. Most documentation, tutorials, books, presentations and academic papers become obsolete within months. Developers may spend time becoming experts in specific tools, only to find that they need to start over a few months later,” he warns.

There are also some AI frameworks that are proprietary. Joe Michael, a solutions architect at IPSoft, suggests the reason why some end-to-end AI frameworks have not typically been offered under an open source licence is due to their inherent value.

“The reality is that AI companies similarly want developers to learn how to build with their architecture, rather than upskilling in AI more broadly or in a competitor’s technology, as it increases demand for their solutions in the community,” says Michael.

He says some suppliers with an interest in AI tooling will offer elements of their technology as open source to incentivise developers to use particular technology stacks. For instance, Nvidia’s Cuda is offered as open source as a means of incentivising people to use Nvidia graphics processors for machine learning.

What makes AI complex?

Choosing the right AI framework for the job is quite complex. In a CWDN blog post, Boris Cipot, a senior engineer at Synopsys, which makes Design Compiler, a logic-synthesis tool, describes three main considerations.

“The first factor is the programming language we choose for AI programming, which libraries are available to use and what functions they provide. The second factor involves the hardware required to process speech, images and other sensory elements taking place, along with the mathematical functions, that then gives enough data for the decision logic to choose the appropriate response,” Cipot writes.

Most of the decision logic involved in AI processing runs on powerful servers, rather than on smaller devices like mobile phones. But there are also other cases – the automotive industry, for example – where data connectivity is not a given and the decision logic cannot be dependent on a fragile factor such as data connectivity. This is why there is a lot of talk in the industry around edge computing, particularly in internet of things applications, where sensor data is processed locally.

For Cipot, the third factor is the “learning dataset” on which the AI model is built. It needs to be refined based on the responses achieved for different inputs. This is the feedback loop that reinforces the learning process. Such datasets need to be immense to capture all possible variants the AI application could encounter in real use. “Imagine how many languages AI would need to understand to be of use to the entirety of the world’s population. Now consider all the different accents for each language so that users are able to understand the sentence at hand and its context. This is just one example of how massive and complex these datasets can become,” he writes.

Architectural considerations

MathWorks’ Martin says as a user of deep learning you are piecing together layers, thinking about the data you will train them and then doing the training. “I’m not going to underestimate the difficulty of coming up with appropriate network architectures for the layers, but since that is problem-dependent, it seems reasonable that someone who wants to focus on building smarter applications needs to think about their deep network architecture,” he says.

So how does a developer team use a framework in a practical way? The way DeepMind’s AlphaGo deep learning algorithm evolved was the inspiration behind mobile games developer King’s work to build a simulation of a games player using Google’s TensorFlow. The simulation was a key part of the company’s drive to test and release new levels of Candy Crush rapidly.

Alex Nodet, an artificial intelligence engineer at King, says: “We did quite a lot of experiments to get it closer to a real player and changed  the architecture to build a real player.”

The company went through 20 iterations of the model to get to the point where it could be used for its intended purpose. In Nodet’s experience, most of the time spent on a machine learning project is not in tweaking the machine learning model – instead, the difficult part is getting the data in the right format and then transforming it into something that can be used.

However, Philipp Eisen, a machine learning engineer at King, points out that having an ML model that is trained and learning is only half the work. He says it is also necessary to consider how machine learning works in production. n

Additional reporting by Cliff Saran.