listen. innovation

The More Machines Learn, the Less We Understand Them (For Now)

Viktoriia Samatova | State Street Corporation

May 24,2018

Want to spot a fibber in finance?

Here's an easy test: Ask if he or she 100% understands the algorithms that help them decide among potential investments.

If they respond with "Yes, of course," chances are they’re stretching the truth.

When we're being honest with ourselves (and our clients), financial professionals will admit that though we're excited by new technology, we have limited insight into how sophisticated algorithms reach their conclusions. That bugs us — no one likes to admit they don’t understand something.

Computer scientists, for all their brilliance in creating algorithms, can't always clarify things for us, either. They design the models and continuously improve them with what's known as training data — information that shapes how algorithms analyze future data inputs. But after that, a model can take on a life of its own, using multiple computations to find patterns indiscernible to the human brain. We can measure the model's accuracy; asking the algorithm to tackle questions to which we already know the answers. But we can't ask machines to, as your elementary school math teacher might say, "show their work."

Of course, this issue isn't limited to finance. It applies to nearly any space where machine learning is revolutionizing business or, in some cases, game play. In 2016, an artificially intelligent system upended conventional wisdom in the world of Go, an ancient Chinese board game. AlphaGo beat renown Go champion Lee Sedol in a five-game series. The series' most iconic moment? When AlphaGo made its 37th move in the second game, placing a stone on the fifth row from the edge of the board.

Commentators freaked out about this move because it made no sense; humans had never played Go that way. We're learning that AlphaGo was able to see enough different ways that the game could unfold where that move could turn out to be a smart one. But we're not 100 percent sure as to how or why.

Uncertainty sits better with computer scientists than with financial professionals.

Yet we can't ignore that machine learning models are accurate and efficient, performing analysis with more speed and less manpower than ever thought possible. And then there are the insights they provide that no human ever could, from an unprecedented winning strategy in Go to exciting advances in industries ranging from medicine1 to agriculture2. Even when we don't have the answers to "why," we must carefully forge ahead, using proven tools to inform our decisions.

We can also anticipate a future where there is greater transparency in machine learning. In finance, as we speak, computer scientists are working to help answer our "why" questions. State Street uses machine learning to recommend what research is most pertinent to our clients' interests. Our developers recently isolated a key factor influencing our model’s decision-making. When it reviews a client's research history, it gives the most weight to reports that inspired interaction. Did the client bookmark a report or share it with others? If so, our algorithm finds similar research to top the client's reading list.

We believe that, as time goes by, more solutions like this will emerge, providing veritable windows into the "minds" of the machines. We may never fully comprehend the multiple iterations and computations performed by the most complex algorithms, but we'll get pretty close. In doing so, we'll finally be able to answer more "why" questions...without setting a single pant leg alight.

1. Paschalidis, Y. (2017, May 30). How Machine Learning Is Helping Us Predict Heart Disease and Diabetes. Retrieved from

2. Hardy, Q. (2017, Aug. 23). 3 Ways Companies Are Building Business Around AI. Retrieved


Topics: Advanced Technology

Viktoriia Samatova | State Street Corporation

In her role as Head of Research and Product Development, Quantextual Research, Viktoriia Samatova is primarily focused on teaching machine learning and natural language processing models to understand financial industry research and, thus, improve investment process. She is listening to foreign language learning programs, science fiction books and innovation in finance talks.