Alibaba's AI software surpasses humans in reading test.

SINGAPORE: Alibaba Group's research team on Monday (Jan 15) said its deep learning software has scored higher than a human being on Stanford University's reading comprehension test - an achievement it touted as the first win for the machines. 

In a press release, Alibaba's Institute of Data Science of Technologies (iDST) said its neural network generated a score of 82.44 in providing exact answers to questions, beating the humans' score of 82.304, on the Stanford Question Answering Dataset (SQuAD). 

However, Microsoft Research Asia appears to have at least equalled the feat, and earlier, as it scored 82.650 on Jan 3, two days before iDST notched its achievement on Jan 5, according to the SQuAD's website. 

SQuAD is a large-scale reading comprehension dataset that consists of more than 100,000 question-answer pairs based on more than 500 Wikipedia articles, and participating teams are to build machine-learning models to provide answers to questions, the online site said.  

Regardless, the performance of the deep learning software is a "milestone" worth noting, said iDST's chief scientist of natural language processing Si Luo. 

"That means objective questions such as 'what causes rain' can now be answered with high accuracy by machines," the chief scientist said in the press release.

"To our excitement, we believe the technology underneath can be gradually applied to numerous applications such as customer services, museum tutorial and online response to inquiry from patients, freeing up human efforts in an (unprecedented) way.” 

According to a South China Morning Post report on Monday, the scientist said the system currently only works with questions that offer clear-cut answers, and if they are "too vague or ungrammatical, or there is no prepared answer", the software may not work as it should.

The research unit said the model is based off its Hierarchical Attention Network, which reads from paragraphs to sentences to words, in order to locate the precise phrases with potential answers. 

This has been applied during Alibaba's major global shopping festivals, such as Singles Day, over the years, with machines answering the large volumes of questions sent by consumers during this period, it added. 

Artificial intelligence technology have been making significant strides in recent times, with Google's AlphaGo defeating Chinese Go master Ke Jie in a best-of-three series last May, after it had defeated South Korean Go grandmaster Lee Se-dol in March the same year. 

Source