Microsoft researchers reveal neural network with 135B parameters
Microsoft Corp. announced today that its researchers have developed a neural network with 135 billion parameters and deployed it in Bing to improve search results for users.
With 135 billion parameters, the neural network is described as the largest “universal” artificial intelligence the company has in production. It is also one of the most sophisticated AI models ever publicly detailed. The largest neural network built to date, OpenAI LLC’s GPT-3 natural language processing model, has 175 billion parameters.
Parameters are the configuration parameters that define how an AI performs a task. Basically, each of these parameters is information that helps the neural network determine the best way to perform calculations. The more parameters an AI has, the better it can perform the task for which it was developed.
Microsoft researchers call the 135 billion parameters AI detailed today MEB. It analyzes the queries that users enter into the company’s Bing search engine and helps identify the most relevant pages on the web. MEB does not perform the task entirely on its own, but rather shares the work with a set of other machine learning algorithms included in Bing.
“MEB works in production for 100% of Bing searches, in all regions and all languages. It is the largest universal model that we serve at Microsoft, and it demonstrates an excellent ability to memorize facts, ”detailed a group of Microsoft researchers in a report. blog post today.
With its over one hundred billion parameters, MEB takes a unique approach to determining whether or not a page is relevant to the user’s query.
Neural networks make a decision by weighing a large number of factors about the data they process. These factors are called characteristics. For example, a revenue forecasting AI can use average daily store sales as a feature to help it generate a forecast for a company’s quarterly revenue. Neural networks used to power search engines like Bing rely on the same technique to determine whether or not a page is relevant to a user’s query.
Normally, neural networks powering search engines rely on a specific type of functionality called digital functionality to help them make decisions. There are usually a few thousand digital characteristics which are manually defined by the developers. Microsoft’s MEB model, on the other hand, uses not a few thousand but rather billions of features to decide whether web content is relevant to the keywords a user has searched for in Bing.
AI sets itself apart from other neural networks in another way. It doesn’t use many features like many search-driven AI algorithms, but rather binary features, which Microsoft has found to more accurately capture information about the relevance of web pages to user queries. Specifically, the binary features allow MEB to determine that a web page contains relevant information even if it does not include the same or similar keywords as the ones the user included in their search query.
“For example, MEB learned that ‘Hotmail’ correlates strongly with ‘Microsoft Outlook’, even though they are not close to each other in terms of semantic meaning,” Microsoft researchers explained. “MEB takes up a nuanced relationship between these words: Hotmail was a free webmail service provided by Microsoft which later changed its name to Microsoft Outlook. “
After adding MEB to Bing, Microsoft saw a 2% increase in click-through rates on early search results. The company also saw a reduction of more than 1% in the number of times users rewrite a query because Bing didn’t return any relevant results.
“The model is updated daily by constantly training with the latest daily click data,” Microsoft researchers detailed. “To avoid the negative impact of obsolete features, an automatic expiration policy checks the timestamp of each feature and filters out features that have not shown in the past 500 days. After continuous training, the daily deployment of the updated model is fully automated.
Show your support for our mission by joining our community of Cube Club and Cube Event experts. Join the community that includes Amazon Web Services and will soon be Amazon.com CEO Andy Jassy, Dell Technologies Founder and CEO Michael Dell, Intel CEO Pat Gelsinger and many other luminaries and experts.
We’re hosting our second cloud startup showcase on June 16. Click here to join the free and open Startup Showcase event.
We really want to hear from you. Thank you for taking the time to read this post. We look forward to seeing you at the event and at Club CUBE.