The Bing Search team shared how it helped make Bing Search and Bing’s Deep Search faster, more accurate and more cost-effective by transitioning to SLM models and the integration of TensorRT-LLM.
Bing wrote, “to improve efficiency, we trained SLM models (~100x throughput improvement over LLM), which process and understand search queries more precisely.”
The benefits. Bing said it has helped make search better overall by bringing these three core benefits to its searchers:
Faster Search Results: With optimized inference, Bing users can enjoy quicker response times, making the Bing search experience more seamless and efficient.
Improved Accuracy: The enhanced capabilities of SLM models allow Microsoft to deliver more accurate and contextualized search results, helping bing searchers find the information they need more effectively.
Cost Efficiency: By reducing the cost of hosting and running large models, Microsoft said it can continue to invest in further innovations and improvements, ensuring that Bing remains at the forefront of search technology.
Why we care. A faster searcher experience and a more accurate search experience can help Bing become more trusted and useful to searchers. This may lead to more searchers adopting Bing Search in the future, taking search market share away from bigger players like Google.
Add Search Engine Land to your Google News feed.
New on Search Engine Land
About the author
Barry Schwartz is a technologist and a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics.In 2019, Barry was awarded the Outstanding Community Services Award from Search Engine Land, in 2018 he was awarded the US Search Awards the “US Search Personality Of The Year,” you can learn more over here and in 2023 he was listed as a top 50 most influential PPCer by Marketing O’Clock.
Barry can be followed on X here and you can learn more about Barry Schwartz over here or on his personal site.