Microsoft is using AI models to greatly improve Windows search on its new Copilot Plus PCs, including the addition of a new Click to Do feature that’s very similar to Google’s Circle to Search. These search improvements will make it easier to find and interact with images, emails, documents, and even videos and are just a few of the AI-based features coming to Copilot Plus PCs starting in November.
Technology
Microsoft is using AI to improve Windows search
Microsoft is improving its Windows search across File Explorer and Settings thanks to AI. A new Click to Do feature is also coming to Copilot Plus PCs.
The improved Windows search will first show up in File Explorer on Copilot Plus PCs next month, allowing you to search for pictures using words, even if the search word isn’t found in the photo or file name.
“AI-powered search makes it dramatically easier to find virtually anything,†says Yusuf Mehdi, executive vice president and consumer chief marketing officer at Microsoft. “You no longer need to remember file names and document locations, nor even specific names of words. Windows will better understand your intent and match the right document, image, file, or email.â€
This improved search will also be available “in the coming months†in the main Windows search interface and through the search box that appears in the Settings interface. You can type things like “add my headphones†into the Settings search box and it will help you find the right settings.
Search hasn’t been great in Windows for years, so this AI-powered natural language search should greatly improve things — as long as it works as well as Microsoft promises. Microsoft is leveraging the NPU chips on new Copilot Plus PCs to enable local search on OneDrive content without having to be connected to the internet.
Alongside Windows search, Microsoft will also start rolling out Click to Do on Copilot Plus PCs next month, which is very similar to Google’s Circle to Search feature. With Click to Do, you hit the Windows key on a keyboard and left-click on a mouse to see an interactive overlay appear on your screen, which lets you select images or text to perform clickable actions.
“Click to Do works by first understanding everything you’ve seen on your screen and enabling useful shortcuts to actions to help you more quickly search, learn, edit, shop, or act on those items,†explains Mehdi. “It works on any windows, document, image, or even video.â€
You can use Click to Do on things like a YouTube video you’re watching to do a visual search on an item that appears in the video using Bing. Click to Do is also context-aware, so it can help with text-related actions like rewriting or summarizing documents or explaining text and sending emails.
Microsoft will start testing these new search and Click to Do features with Windows Insiders on Copilot Plus PCs in October, followed by a gradual rollout in November. The previously announced Recall feature is also coming to testers in October on Qualcomm-powered devices, before being available to Windows Insiders on Intel- or AMD-powered Copilot Plus PCs in November. Microsoft says “timing details on the broad availability of Recall will be shared soon.â€
Microsoft is also adding generative fill and erase to Microsoft Paint as part of these new Windows AI features. You can read more about the Paint and Photos AI additions right here.
(function (window, document) {
window.MemberfulOptions = { site: "https://subs.theverge.com" };
const initialElement =
document.getElementsByTagName("head")[0] ||
document.getElementsByTagName("body")[0];
const script = document.createElement("script");
script.type = "text/javascript";
script.async = true;
script.src = "https://d35xxde4fgg0cx.cloudfront.net/assets/embedded.js";
script.addEventListener("load", () => {
window.MemberfulEmbedded && window.MemberfulEmbedded.setup();
});
if (initialElement) {
initialElement.appendChild(script);
}
})(window, document);