Copilot in Edge Can Now See Your Screen and Answer Queries About It-OxBig News Network

Microsoft announced the rollout of Copilot Vision to all users in the Microsoft Edge browser last week. The computer vision-enabled artificial intelligence (AI) feature was first unveiled by the company in December 2024, and so far, it has only been available to Copilot Pro subscribers. The Redmond-based tech giant is now expanding the AI feature to all Edge browser users. With Copilot Vision, the AI chatbot can see the content of web pages and assist the user in navigating through various tasks.

Copilot Vision Is Now Available to Everyone for Free

In a post on X (formerly known as Twitter), the CEO of Microsoft AI, Mustafa Suleyman, announced the expansion of Copilot Vision to all users irrespective of whether they are a paid subscriber of Copilot or not. “It’ll think out loud with you when you’re browsing online. No more over-explaining, copy-pasting, or struggling to put something into words,” Suleyman said.

Essentially, Copilot Vision is a real-time computer vision feature, similar to Gemini Live’s screen sharing capability. The AI chatbot can see the web pages and all the content in the Edge browser, and process that information as well as the user’s prompt in real time. The feature supports voice mode, and users can simply speak what they want instead of typing it.

Microsoft has made Copilot Vision an opt-in feature, as some users may have concerns about their privacy and data security. To opt in, users can open this link in the Edge browser and follow the instructions. Once activated, the users can tap the microphone icon in the Copilot sidebar to activate Vision mode. Copilot Vision will show a floating bar with a text field and a microphone button.

Suleyman said Copilot Vision can help users in a variety of ways. It can read multiple reviews and summarise them for the user. It can describe visually confusing objects, for example, it can tell users if a couch they see on a shopping website is “roll arm” or “midcentury modern”. It can also help users in shopping by finding the right products based on natural language descriptions.

At the time of launch, Microsoft had said that even after opting in, users will have full control over when to turn it on and when not to. Copilot Vision cannot see the user’s screen when it is deactivated. Additionally, the company claims that any conversation data, including user prompts and images and content from web pages will all be deleted at the end of each session. However, Copilot’s responses are logged and used to improve the company’s safety systems.