Microsoft Enhances Windows Copilot with "Add a Screenshot" Feature for Intuitive Assistance

Microsoft is gearing up to transform the functionality of Windows Copilot by introducing a new feature called "Add a Screenshot." Although Windows Copilot is currently relatively modest in its offerings, this update is expected to significantly enhance its capabilities.

source: Microsoft

The "Add a Screenshot" feature enables users to capture their screen and prompt the AI to provide explanations or insights. This feature, now rolling out to the general public, adds a new button to the Copilot panel, allowing users to directly upload a screenshot for analysis. Once uploaded, users can leverage Bing Chat to engage in discussions related to the captured image.

This functionality is akin to the existing image upload feature on copilot.microsoft.com or Bing.com's chat, but with a more streamlined focus on screenshots. Users can seek explanations or information about any aspect of the screenshot, making it a valuable tool for various applications.

As an example, a demonstration involving a Recycle Bin screenshot showed Copilot quickly providing a concise description of the image. The integration is optimized for efficient handling of screenshots.

Furthermore, the screenshot feature, powered by Microsoft Edge, allows users to annotate the screenshot with drawings or specific instructions that Copilot can interpret.

For users with access to ChatGPT-4 Turbo, an advanced version, there's an added capability to inquire about emotions, even in abstract images. While currently available to select users, Microsoft plans to expand the rollout of this advanced feature in the coming weeks.

Microsoft is positioning Copilot as a central feature across its products, with plans to introduce a dedicated Copilot button to Windows hardware later in 2024. Described as the most significant upgrade to the Windows keyboard in nearly three decades, Microsoft envisions Copilot as the entry point into the world of AI on the PC. New-generation hardware is expected to ship with the Copilot key, emphasizing Microsoft's commitment to making AI-driven assistance accessible and integral to the computing experience.