Microsoft wants to turn Windows into an AI operating system and is launching Copilot+ PCs

Microsoft wants to bring generative AI to the forefront of Windows – and the PCs it runs on.

During a pair of keynotes at the annual Build developer conference this week, the company unveiled a new line of Windows machines it's calling Copilot+ PCs, plus generative AI-powered features like Recall, which will let users find apps, files and other content they have in viewed the past. Copilot, Microsoft's brand of generative AI, will soon be much more deeply integrated into the Windows 11 experience. And new Microsoft Surface devices are on the way.

We have collected all the important announcements from Monday and Tuesday here.

Volumetric apps

Microsoft is bringing Windows Volumetric Apps – essentially spatially aware, interactive VR apps – to Meta Quest headsets. Through a partnership with Meta, Microsoft says it will deliver Windows 365 and local PC connectivity to Quest headsets, allowing developers to extend their apps into 3D space.

Image credits: Microsoft

During Tuesday's keynote, Microsoft showed off a digitally exploded 3D view of an Xbox controller from the perspective of a Meta Quest 3 headset – a digital object that the wearer could manipulate with their hands. “We are deepening our partnership with Meta to make Windows a best-in-class experience on Quest devices,” Pavan Davuluri, CVP of Windows and Devices at Microsoft, said during the demo.

Developers can do that sign up for a preview today to access Microsoft's new Volumetric API.

Copilot+ PCs

Microsoft Copilot+ PCs
Image credits: Microsoft

Copilot+ PCs are Microsoft's take on AI-first, flagship Windows hardware. They all contain special chips called NPUs to power AI experiences like Recall. And they come with at least 16 GB of RAM, paired with SSD storage.

The first Copilot+ PCs will feature Qualcomm's Snapdragon Chipmakers Intel and AMD are also committed to building processors for Copilot+ devices in collaboration with a range of manufacturers, including Acer, Asus, Dell, HP, Lenovo and Samsung.

Copilot+ PCs start at $999, and some are available for pre-order starting today.

Surface Pro and Surface Laptop

Microsoft's newly unveiled Surface devices, the Surface Laptop and Surface Pro, focus on performance and battery.

Microsoft Surface laptop
Image credits: Microsoft

The latest Surface Laptop – available with a 13.8- or 15-inch screen – has been redesigned with 'modern lines' and thinner screen bezels. It lasts up to 22 hours on a charge and is up to 86% faster than the Surface Laptop 5, the company says. It also supports Wi-Fi 7 and has a touchpad with haptic feedback.

Microsoft Surface Pro
Image credits: Microsoft

As for the new Surface Pro, Microsoft says it's up to 90% faster than the previous-generation Surface Pro (the Surface Pro 9), and will feature a new OLED with HDR display, Wi-Fi 7 (and optional 5G ) and an improved ultra-wide front camera. Additionally, the detachable keyboard – which is reinforced with additional carbon fiber – now has haptic feedback.

To remind

Windows 11's upcoming Recall feature can “remember” apps and content a user visited on their PC weeks or even months ago, helping them, for example, find a Discord chat in which they discussed the clothes they wanted to buy. Users can use Recall's timeline to “scroll back” to see what they've worked on in the recent past and zoom into files such as PowerPoint presentations to surface information that may be relevant to their searches.

Microsoft says Recall can create associations between colors, images and more, allowing users to search for virtually anything on their PC using natural language (not unlike startup Rewind's technology); developers can improve Recall by adding contextual information to their apps. And Microsoft claims that all user data associated with Recall is kept private and on-device – and more importantly, not used to train AI models.

Here is more from Microsoft: “Your snapshots are yours; they remain local on your PC. You can delete individual snapshots, adjust and delete time ranges in Settings, or pause at any time, right from the notification area icon on your taskbar. You can also prevent apps and websites from ever being saved.”

Image editing and live translations

There's now more AI in Windows than ever, and some of it exclusive to the new Copilot+ PCs.

A new feature called Super Resolution can restore old photos by upscaling them automatically. And Copilot can now analyze images to give users ideas for creative compositions. Through a feature called Cocreator, users can generate images and also ask the AI ​​model to track what they draw to change or restyle the image.

Live subtitles with live translation
Image credits: Microsoft

Elsewhere, Live Captions with live translations translates all audio passing through a PC (whether from YouTube or a local file) into the language of the user's choice. Live translations initially support approximately 40 languages, including English, Spanish, Mandarin and Russian.

A separate but related new feature in Microsoft Edge offers real-time video translation on sites like LinkedIn, YouTube, Coursera, Reuters, CNBC, Bloomberg, and more. The feature, which will be available in the near future, will translate spoken content live via both dubbing and subtitles.

Team Copilot and expansions

Team Copilot is the latest extension of Microsoft's growing Copilot suite of generative AI technology. It integrates with Teams, the company's video conferencing app, to help manage meeting agendas and create notes that everyone in a meeting can co-author. And it extends to Loop and Planner, Microsoft's collaboration and scheduling platforms, to create and assign tasks, track deadlines, and notify team members when their input is needed.

copilot
Image credits: GitHub/Microsoft

In somewhat related Copilot news, Microsoft has launched (in private preview) Copilot Extensions, allowing developers to extend GitHub's code generation tool GitHub Copilot with third-party apps and skills. Launch partners include DataStax, Docker and LambdaTest; extensions will be available on the GitHub Marketplace, but developers can also create their own private extensions to integrate with their internal systems and APIs.

Windows Copilot runtime

Microsoft Windows Copilot Runtime
Image credits: Microsoft

Capabilities like Recall and Super Resolution are powered by the Windows Copilot Runtime, a collection of ~40 generative AI models that are part of what Microsoft describes as “a new layer” of Windows. Together with the Semantic Index, a vector-based system local to an individual Copilot+ PC, the Windows Copilot Runtime allows generative AI-powered apps (including third-party apps) to run without necessarily requiring an internet connection.

“[The runtime] consists of ready-made AI APIs such as Studio Effects, Live Captions translations, OCR, User Activity Recall and [more]which will be available to developers in June,” Davuluri said on Tuesday.

Microsoft says CapCut, the popular video editor from TikTok owner ByteDance, will use the Windows Copilot Runtime and its new Windows Copilot Library, a set of APIs and AI development tools, to accelerate its AI features. And Meta will add the aforementioned Studio Effects to WhatsApp to provide features like background blur and eye contact during video calls.

Improved bot builders

Azure AI Studio, the toolset within Microsoft's Azure OpenAI Service that lets customers combine an AI model and build an app that “reasons” on that data, will soon allow developers to create apps using pay-as -you-go' inference APIs – the APIs that allow developers to access and refine generative AI models hosted on Azure infrastructure. Microsoft calls this 'model-as-a-service' and will start with models from Nixtla and Core42.

In the adjacent Copilot Studio product suite, Microsoft is launching Copilot agents, which the company describes as AI bots that can “independently orchestrate tasks tailored to specific roles and functions.” (Copilot Studio provides tools to connect Copilot for Microsoft 365, the AI-powered “copilot” in apps like Excel and Word, to third-party data.) Using memory and context knowledge, Copilot agents can navigate through different navigate types of business workflows, learn from user feedback, and ask for help when they encounter situations they don't know how to handle.

Snapdragon developer kit

Qualcomm development kit
Image credits: Microsoft

There's a new dev kit from Qualcomm aimed at developers building apps for Copilot+ PCs with arm chips

The $899.99 Snapdragon Dev Kit for Windows (which, by the way, measures about the same width, height, and length as Apple's Mac Mini) includes Qualcomm's Snapdragon IO. The Dev Kit supports Wi-Fi 7 and Bluetooth 5.4, and can control up to three 4K monitors simultaneously via the various USB-C and HDMI ports.

Phi-3

Microsoft has announced a new addition to its Phi generative AI model family, Phi-3-vision, which can perform common visual analysis and reasoning tasks such as answering questions about graphs and images. The model can read both text and images and is efficient enough to run on a mobile device.

Phi-3-vision is available in preview, while the previously announced counterparts to the text-only model – Phi-3-mini, Phi-3-small and Phi-3-medium – are now generally available.

Collaboration with Khan Academy

Microsoft has partnered with Khan Academy to donate access to cloud computing infrastructure, allowing Khan Academy to offer teachers in the US free access to Khan Academy's AI-powered tools. The two companies will also work together to explore opportunities to improve AI apps for math tutoring through generative AI, Microsoft announced Tuesday.

We're launching an AI newsletter! To register here to receive it in your inbox from June 5.

Related Posts

  • Technology
  • June 17, 2024
  • 2 views
  • 14 minutes Read
Can we prevent a bird flu pandemic in humans?

A highly pathogenic subtype of avian influenza (or bird flu), known as H5N1 is spreading among animals worldwide. For the past couple of decades, global health agencies have kept tabs…

  • Technology
  • June 17, 2024
  • 3 views
  • 10 minutes Read
Top 10 Node.js Frameworks To Consider In 2024

by Rahul Gauswami Node.js frameworks play a crucial role in streamlining the web and API development process. Node.js offers a wide variety of frameworks, each with unique features and benefits.…

Leave a Reply

Your email address will not be published. Required fields are marked *

You Missed

Euro 2024 schedule, standings, scores, live stream: how to watch Ukraine play Romania and France play later

  • June 17, 2024
Euro 2024 schedule, standings, scores, live stream: how to watch Ukraine play Romania and France play later

Can we prevent a bird flu pandemic in humans?

  • June 17, 2024
Can we prevent a bird flu pandemic in humans?

Elektrische voertuigen komen naar voren als brandpunt bij de verkiezingen van 2024

  • June 17, 2024
Elektrische voertuigen komen naar voren als brandpunt bij de verkiezingen van 2024

Carl Maughan, Kansas lawmaker arrested in March, has had his law license suspended due to conflict of interest in murder case

  • June 17, 2024
Carl Maughan, Kansas lawmaker arrested in March, has had his law license suspended due to conflict of interest in murder case

Why do we get weather-related headaches? Explain weather whiplash

  • June 17, 2024
Why do we get weather-related headaches?  Explain weather whiplash

Top 10 Node.js Frameworks To Consider In 2024

  • June 17, 2024
Top 10 Node.js Frameworks To Consider In 2024

Trade fair today: live updates

  • June 17, 2024
Trade fair today: live updates

Immigrants help keep job growth high while inflation cools

  • June 17, 2024
Immigrants help keep job growth high while inflation cools

Do you have a happy place in LA? We want to hear from you

  • June 17, 2024
Do you have a happy place in LA?  We want to hear from you

Surgeon General Urges Safety Labels for Young People on Social Media

  • June 17, 2024
Surgeon General Urges Safety Labels for Young People on Social Media

Tinybird raises another $30 million to turn data into real-time APIs

  • June 17, 2024
Tinybird raises another $30 million to turn data into real-time APIs