Google’s experimental Project Astra, a significant upgrade to its Gemini AI assistant, is starting its phased rollout to Android users. Initially announced at MWC 2025, this innovative feature promises to revolutionize how users interact with their devices, offering real-time analysis of text, audio, images, video, and even live camera feeds.
alt text: A smartphone displaying the Gemini assistant with the "Share Screen With Live" option highlighted.
Early access users have already begun sharing their experiences, with one Reddit user demonstrating the new “Share Screen With Live” option within the Gemini Assistant interface. This functionality, along with new options for live video and screen sharing within Gemini Live, has also been confirmed by Google to The Verge. Previously, Gemini’s on-screen awareness was limited to the “Ask about screen” feature.
Understanding the Power of Project Astra
Project Astra represents a leap forward in AI assistance. Described by Google Deepmind’s research director, Greg Wayne, as akin to “a little parrot on your shoulder,” it offers contextual understanding of on-screen content in real time.
With the “Share Screen With Live” option enabled, Gemini can analyze the content displayed in any app, responding to queries based on the information presented. Imagine asking Gemini to summarize an article you’re reading, describe the action in a mobile game, or explain a scene during video playback.
alt text: A user interacting with the Gemini Live app, using the camera to focus on a book passage.
Real-World Applications with Gemini Live
Even more impressive is Gemini Live’s enhanced world understanding. By activating the video feed option within Live mode, users can point their camera at objects, allowing Gemini to see, comprehend, and answer questions based on its visual input. This opens up a plethora of possibilities, from learning more about historical landmarks to getting advice on interior design or even solving math problems written on a whiteboard. This technology aligns with similar advancements like Apple’s Visual Intelligence and the open-source HuggingSnap app.
Availability and Access
While promising, access to Project Astra appears limited for now. The feature seems restricted to Gemini Advanced subscribers, typically bundled with the Google One AI Premium subscription. Even with a qualifying subscription, the rollout remains gradual, with some users on the latest Android versions still awaiting access.
The Future of AI Assistance
Project Astra marks a significant milestone in the evolution of AI assistants. Its real-time interaction with both digital and physical environments offers a glimpse into a future where AI becomes an even more integrated and helpful part of our daily lives. As the rollout continues, we can expect to see even more innovative applications of this groundbreaking technology emerge.
