One of the most anticipated developers’ conferences worldwide, Meta Connect 2024, was held on September 25th, 2024. The buzz and buildup to this conference had developers on the edge of their seats, though a few leaks threatened to dampen the excitement. Despite the spoilers, the conference came and went, leaving us with plenty to digest. Here’s everything you need to know about it from a developer’s perspective.
This year’s event not only met expectations but redefined what’s possible in the realm of AI and XR.
Meta is nowhere close to slowing down just yet. There’s no slowing down, with more image generator prompts in your feed, celebrity chatbot voices, and a new AI model: Llama 3.2, which has been incorporated into the platforms it owns.
Meta Quest 3S: Redefining the VR Experience
Meta has just launched the Quest 3S, quickly gaining attention as the most affordable mixed-reality headset on the market. This device offers the perfect balance of power and value at $299.99 for 128 GB and $399.99 for 256GB. The key features that brighten the arrival of such a device as we bask in the developments of AR and AI are as follows:
Advanced Mixed Reality Sensors:
The Quest 3S takes a page from its predecessor, the Quest 3, but cranks it to eleven. These sensors don’t just detect the environment; they create a seamless blend of virtual and physical worlds. As an ML enthusiast, I’m fascinated by the potential for real-time environment mapping and object recognition algorithms at play here.
Customizable Action Button:
This seemingly simple addition opens up possibilities. I’m already imagining the machine learning models that could adapt this button’s functionality based on user behavior patterns. It’s personalization at its finest.
Variable Refresh Rate Display:
This is where the rubber meets the road for immersive experiences. The display’s ability to dynamically adjust its refresh rate isn’t just about smooth visuals; it’s about optimizing performance and power consumption. I’d love to see the algorithms behind this balancing act.
Powerhouse Processor:
Sharing the same processor as the Quest 3 means serious computational muscle. This is crucial for running complex AI models locally, potentially enabling more responsive and privacy-conscious VR experiences.
Ray-Ban Meta Smart Glasses
The next-gen Ray-Ban Meta smart glasses go beyond where fashion meets cutting-edge AI. This new update improves these glasses.
No More Repeating “Hey Meta”
Meta has made interacting with their AI smoother. You no longer need to say “Hey Meta” for follow-up questions, and you can ask about objects you’re looking at without the “look and” phrase.
Memory and Reminders
The smart glasses can now remember helpful information like where you parked your car and can set reminders for you.
QR Code Scanning & Text Translation
You can now ask Meta AI to scan QR codes, make phone calls from flyers, and translate text on posters or books in real-time.
Voice Messaging on WhatsApp & Messenger
No more reaching for your phone—Meta AI can now record and send voice messages through WhatsApp and Messenger.
Real-Time Speech Translation
Real-time speech translation is coming soon. It currently supports Spanish, French, and Italian, with more languages on the way.
Orion: The Holy Grail of AR
AR glasses represent the next big leap in human-oriented computing by offering digital experiences that go beyond the limitations of smartphone screens. With their expansive holographic displays, users can transform the physical world into an interactive canvas, placing 2D and 3D content wherever they want. This freedom enables richer and more immersive digital interactions.
Orion is nothing short of a marvel in AR innovation. This sleek device is a testament to extreme miniaturization, with components packed down to fractions of millimeters. It took dozens of groundbreaking innovations to reduce the design into something so compact that you’d feel comfortable wearing it daily. But despite its size, Orion offers unmatched capability.
Here’s what makes it an important thing to add to your wishlist:
Generative AI with Visual Integration
Orion elevates the AI functionality found in Ray-Ban Meta smart glasses by adding a visual layer to what you’re looking at. For example, Meta AI can visually label objects and provide instructions in real-time, such as identifying ingredients and generating a smoothie recipe directly in your field of view.
Shared Holographic Experiences
Orion allows users to interact with the same virtual objects simultaneously. In a demo, I played a 3D version of Pong with Zuckerberg using hand tracking to control the paddles. The glasses pair via QR code and deliver smooth, lag-free multiplayer experiences, opening the door for immersive social AR interactions.
Immersive Communication
The glasses support video calls through apps like Messenger. While avatars that track facial movements are still developing, Orion enables users to see and hear contacts in real time with floating 2D windows, enhancing virtual communication.
Neural Wristband Control
Orion has a neural wristband that interprets hand gestures through electromyography (EMG). This enables users to control virtual objects effortlessly by pinching fingers or making small gestures, translating neural signals into input precisely and quickly.
Expansive Field of View
With a 70-degree field of view, Orion provides a more immersive experience than other AR glasses. Virtual objects maintain their presence as you move, anchoring themselves in the physical world, and keeping the AR content fully engaging.
Lightweight Design & Heat Management
Weighing only 98 grams, Orion is lighter than most mixed-reality headsets. Yet it is built with magnesium frames to distribute heat evenly, making it comfortable for extended use without sacrificing performance.
Seven-Camera Setup for AR Accuracy
Seven cameras are embedded in the glasses to anchor virtual objects in real space, assist with eye and hand tracking, and enhance the accuracy of Meta’s AI assistant. This ensures seamless interaction between the physical and digital worlds.
Meta AI: The Foundation for Next-Gen Applications
The company unveiled groundbreaking updates to its AI capabilities across messaging platforms, including WhatsApp, Messenger, Facebook, and Instagram Direct Messages. These innovations focus on enhancing user interactions through real-time voice and photo capabilities with Meta AI.
One of the standout features is the introduction of voice conversations with Meta AI. Users can now engage in verbal exchanges by activating a waveform button, asking questions, and receiving spoken responses.
To make this experience even more engaging, Meta plans to roll out various voice options, including those of celebrities like Awkwafina, Dame Judi Dench, John Cena, Keegan-Michael Key, and Kristen Bell. This new functionality aims to create a more accessible and interactive AI experience, allowing users to seek answers, explanations, and even jokes through audio.
Let’s take a brief look at the AI capabilities that was announced at the Meta connect 2024:
1. Voice Interactions with Meta AI
Users can engage in real-time voice conversations across WhatsApp, Messenger, Facebook, and Instagram Direct Messages.
A waveform button allows users to ask questions and receive spoken responses verbally.
Upcoming voice options will feature celebrity voices, including Awkwafina, Dame Judi Dench, John Cena, Keegan-Michael Key, and Kristen Bell.
2. Photo-Based AI Capabilities
Users can send images to Meta AI on WhatsApp for analysis and information retrieval.
New use cases include translating foreign language menus and providing care instructions for plants based on user-submitted photos.
3. Enhanced Services for Businesses
Meta is expanding its AI services within the WhatsApp Business app to improve customer service efficiency.
The initial rollout will involve thousands of businesses, with plans for broader service expansion over the next year.
4. User Experience Focus
The updates enhance user experience, facilitating idea exploration and improving chat interactions.
Positive feedback has been reported since the introduction of AI features on WhatsApp last year.
5. Commitment to Ongoing AI Advancements
Meta emphasizes its commitment to developing new AI features and encourages user feedback on Threads.
These developments reflect Meta’s strategy to integrate AI across its ecosystem, impacting user engagement and business operations.
6. Future Implications
The introduction of voice and image-based AI interactions represents a significant shift in how users interact with messaging apps.
Long-term implications for user privacy, data usage, and the messaging landscape are yet to be determined.
These AI advancements provide powerful new tools for creating more intelligent and responsive applications. However, they also become responsible for ethical AI implementation and data privacy considerations.
Takeaways
While the AI hardware landscape has posed challenges for some tech companies—such as the Humane Pin’s returns surpassing sales and user complaints about Rabbit R1 devices—Meta’s smart glasses present a more optimistic scenario. These glasses are not only stylish but also come at an affordable price point. Equipped with advanced AI features, they can identify objects and provide wearers with detailed information about them. Additionally, they seamlessly integrate with Meta AI, allowing users to translate menus or craft Instagram captions. As we look ahead, we’re excited to explore what other AI enhancements may be on the horizon for Meta’s smart glasses.
Meta Connect 2024 has set the stage for the next generation of AI and XR applications. It’s an exciting time to be a developer in this space, with challenges and opportunities that will push the boundaries of what’s possible in technology. As we move forward, our creativity and innovation will play a crucial role in shaping the future of human-computer interaction.