✅ Must See AI News This Week - DeepSeek-V3, OpenAI o3 & o3-mini, META AI personas
DeepSeek-V3 rivals GPT-4o, OpenAI announces o3 & o3-mini, Meta plans AI personas on Facebook & Instagram, OpenAI restructures as a public benefit corp.
Happy New Year, everyone! 🎉 Hope it brings you lots of joy and happiness!
In the meantime, AI developments are not sleeping, and over the holidays a lot of updates came out. Let’s dive in ⤵️
Most Interesting This Week
1️⃣ DeepSeek-V3 Rivals GPT-4o and Claude 3.5 Sonnet
DeepSeek introduced DeepSeek-V3, a 671 billion parameter Mixture-of-Experts (MoE) language model.
It activates only 37 billion parameters per token, reducing computational needs while maintaining performance.
The model was trained on 14.8 trillion tokens, requiring only 2.8 million GPU hours, significantly less than comparable models.
Key techniques include:
Auxiliary-loss-free load balancing for even workload distribution.
FP8 mixed precision for reduced memory and power usage.
Multi-token prediction (MTP) for faster processing.
DeepSeek-V3 scores 65.2% on the HumanEval benchmark, outperforming Claude Sonnet 3.5.
It excels in multilingual benchmarks like XSum and TyDi QA, competing with GPT-4o and LLaMA 3.
The model is available on Hugging Face for testing via API or a free chat interface.
API pricing is fixed and minimal until February 8, 2024.
2️⃣ OpenAI Announces o3 and o3-mini Reasoning Models
OpenAI introduced o3 and o3-mini, AI reasoning models using a "private chain of thought" approach.
o3 achieved record scores on benchmarks like the ARC-AGI visual reasoning test and graduate-level exams.
These models will be available for public safety testing and research access.
o3-mini is expected to launch in late January, followed by o3.
3️⃣ Qwen's QVQ: Open-Weight Visual Reasoning Model
Qwen released QVQ, an open-weight visual reasoning model built on Qwen2-VL-72B.
QVQ scored 70.3 on the MMMU benchmark, surpassing GPT4o and Claude Sonnet 3.5, and improved on math-related tasks.
The model excels at visual reasoning through step-by-step analysis but has limitations like language mixing and potential hallucinations in multi-step reasoning.
4️⃣ Meta's AI Personas on Facebook and Instagram
Meta plans to integrate AI-generated profiles across Facebook and Instagram, complete with bios, photos, and content creation abilities.
The company has launched trial AI character creation tools, producing hundreds of thousands of characters.
New text-to-video generation software is planned, allowing creators to insert themselves into AI-created videos.
Experts warn about potential risks, including the spread of false narratives.
AI Business Moves
1️⃣ OpenAI's Restructuring for Public Benefit Corporation
OpenAI plans to transform into a public benefit corporation (PBC).
The restructuring will convert OpenAI's for-profit arm into a Delaware-based PBC, with the original nonprofit gaining significant shares.
OpenAI claims this will result in "one of the best-resourced non-profits in history," enabling the pursuit of charitable goals in health care, education, and science.
The process follows a $6.6 billion funding round at a $157 billion valuation.
Elon Musk sued OpenAI in December to prevent the move, and California nonprofit Encode is also pushing for a pause.
2️⃣ AI Hiring Surge in 2024
ZoomInfo data shows a dramatic growth in AI-focused roles across industries.
AI-related C-suite positions have surged 428% since 2022, VP roles by 199%, and director positions by 197%.
Engineering and development roles dominate the AI job landscape.
Generative AI job titles, while only 3% of total AI positions, have increased 250x since late 2022.
Over 10,875 new AI leadership roles were created in Q2 2024 alone.
3️⃣ DeepSeek-V3's Cost-Effective Training
DeepSeek-V3 was trained in just two months at an estimated cost of $5.57 million.
This is dramatically less than the reported $500+ million spent on models like LLaMA 3.1.
The model's efficiency challenges the need for massive resources in AI development.
4️⃣ Alibaba Cloud's Price Cuts on Qwen-VL
Alibaba Cloud announced price cuts of up to 85% on its Qwen-VL visual language model.
The move aims to incentivize enterprise adoption of the Chinese tech giant's models.
AI in Action
1️⃣ Genesis: Physics Simulation for Robotics
Genesis is a new physics simulation platform for robotics and embodied AI applications.
It integrates a universal physics engine with generative AI for realistic simulations across video, 3D scenes, and robotic motions.
Genesis claims simulation speeds up to 430,000 times faster than real-time in certain scenarios.
The physics engine is open source, with the full generative framework to be released gradually.
2️⃣ ModernBERT: Updated BERT Encoder Models
Answer.AI and LightOn released ModernBERT, a family of encoder-only models outperforming older BERT-style models.
ModernBERT incorporates recent advances from LLMs, including an 8,192 token context length, improved architecture, and training on diverse data, including code.
The models aim to be drop-in replacements for BERT in applications like retrieval, classification, and entity extraction.
3️⃣ CodeLLM: Multi-Language Model Code Editor
Abacus.AI released CodeLLM, an AI-powered code editor that helps developers write, review, and refactor code.
CodeLLM provides access to multiple language models optimized for different coding tasks and automatically switches between them based on the language and query.
Integrated models include Claude Sonnet 3.5, OpenAI's o1, Qwen 72B, and others.
The Visual Studio Code-based editor offers features like code completion, code chat, and integration with ChatLLM Teams for Git functionality and pull requests.
CodeLLM is available as part of a $10 monthly subscription.
4️⃣ AI Reveals Hidden Details in Raphael Painting
Scientists used AI to confirm that Raphael's Madonna della Rosa wasn't painted entirely by him.
An AI system trained on authenticated Raphael paintings was 98% accurate in identifying his genuine works.
The AI revealed St. Joseph's face was likely painted by another artist, possibly Giulio Romano.
This demonstrates AI's potential in art authentication, conservation, and historical research.