AI Weekly: 11/27/23
Sam Altman returns as CEO, Stability AI announces a video model, and AI Weekly experiments with a new approach
Good morning and welcome to this week’s edition of AI Weekly! In this week’s news, Sam Altman has returned to OpenAI as CEO *sigh of relief*. This decision came after over 90% of OpenAI’s employees signed an open letter stating their intent to resign if Altman was not brought back as CEO. Early last week, OpenAI’s employees collectively took to X to voice that the company is “nothing without its people,” and the board clearly recognized this sentiment.
In other news, Stability AI has announced Stable Video Diffusion, an AI model that generates videos by animating existing images. This product enters the market alongside Runway’s Gen-2 model and Pika Labs’ model. Given that we are still in the early days of high quality video models, it remains unclear what attributes users will look for when choosing a model to use in production.
You’ll notice that aside from this week being slower due to Thanksgiving, I’ve also experimented with a bit less content within each story. My goal is to make this newsletter the most valuable version of itself, and I look forward to hearing whether this shift makes AI Weekly more readable and digestible. Happy reading!
- ZG
Here are the most important stories of the week:
OPENAI/ALTMAN SAGA
Sam Altman has returned to OpenAI as CEO. Link.
His recent ousting involved tensions between OpenAI's for-profit and nonprofit sides, with some board members and employees allegedly concerned about the focus on profit and product development over AI safety.
Microsoft, a major investor in OpenAI, played a role in bringing Altman back to lead a new advanced AI research team at Microsoft.
Altman's return as CEO was officially announced after negotiations, with a new board of directors appointed to the company. Many questions about the reasons for Altman's initial firing remain unanswered.
OpenAI CEO Sam Altman's firing is reportedly linked to a mysterious AI model called Q* that was developed by a team led by OpenAI chief scientist Ilya Sutskever. Link.
The model, which can solve basic math problems, created internal controversy, with some staff members expressing concerns about its potential threat to humanity.
Experts believe that Q* represents a significant advancement in AI as it can logically reason and generalize beyond its training data, a challenge for existing models.
Q* combines deep learning techniques with human-programmed rules and could potentially lead to artificial general intelligence (AGI) by allowing the model to have new ideas and solve unseen problems.
TEXT
Anthropic has reduced the per-token pricing of its flagship LLM, Claude 2.1, to stay competitive in the growing conversational AI market. Link.
The proliferation of open-source AI models, such as Mistral and Poro, provides companies with more options and customization potential, challenging closed-source vendors like Anthropic.
Open source AI tools can significantly reduce costs and offer greater customization for enterprises, making them an attractive option for companies investing in AI.
To maintain leadership in conversational AI, companies must adapt to market shifts, respond to competition, and consider a balance between proprietary and open-source approaches to stay competitive and innovative.
Researchers at ETH Zurich have developed a technique that significantly boosts the speed of neural networks by altering the inference process, reducing computational requirements. Link.
In experiments on BERT, a transformer model, they achieved a reduction of over 99% in computations, which can be applied to other LLMs like GPT-3.
They introduced "fast feedforward" layers (FFF) using conditional matrix multiplication (CMM) instead of traditional dense matrix multiplications (DMM) in feedforward layers, reducing computational load.
Their FastBERT model showed impressive performance on language understanding tasks while using only a fraction of feedforward neurons, opening possibilities for accelerating LLMs like GPT-3.
Microsoft Research has introduced Orca 2, a pair of small language models (7 billion and 13 billion parameters) that can match or outperform much larger language models, including Meta's Llama-2 Chat-70B, on complex reasoning tasks in zero-shot settings. Link.
These models aim to address the gap between large language models and their smaller counterparts by teaching them various reasoning techniques and strategies rather than replicating the behavior of larger models.
Orca 2 performed exceptionally well on 15 diverse benchmarks, often outperforming models that are five to ten times larger in size.
Microsoft has open-sourced the Orca 2 models to encourage further research and development of high-performing small language models, providing enterprises with cost-effective options for business applications.
IMAGE/VIDEO
Stability AI has announced Stable Video Diffusion, an AI model that generates videos by animating existing images. Link.
The model is currently in a "research preview," and users must agree to certain terms of use.
Stable Video Diffusion includes two models, SVD and SVD-XT, which can transform still images into videos with 14 or 24 frames.
The models were trained on a dataset of millions of videos, but the sources of these videos aren't entirely clear, potentially raising issues related to copyright and ethical usage rights.
Google has expanded its AI chatbot, Bard, to understand and respond to content within YouTube videos. Link.
This expansion allows users to ask Bard for specific videos and follow-up with complex questions about the video's content, including summaries and specific details.
Google has been actively enhancing Bard's features, including integrations with various Google services such as Google Flights, Google Maps, and Gmail, as well as adding a tool for users to verify Bard's responses.
Bard access has also been extended to teenagers.
Off/Script, a community-powered product creation platform, has launched a mobile app that allows users to conceptualize, share, and monetize product mock-ups. Link.
Users can design a wide range of products, and the most popular designs are funded, manufactured, and shipped by Off/Script.
The platform leverages generative AI models, including Stable Diffusion and ControlNet, to assist users in creating product mock-ups.
Off/Script has a library of customizable clothing and accessories and works with over 1,000 manufacturers to bring designs to life.
Creators earn 20% of sales and receive a $500 upfront fee, and they can also sell their items on their own websites or marketplaces.
SPEECH/AUDIO
OpenAI has made ChatGPT's voice feature available to all users for free, allowing users to interact with ChatGPT using their voice and receive audible responses. Link.
Initially, this feature was limited to paying users but has now been expanded to a broader audience.
The release coincides with internal upheaval at OpenAI, including the ousting of CEO Sam Altman and ongoing negotiations for his return.
The example provided in OpenAI's announcement humorously references the internal drama, highlighting the recent turmoil within the organization.
HARDWARE
Neuralink, the brain-machine interface company founded by Elon Musk, has raised an additional $43 million in venture capital. Link.
This funding round increased the previous one, led by Peter Thiel's Founders Fund, from $280 million to $323 million in August.
Neuralink's technology involves implanting ultra-thin threads in the brain that connect to a custom-designed chip with electrodes to read brain signals.
The company faced scrutiny over allegations of a toxic workplace culture and unethical research practices, including mistreatment of animals involved in testing.
HEALTHCARE
Nvidia is collaborating with Genentech, a Roche Group company, on using generative AI in drug discovery. Link.
The partnership aims to expedite the discovery and delivery of novel therapies by combining Genentech's models and datasets with Nvidia's AI computing stack, including DGX Cloud and BioNemo offerings.
Genentech's AI/ML teams will optimize their custom algorithms using Nvidia's DGX Cloud, with BioNemo for pre-training or fine-tuning models.
Nvidia's offerings will accelerate the training and inference of generative models, enabling iterative development of better therapies by uncovering patterns and making new predictions from extensive experimental data.
OTHER
Amazon has launched the "AI Ready" initiative with the goal of offering free AI training to two million people by 2025. Link.
The initiative includes eight free AI courses, with five courses designed for developers and technical audiences and three for business leaders and nontechnical audiences.
Some of the courses focus on generative AI, which has gained popularity with the emergence of AI-powered chatbots like ChatGPT.
Amazon also plans to provide $12 million in scholarships to over 50,000 high school and college students for courses on generative AI through Udacity, aiming to address the growing demand for AI skills in the job market.
French startup Osium AI has secured $2.6 million in seed funding for its AI-powered materials science research and development platform. Link.
The platform optimizes the feedback loop between materials formulation and testing using data-driven techniques.
Osium AI helps industrial companies predict the physical properties of new materials based on specific criteria and refines and optimizes materials development while reducing trial and error.
The company is already working with several industrial partners and aims to expand its team and client base, with discussions underway with 30 potential industrial clients.