Two prominent artificial intelligence researchers, David Ha and Llion Jones, have recently launched a startup called Sakana AI. The Tokyo-based company aims to build a new kind of nature-inspired neural network, leveraging their expertise in the field.
David Ha, the Chief Executive Officer of Sakana AI, was previously the head of research at Stability AI Ltd. He is best known as the developer of the open-source Stable Diffusion image generation model. Before joining Stability AI, Ha led Google LLC’s AI research group in Japan, where he made significant contributions to the field.
Llion Jones, a fellow machine learning researcher, co-founded Sakana AI with Ha. Jones spent more than a decade at Google as part of the Alphabet Inc. unit. In 2017, he co-authored the groundbreaking AI research paper titled “Attention is All You Need” along with seven other Google colleagues. This paper introduced the concept of Transformers, which has become a foundational principle behind many advanced AI models, including OpenAI’s GPT-4.
The primary innovation in the technology developed by Sakana AI is a software mechanism called attention. This mechanism allows an AI model to consider a vast amount of contextual information when analyzing data and prioritize the most important details. By taking inspiration from nature, Sakana AI aims to develop generative AI models that can simulate complex patterns and behaviors found in natural systems.
As part of their development roadmap, Sakana AI plans to employ a technique known as evolutionary computation to train their AI models. This approach involves creating multiple versions of a neural network with slightly different configurations and comparing their performance to identify the most accurate and efficient models. The less optimal versions are discarded, and the process is repeated several times to further refine the AI model.
One of Sakana AI’s key goals is to avoid creating massive AI models that require extensive hardware resources to run and train. This would help reduce costs associated with running AI systems and make them more accessible to a broader range of applications. The startup also aims to make their neural networks more adaptable to changing requirements, addressing the issue of brittleness often associated with current software.
Sakana AI’s team currently consists of Ha, Jones, and a part-time researcher hired from academia. However, the startup plans to expand its workforce by recruiting additional staff in the future to accelerate research and development efforts.
It is unclear at this point whether Sakana AI has raised any outside capital. The startup enters a competitive market, with tech giants like Microsoft Corp. and other well-funded players also exploring advancements in neural network technologies. Interestingly, some of these players, including Cohere Inc., are led by former Google researchers who co-authored the “Attention is All You Need” paper with Llion Jones.
Sakana AI’s launch marks an exciting development in the field of artificial intelligence, as it brings together the expertise of two renowned researchers to explore nature-inspired neural networks. With their focus on generative AI models and the innovative use of attention mechanisms, Sakana AI has the potential to revolutionize the industry and create more efficient and adaptable AI systems.
