At the Google I/O conference in May this year, Pichai announced PaLM 2, a large model that benchmarks GPT-4, but also mentioned that Google’s research focus is shifting to Gemini, which is a multi-modal and efficient model. machine learning tools.
In order to develop Gemini faster, Google merged two internal artificial intelligence laboratories in April this year: Google Brain and DeepMind. This joint project of Gemini consists of a team of researchers from the two laboratories. Take the lead.
In the next few months, the mystery of Gemini was unveiled bit by bit: We generally know that the model was developed after the merger of Google Brain and DeepMind, and will have trillions of parameters like GPT-4; Gemini has already been trained Demonstrating multi-modal capabilities never seen in previous models; once fine-tuned and rigorously tested for security, Google will also provide Gemini versions of different sizes and functions to ensure deployment on different products, applications and devices .
The latest news is that, according to three people with direct knowledge, Google has allowed a small number of companies to use an early version of the Gemini software, which means that Google is about to incorporate it into consumer services and sell it to enterprises through the company’s cloud computing services.
Can it surpass GPT-4?
Recently, SemiAnalysis analysts Dylan Patel and Daniel Nishball have brought more revelations about Gemini. Dylan Patel exposed the architecture of GPT-4 on July 11 this year.
Dylan Patel and Daniel Nishball revealed that the first generation Gemini should be trained on TPUv4, and these pods did not integrate the maximum number of chips – 4096 chips, but used a smaller number of chips to ensure the reliability and reliability of the chips. Hot swappable. If all 14 pods are used at reasonable Mask Field Utilization (MFU) for about 100 days, the hardware FLOPS for training Gemini will exceed 1e26.
However, Gemini has begun training on the new TPUv5 Pod, with a computing power of up to ~1e26 FLOPS, which is 5 times greater than the computing power used to train GPT-4.
In addition, Gemini’s training database is 9.36 billion minutes of video subtitles on Youtube, and the total data set size is approximately twice that of GPT-4.
Gemini consists of a set of large language models. It may use MOE architecture and speculative sampling technology to generate tokens in advance through small models and transfer them to large models for evaluation, thereby improving the overall inference speed of the model.
Capabilities-wise, Gemini supports everything from chatbots to summarizing text or generating raw text (such as email drafts, song lyrics, or news articles) based on a description of what the user wants to read. In addition, Gemini helps software engineers write code and generate original images based on user requirements.
According to a previous report by The Information, Google hopes that Gemini will greatly improve the code generation capabilities of software developers to catch up with Microsoft’s GitHub Copilot code assistant.
Google employees have also discussed using Gemini to perform functions such as graph analysis, such as asking the model to interpret the meaning of a completed graph, and using text or voice commands to navigate a web browser or other software.
One person who has tested GPT-4 said that Gemini has an advantage over GPT-4 in at least one way: In addition to public information on the web, Gemini leverages the vast amounts of proprietary data Google obtains from its consumer products. Therefore, the model should be particularly accurate at understanding user intent for a specific query, and it appears to produce fewer incorrect answers (i.e., hallucinations).
Opportunity for Google Cloud services to catch up
Since OpenAI began selling access to GPT-4 earlier this year, Google has been actively making its existing commercial model available to more developers in recent months.
In May of this year, Google announced that it would provide PaLM 2 to Google Cloud customers through Vertex AI. Another person familiar with the matter said that Google plans to provide “Gemini” to enterprises through the Google Cloud Vertex AI service, including versions of different sizes, so that developers can choose to pay for a less complex version to handle simple tasks, or purchase A version small enough to run on personal devices.
The person added that Google is currently letting developers use the relatively large version of Gemini, but not the largest version under development, which is closer to GPT-4.
For Google, the launch of Gemini is a big deal. Google spent a lot of computing resources and manpower developing it as a tool to compete with OpenAI, hoping that the software would not only promote its cloud server rental business, but also provide support for new features from the Bard chatbot to the Workspace software.
OpenAI and other software companies such as Databricks, which helps companies develop and use artificial intelligence, also predict they will generate significant revenue from conversational AI, The Information reported. However, the rise of open source large models may diminish the focus on Google and OpenAI selling access to their proprietary models.
Maybe one day you wake up and Gemini has been officially released, and its mystery will be completely unveiled.
Can Google turn things around with Gemini? We just have to be patient and wait.