Do you like playing Minecraft, but don’t really have time to break down thousands of blocks of stone and wood for your next monumental building? Then maybe you’re lucky, because in the future, a well-trained AI could do that for you. Responsible for this is OpenAI, a research company that studies artificial intelligence. OpenAI is known for the development of FROM AND 2an AI that creates images based on text input.
One free variant of DALL E By the way, you can try it yourself, the results can be funny, but also quite disturbing. But back to the topic: The researchers’ goal was to teach the AI actions not through voice input, but through video material. Here you can find out how they did it and what artificial intelligence can do.
Fed with 70,000 hours of gameplay
To teach the AI Minecraft, the researchers proceeded in several steps – you can find the full report here in her blog entry
For this reason, we first recorded our own gameplay with the respective mouse and keyboard inputs. With this data, a Inverse Dynamics Model
trained to predict the next action in the video. It could access both past and future frames of the video. The model was then used in turn to match 70,000 hours of blank video with the appropriate mouse and keyboard inputs.
But not only the AI, Minecraft is also constantly evolving, most recently with the update The Wild
. In an interview, the developers explain to us how the game should still keep you busy in 50 years:
Future of Minecraft
How it wants to remain exciting for another 50 years
AI even builds diamond pickaxes
Now you could feed the AI the processed video footage and have it mimic the behavior demonstrated in the gameplay. For example, she managed to cut down trees, use them to make boards and finally a workbench. This may not sound impressive at first, but it requires around 1,000 inputs from the player, all of which have to be mimicked by the AI.
The artificial intelligence learned more human
Skills such as swimming, hunting and eating animals, and even pillar jumping. But this was far from the end for the researchers.
link to YouTube content
The AI now had some basic abilities, but should use Fine-Tuning
learn to use them more purposefully. So she was given gameplay from newly started Minecraft worlds to get skills from the Early Game
to strengthen. As a result, the AI even learned to craft wooden and stone tools, build (really very) rudimentary shelters, and loot chests in NPC villages.
Using another method of fine-tuning, the Reinforcement Learning
Such a competent AI is not available to Minecraft players, but you can still install many practical mods. We present the best in our article:
Minecraft
The 20 best and strangest mods of 2022
What’s the point?
As you can probably already guess, the research is not about practical AI colleagues for stressed Minecraft players. Rather, artificial intelligence should learn actions using the numerous videos available on the Internet. The researchers write in their blog entry that VPT (their technology) offers the exciting opportunity to learn large-scale behavioral patterns directly in areas other than language
.
Minecraft is excellent for these experiments because it has a very generic interface with a mouse and keyboard and offers a very open world with many possibilities. The results achieved there can be easily transferred to similar areas, such as the general operation of computers.
What do you think of the AI researchers’ Minecraft experiment? Do you find artificial intelligence boring and don’t expect much from it or do you believe in its future importance? Write us your opinion in the comments!