Do you like playing Minecraft, but don’t really have time to break down thousands of blocks of stone and wood for your next monumental building? Then maybe you’re lucky, because in the future, a well-trained AI could do that for you. Responsible for this is OpenAI, a research company that studies artificial intelligence. OpenAI is known for the development of FROM AND 2an AI that creates images based on text input.
One free variant of DALL E By the way, you can try it yourself, the results can be funny, but also quite disturbing. But back to the topic: The researchers’ goal was to teach the AI actions not through voice input, but through video material. Here you can find out how they did it and what artificial intelligence can do.
Fed with 70,000 hours of gameplay
To teach the AI Minecraft, the researchers proceeded in several steps – you can find the full report here in her blog entry. First you had to make sure that the AI could learn from video material from YouTube at all. Because the actions carried out there are not specifically described: the AI cannot know which keys, mouse clicks and movements players use to move, open their inventory and break down blocks.
For this reason, we first recorded our own gameplay with the respective mouse and keyboard inputs. With this data, a Inverse Dynamics Model
trained to predict the next action in the video. It could access both past and future frames of the video. The model was then used in turn to match 70,000 hours of blank video with the appropriate mouse and keyboard inputs.
But not only the AI, Minecraft is also constantly evolving, most recently with the update The Wild
. In an interview, the developers explain to us how the game should still keep you busy in 50 years:
Future of Minecraft
How it wants to remain exciting for another 50 years
AI even builds diamond pickaxes
Now you could feed the AI the processed video footage and have it mimic the behavior demonstrated in the gameplay. For example, she managed to cut down trees, use them to make boards and finally a workbench. This may not sound impressive at first, but it requires around 1,000 inputs from the player, all of which have to be mimicked by the AI.
The artificial intelligence learned more human
Skills such as swimming, hunting and eating animals, and even pillar jumping. But this was far from the end for the researchers.
link to YouTube content
The AI now had some basic abilities, but should use Fine-Tuning
learn to use them more purposefully. So she was given gameplay from newly started Minecraft worlds to get skills from the Early Game
to strengthen. As a result, the AI even learned to craft wooden and stone tools, build (really very) rudimentary shelters, and loot chests in NPC villages.
Using another method of fine-tuning, the Reinforcement Learning
it was even possible at the end of the AI to craft a diamond pickaxe! And this despite the fact that she carelessly digs straight down! For comparison: While a workbench required around 1,000 actions, humans need around 24,000 actions to craft a diamond tool, all of which were performed by the artificial intelligence.
Such a competent AI is not available to Minecraft players, but you can still install many practical mods. We present the best in our article:
Minecraft
The 20 best and strangest mods of 2022
What’s the point?
As you can probably already guess, the research is not about practical AI colleagues for stressed Minecraft players. Rather, artificial intelligence should learn actions using the numerous videos available on the Internet. The researchers write in their blog entry that VPT (their technology) offers the exciting opportunity to learn large-scale behavioral patterns directly in areas other than language
.
Minecraft is excellent for these experiments because it has a very generic interface with a mouse and keyboard and offers a very open world with many possibilities. The results achieved there can be easily transferred to similar areas, such as the general operation of computers.
What do you think of the AI researchers’ Minecraft experiment? Do you find artificial intelligence boring and don’t expect much from it or do you believe in its future importance? Write us your opinion in the comments!
