Reinforcement Learning in Games and Entertainment

Modeling real-world problems

When we consider actual tasks performed by humans in the real world, we can define some descriptors. In any task, there will always be an Agent, which is the human performing the task, a Goal of the task, and a Reward or Penalty for completing or failing the task. In order to complete the task, the Agent needs to perform a series of Actions on the Environment which contains external forces that either help or obstruct the Agent while keeping track of its State.

  1. At t₀, the Agent does not know what Action to take so it can take a random Action, or other strategies if there is preliminary knowledge.
  2. At a timestep tₜ, the Agent performs an Action (Aₜ)
  3. At the next timestep (tₜ ₊ ₁) the Agent perceives its new State (Sₜ ₊ ₁) and also considers the Rewards (Rₜ ₊ ₁) it got from the Environment as the result of its Action.
  4. If the Rewards get smaller then the Agent will adjust to choose a different Action
  5. This process is repeated until the Agent completes running its episode

Reinforcement Learning applications in the game industry

From the examples above, it is easy to notice that this definition of reinforcement learning fits perfectly with many real-world problems, and because games are often based on real-world scenarios, games become a perfect application of reinforcement learning. We have shown an example of an application in a simple game in the previous section and here we will see how it is used in the gaming industry.

Automated game testing

Automated game tuning

Another common usage of reinforcement learning is in NPC or enemy behavior tuning. For example, in a fighting game developers normally want to create various characters with different skills and abilities but overall evenly matched. Reinforcement learning can help by automating the fight between AI characters and taking the win/ lose statistics. If the statistics are heavily skewed towards a certain character, the developers can adjust that character’s abilities and damage dealt. All these testings can be done automatically so it will remove a greater amount of burden from testers.

Industry and market conditions

Unfortunately, deploying Reinforcement learning on actual game development workflows have quite considerable challenges, and even more to create a common platform or best practices. At the moment, although most game companies are starting to develop some kind of machine learning solutions in-house, there are no major players who provide Reinforcement learning services to the game industry. We believe the following factors are the greatest obstacles to widely implement a common Reinforcement learning platform/pipeline in the game industry.

Challenges in implementing Reinforcement learning

Reinforcement learning behavior is unpredictable

What gaming companies can do to mitigate the unpredictability of reinforcement learning

Game developers can apply reinforcement learning to non-critical aspects of the game. Enemy behavior for the most common low-level enemies, testing but only on limited areas of the game, etc. are good candidates for limited implementation of reinforcement learning.

What non-gaming companies are doing to spur the growth of Reinforcement learning

Since game companies are reluctant to share code and knowledge, many companies have started providing tools and platforms to help other companies build their own reinforcement learning systems.

Recap

In this article, we discussed the basics of reinforcement learning and how it mimics most real-world problems. Then we moved on to look at some applications of reinforcement learning in the gaming industry and the challenges of implementing it in-game development workflows. Finally, we mentioned some ways to mitigate the challenges and described the situation of reinforcement learning adoption in the industry.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Cinnamon AI

Cinnamon AI

Cinnamon AI is the pioneer company in consulting and designing innovative solutions using AI technology for business. Follow our official Medium for more :)