How to create a new gym environment in OpenAI?

I have an assignment to make an AI Agent that will learn to play a video game using ML. I want to create a new environment using OpenAI Gym because I don't want to use an existing environment. How can I create a new, custom Environment?

Also, is there any other way I can start to develop making AI Agent to play a specific video game without the help of OpenAI Gym?

Solution 1:

See my banana-gym for an extremely small environment.

Create new environments

See the main page of the repository:

https://github.com/openai/gym/blob/master/docs/creating_environments.md

The steps are:

1. Create a new repository with a PIP-package structure

It should look like this

gym-foo/
  README.md
  setup.py
  gym_foo/
    __init__.py
    envs/
      __init__.py
      foo_env.py
      foo_extrahard_env.py

For the contents of it, follow the link above. Details which are not mentioned there are especially how some functions in foo_env.py should look like. Looking at examples and at gym.openai.com/docs/ helps. Here is an example:

class FooEnv(gym.Env):
    metadata = {'render.modes': ['human']}

    def __init__(self):
        pass

    def _step(self, action):
        """

        Parameters
        ----------
        action :

        Returns
        -------
        ob, reward, episode_over, info : tuple
            ob (object) :
                an environment-specific object representing your observation of
                the environment.
            reward (float) :
                amount of reward achieved by the previous action. The scale
                varies between environments, but the goal is always to increase
                your total reward.
            episode_over (bool) :
                whether it's time to reset the environment again. Most (but not
                all) tasks are divided up into well-defined episodes, and done
                being True indicates the episode has terminated. (For example,
                perhaps the pole tipped too far, or you lost your last life.)
            info (dict) :
                 diagnostic information useful for debugging. It can sometimes
                 be useful for learning (for example, it might contain the raw
                 probabilities behind the environment's last state change).
                 However, official evaluations of your agent are not allowed to
                 use this for learning.
        """
        self._take_action(action)
        self.status = self.env.step()
        reward = self._get_reward()
        ob = self.env.getState()
        episode_over = self.status != hfo_py.IN_GAME
        return ob, reward, episode_over, {}

    def _reset(self):
        pass

    def _render(self, mode='human', close=False):
        pass

    def _take_action(self, action):
        pass

    def _get_reward(self):
        """ Reward is given for XY. """
        if self.status == FOOBAR:
            return 1
        elif self.status == ABC:
            return self.somestate ** 2
        else:
            return 0

Use your environment

import gym
import gym_foo
env = gym.make('MyEnv-v0')

Examples

1. https://github.com/openai/gym-soccer
2. https://github.com/openai/gym-wikinav
3. https://github.com/alibaba/gym-starcraft
4. https://github.com/endgameinc/gym-malware
5. https://github.com/hackthemarket/gym-trading
6. https://github.com/tambetm/gym-minecraft
7. https://github.com/ppaquette/gym-doom
8. https://github.com/ppaquette/gym-super-mario
9. https://github.com/tuzzer/gym-maze

Solution 2:

Its definitely possible. They say so in the Documentation page, close to the end.

https://gym.openai.com/docs

As to how to do it, you should look at the source code of the existing environments for inspiration. Its available in github:

https://github.com/openai/gym#installation

Most of their environments they did not implement from scratch, but rather created a wrapper around existing environments and gave it all an interface that is convenient for reinforcement learning.

If you want to make your own, you should probably go in this direction and try to adapt something that already exists to the gym interface. Although there is a good chance that this is very time consuming.

There is another option that may be interesting for your purpose. It's OpenAI's Universe

https://universe.openai.com/

It can integrate with websites so that you train your models on kongregate games, for example. But Universe is not as easy to use as Gym.

If you are a beginner, my recommendation is that you start with a vanilla implementation on a standard environment. After you get passed the problems with the basics, go on to increment...