Adi Makmal, Alexey A. Melnikov, Vedran Dunjko, Hans J. Briegel
IEEE Access 4, 2110–2122
Publication year: 2016
  

Learning models of artificial intelligence can nowadays perform very well on a large variety of tasks. However, in practice, different task environments are best handled by different learning models, rather than a single universal approach. Most non-trivial models thus require the adjustment of several to many learning parameters, which is often done on a case-by-case basis by an external party. Meta-learning refers to the ability of an agent to autonomously and dynamically adjust its own learning parameters or meta-parameters. In this paper, we show how projective simulation, a recently developed model of artificial intelligence, can naturally be extended to account for meta-learning in reinforcement learning settings. The projective simulation approach is based on a random walk process over a network of clips. The suggested meta-learning scheme builds upon the same design and employs clip networks to monitor the agent’s performance and to adjust its meta-parameters on the fly. We distinguish between reflex-type adaptation and adaptation through learning, and show the utility of both approaches. In addition, a trade-off between flexibility and learning-time is addressed. The extended model is examined on three different kinds of reinforcement learning tasks, in which the agent has different optimal values of the meta-parameters, and is shown to perform well, reaching near-optimal to optimal success rates in all of them, without ever needing to manually adjust any meta-parameter.

Leave a Reply

Your email address will not be published. Required fields are marked *

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. Cookies are used to retrieve and process the following user data using Google Analytics: a location of user’s device, type and version of operating system, type and version of a browser, type of user’s device and screen resolution, the source from which user came from, a language of operating system and browser. The following operations will be performed with cookies and user data: collection, recording, arrangement, accumulation, storage, specification (updating, alteration), retrieval, use, transmission (providing access), blocking, erasure or destruction of data. If you continue to use this website or you click "Accept" then you are consenting to this.

Close