Reward-Dependent Sensory Coding in Free-Energy-Based Reinforcement Learning

Summary

---

The free-energy-based reinforcement learning is a new approach to handling high-dimensional states and actions. We investigate its properties using a new experimental platform called the digit floor task. In this task, the high-dimensional pixel data of hand-written digits were directly used as sensory inputs to the reinforcement learning agent. The simulation results showed the robustness of the free-energy-based reinforcement learning method against noise applied in both the training and testing phases. In addition, reward-dependent sensory representations were found in the distributed activation patterns of hidden units. The representations coded in a distributed fashion persisted even when the number of hidden nodes were varied.

See the full content of this document

Extract

---

Reward-Dependent Sensory Coding in Free-Energy-Based Reinforcement Learning

(ProQuest: ... denotes formulae omitted.)

1. Introduction

Sensory systems of a biological agent are ceaselessly bombarded with rich sensory inputs from the environment. A living organism has to deal with extremely highdimensional sensory data in order to make appropriate decisions swiftly and to protect its niche in the game of survival. The same analogy holds for an artificial agent. For example, a robot with a camera vision aiming at capturing battery packs has to handle a continuous flow of high-dimensional pixel images to keep navigating the environment without running out of battery. It is not hard to imagine that the hardware resource itself cannot provide the robot with a skill for successful foraging. As long as an agent - whether it is biological or artificial - lives in a real environment, there always exists the huge gap between the dimensionality of raw sensory data and the dimensionality of features required to solve the given task.

The main difference between biological agents and today's artificial agents is that biological agents somehow extract features involved in a given task and select appropriate actions in the flood of raw sensory data. For artificial agents, important features for a task are often selected by human engineers based on their prior knowledge. In order to create a t...

See the full content of this document

---