Abstraction in Reinforcement Learning

Reinforcement learning is the problem faced by anagent that must learn behavior throughtrial-and-error interactions with a dynamicenvironment. Usually the problem to be solvedcontains subtasks that repeat at different regions ofthe state space. Without any guidancean agent has to learn the solutions of all subtaskinstances independently which in turn degrades theperformance of the learning process. In this work wepropose two novel approaches for building theconnections between different regions of the searchspace. The first approach efficiently discoversabstractions in the form of conditionally terminatingsequences and represents these abstractions compactlyas a single tree structure; this structure is thenused to determine the actions to be executed by theagent. In the second approach a similarity functionbetween states is defined based on the number ofcommon action sequences; by using this similarityfunction updates on the action-value function of astate are re?ected to all similar states that allowsexperience acquired during learning be applied to abroader context. The effectiveness of both approachesis demonstrated empirically over various domains.