Rule distance based rule-base reduction in the expert knowledge-included FRIQ-learning

Abstract

In the expert knowledge-included Fuzzy Rule Interpolation-based Q-learning (expert knowledge-included FRIQ-learning) reinforcement learning system by the expert defined knowledgebase will be extended by the system created fuzzy rules during the learning phase. The system tune (and optimize) the antecedent and consequent parts of fuzzy rules during the learning iterations, due to this method can be the case when any rules will be close to each other. The size of the knowledgebase of the system can be reduced by merging the closing rules. Based on the knowledgebase reduction methodology, the distance between the rules (therefore the measure of rule closure) is determined only in antecedent (state-action) universes. However, it can be problematic in the cases when the rules can be determined as closing rules in the antecedent universes but there is a large difference in the consequent dimension, thus there is a steep slope of the Q-function.  The main contribution of the paper is to introduce a modification of the knowledge reduction methodology that will extend the distance determination to the consequent (Q-value) universe as well.