Reinforcement Learning with Weightless Neural Networks (Master’s Dissertation)
Published:
Abstract: Driving vehicles, allocating resources, controlling industrial plants: these are just a few examples of the many interesting problems that require making decisions over time. Given the real-world impacts and costs of such tasks, the study of methods to automate these is of great importance. Reinforcement learning (RL) is the branch of machine learning that deals with sequential decision-making. Impressive results have been seen in recent years due to RL, especially when making use of deep learning models. Compared to these neural networks, the use of alternative learning models has not been as much of a focus in recent research. The adoption of weightless neural networks, in particular, is underexplored. Nevertheless, the study of their use in this context is worthwhile, as it can provide one more tool in tackling decision-making problems and possibly allows more favorable trade-offs to be made for specific instances. Making use of n-tuple nets in RL, however, is not a matter of simply plugging existing architectures in RL methods, as these are based on assumptions incompatible with the issues imposed by sequential decision-making. To reconcile this class of networks with RL, two novel architectures are proposed, one leveraging a simple mechanism akin to forgetting, and the other drawing inspiration from online learning with kernels. With these two variants, weightless nets can be employed both in action-value and policy search methods. The proposed models are evaluated in four benchmarks tasks, thoroughly assessing the effects of their hyperparameters and comparing them to feedforward networks, with results showing that they can attain competitive performance.