James S. Plank, Charles P. Rizzo, Chris A. White and Catherine D. Schuman

January, 2025

Journal of Low Power Electronics and Applications (Open access)

https://www.mdpi.com/2079-9268/15/1/5

PDF not available yet, or is only available from the conference/journal publisher.

Abstract

The cart-pole application is a well-known control application that is often used to illustrate reinforcement learning algorithms with conventional neural networks. An implementation of the application from OpenAI Gym is ubiquitous and popular. In this paper, we explore using this application as a benchmark for spiking neural networks. We propose four parameter settings that scale the application in difficulty, in particular beyond the default parameter settings which do not pose a difficult test for AI agents. We propose achievement levels for AI agents that are trained on these settings. Next, we perform an experiment that employs the benchmark and its difficulty levels to evaluate the effectiveness of eight neuroprocessor settings on success with the application. Finally, we perform a detailed examination of eight example networks from this experiment, that achieve our goals on the difficulty levels, and comment on features that enable them to be successful. Our goal is to help researchers in neuromorphic computing to utilize the cart-pole application as an effective benchmark.

Citation Information

Text

author     J. S. Plank and C. P. Rizzo and C. A. White and C. D. Schuman
title      The Cart-Pole Application as a Benchmark for Neuromorphic Computing
doi        10.3390/jlpea15010005
url        https://www.mdpi.com/2079-9268/15/1/5
year       2025
publisher  MDPI
article-number 5
issn       2079-9268
journal    Journal of Low Power Electronics and Applications
volume     15
number     1
pages      1-27

Bibtex

@ARTICLE{prw:25:cpa,
    author = "J. S. Plank and C. P. Rizzo and C. A. White and C. D. Schuman",
    title = "The Cart-Pole Application as a Benchmark for Neuromorphic Computing",
    doi = "10.3390/jlpea15010005",
    url = "https://www.mdpi.com/2079-9268/15/1/5",
    year = "2025",
    publisher = "MDPI",
    article-number = "5",
    issn = "2079-9268",
    journal = "Journal of Low Power Electronics and Applications",
    volume = "15",
    number = "1",
    pages = "1-27"
}