Maryam Parsa, Catherine Schuman, Nitin Rathi, Amir Ziabari, Derek Rose, J. Parker Mitchell, J. Travis Johnston, Bill Kay, Steven Young and Kaushik Roy

July, 2021

ICONS: International Conference on Neuromorphic Systems

https://doi.org/10.1145/3477145.3477160

PDF not available yet, or is only available from the conference/journal publisher.

Abstract

Neuromorphic systems allow for extremely efficient hardware implementations for neural networks (NNs). In recent years, several algorithms have been presented to train spiking NNs (SNNs) for neuromorphic hardware. However, SNNs often provide lower accuracy than their artificial NNs (ANNs) counterparts or require computationally expensive and slow training/inference methods. To close this gap, designers typically rely on reconfiguring SNNs through adjustments in the neuron/synapse model or training algorithm itself. Nevertheless, these steps incur significant design time, while still lacking the desired improvement in terms of training/inference times (latency). Designing SNNs that can mimic the accuracy of ANNs with reasonable training times is an exigent challenge in neuromorphic computing. In this work, we present an alternative approach that looks at such designs as an optimization problem rather than algorithm or architecture redesign. We develop a versatile multiobjective hyperparameter optimization (HPO) for automatically tuning HPs of two state-of-the-art SNN training algorithms, SLAYER and HYBRID. We emphasize that, to the best of our knowledge, this is the first work trying to improve SNNs’ computational efficiency, accuracy, and training time using an efficient HPO. We demonstrate significant performance improvements for SNNs on several datasets without the need to redesign or invent new training algorithms/architectures. Our approach results in more accurate networks with lower latency and, in turn, higher energy efficiency than previous implementations. In particular, we demonstrate improvement in accuracy and more than 5 × reduction in the training/inference time for the SLAYER algorithm on the DVS Gesture dataset. In the case of HYBRID, we demonstrate 30% reduction in timesteps while surpassing the accuracy of the state-of-the-art networks on CIFAR10. Further, our analysis suggests that even a seemingly minor change in HPs could change the accuracy by 5 − 6 ×.

Citation Information

Text

author       M. Parsa and C. D. Schuman and N. Rathi and A. Ziabari and D. Rose and
             J. P. Mitchell and J. Travis Johnston and B. Kay and S. R. Young and K. Roy
title        Accurate and Accelerated Neuromorphic Network Design Leveraging A 
             Bayesian Hyperparameter Pareto Optimization Approach
booktitle    International Conference on Neuromorphic Computing Systems (ICONS)
publisher    ACM
pages        1-8
year         2021
url          https://doi.org/10.1145/3477145.3477160
doi          10.1145/3477145.3477160

Bibtex

@INPROCEEDINGS{dks:21:ccn,
    author = "M. Parsa and C. D. Schuman and N. Rathi and A. Ziabari and D. Rose and
                 J. P. Mitchell and J. Travis Johnston and B. Kay and S. R. Young and K. Roy",
    title = "Accurate and Accelerated Neuromorphic Network Design Leveraging A 
                 Bayesian Hyperparameter Pareto Optimization Approach",
    booktitle = "International Conference on Neuromorphic Computing Systems (ICONS)",
    publisher = "ACM",
    pages = "1-8",
    year = "2021",
    url = "https://doi.org/10.1145/3477145.3477160",
    doi = "10.1145/3477145.3477160"
}