A Configurable Activation Function for Variable Bit-Precision DNN Hardware Accelerators

Abstract

This paper introduces a configurable Activation Function (AF) that utilizes ROM/ Cordic architecture to generate sigmoid and tanh with varying bit precision. Two design strategies are explored: a ROM-based approach for low-bit precision and a Cordic-based approach for high-bit precision. The accuracy of the configurable AF is assessed on LeNet and VGG-16 DNN models, revealing minimal accuracy loss (less than 1.5%) compared to the tensorflow-based model. Experimental results on the Zybo Evaluation kit-Xilinx, using a ‘fixed&lt

9, 6&gt

’ arithmetic representation, demonstrate the ROM-based approach’s memory efficiency, achieving 86.66% LUT savings for 4-bit precision and 80.95% LUT savings for 8-bit precision compared to the Cordic-based approach. The Cordic-based approach, on the other hand, shows ≈ 93% LUT savings for 16-bit precision, compared to the ROM-based approach. The proposed AF utilizes the robustness of ROM and Cordic architectures for appropriate bit precision to enhance the overall performance of Deep Neural Networks (DNNs). © 2024, IFIP International Federation for Information Processing.