Deep octonion networks

J. Wu (1, 2, 3) , L. Xu (4, 2, 3) , F. Wu (4, 2, 3) , Y. Kong (4, 2, 3) , L. Senhadji (1, 2) , H. Shu (4, 2, 3)

1 LTSI - Laboratoire Traitement du Signal et de l'Image
2 CRIBS - Centre de Recherche en Information Biomédicale sino-français
3 LIST - Laboratory of Image Science and Technology [Nanjing]
4 SEU - Nanjing Southeast University

Deep learning is a hot research topic in the field of machine learning methods and applications. Real-value neural networks (Real NNs), especially deep real networks (DRNs), have been widely used in many research fields. In recent years, the deep complex networks (DCNs) and the deep quaternion networks (DQNs) have attracted more and more attentions. The octonion algebra, which is an extension of complex algebra and quaternion algebra, can provide more efficient and compact expressions. This paper constructs a general framework of deep octonion networks (DONs) and provides the main building blocks of DONs such as octonion convolution, octonion batch normalization and octonion weight initialization; DONs are then used in image classification tasks for CIFAR-10 and CIFAR-100 data sets. Compared with the DRNs, the DCNs, and the DQNs, the proposed DONs have better convergence and higher classification accuracy. The success of DONs is also explained by multi-task learning.

Keywords: Complex; Convolutional neural network; Image classification; Octonion; Quaternion

Domains: Engineering Sciences [physics]; Signal and Image processing; Computer Science [cs]; Neural and Evolutionary Computing [cs.NE]; Computer Science [cs]; Artificial Intelligence [cs.AI]; Life Sciences [q-bio]; Bioengineering

J. Wu, L. Xu, F. Wu, Y. Kong, L. Senhadji, et al.. Deep octonion networks. Neurocomputing, 2020, 397, pp.179-191. ⟨10.1016/j.neucom.2020.02.053⟩. ⟨hal-02865295⟩