On Dimension-Free Transformer: An Application of STP to AI
By: Daizhan Cheng
Potential Business Impact:
Makes computer learning work with any size information.
The matrix expressions for every parts of a transformer are firstly described. Based on semi-tensor product (STP) of matrices the hypervectors are reconsidered and the linear transformation over hypervectors is constructed by using projection. Its properties and calculating formulas are obtained. Using projection-based transformation of hypervector (PBTH), the framework of dimension-free transformer (DFT) is proposed by verifying each linear transformation in a transformer and replacing it by a proper PBTH, which allows the inputs and outputs being of arbitrary dimensions. Using balanced information about all entries, DFT must be more efficient in dealing with signals.
Similar Papers
Transformers Can Overcome the Curse of Dimensionality: A Theoretical Study from an Approximation Perspective
Machine Learning (CS)
Makes AI understand complex patterns better and faster.
Allocation of Parameters in Transformers
Machine Learning (CS)
Makes smart computer programs work faster and better.
Parameter-Efficient Transformer Embeddings
Computation and Language
Makes AI understand words using less computer memory.