Abstract: Transformers have achieved remarkable success in diverse fields such as Natural Language Processing (NLP) and computer vision (CV). For pre-trained Transformer models involving text ...