[CVPR 2023] IMP: iterative matching and pose estimation with transformer-based recurrent module

[MICCAI 2023] DAE-Former: Dual Attention-guided Efficient Transformer for Medical Image Segmentation

[NeurIPS'21] "Chasing Sparsity in Vision Transformers: An End-to-End Exploration" by Tianlong Chen, Yu Cheng, Zhe Gan, Lu Yuan, Lei Zhang, Zhangyang Wang

[NeurIPS 2022 Spotlight] This is the official PyTorch implementation of "EcoFormer: Energy-Saving Attention with Linear Complexity"

[ICCV 2023] Efficient Video Action Detection with Token Dropout and Context Refinement

Master thesis with code investigating methods for incorporating long-context reasoning in low-resource languages, without the need to pre-train from scratch. We investigated if multilingual models could inherit these properties by making it an Efficient Transformer (s.a. the Longformer architecture).

Official Implementation of Energy Transformer in PyTorch for Mask Image Reconstruction

This repository contains the official code for Energy Transformer---an efficient Energy-based Transformer variant for graph classification

Demo code for CVPR2023 paper "Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers"

A custom Tensorflow implementation of Google's Electra NLP model with compositional embeddings using complementary partitions

This is the source code of article how to create a chatbot in python . i.e A chatbot using the Reformer, also known as the efficient Transformer, to generate dialogues between two bots.

Nonparametric Modern Hopfield Models

MetaFormer-Based Global Contexts-Aware Network for Efficient Semantic Segmentation (Accepted by WACV 2024)

Gated Attention Unit (TensorFlow implementation)