Open in app

Sign up

Sign in

Write

Sign up

Sign in

|LLM|ATTENTION|CONTEXT LENGTH|

Infini-attention: Can we Really have an Infinite Context Length?

Google believes that can we have an LLM with an infinite context length

Salvatore Raieli
Level Up Coding
Published in
8 min readApr 28, 2024

--

Infini-attention
Photo by Angely Acevedo on Unsplash

Our knowledge can only be finite, while our ignorance must necessarily be infinite. — Karl Popper

--

--

Salvatore Raieli
Level Up Coding
Follow

Written by Salvatore Raieli

11.1K Followers
Writer for

Senior data scientist | about science, machine learning, and AI. Top writer in Artificial Intelligence

Follow

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams