An LLM is a lossy encyclopedia