Facebook’s latest giant language AI hits computing wall at 500 Nvidia GPUs

Facebook AI research's latest breakthrough in natural language understanding, called XLM-R, engineers tasks such as question-answering across 100 different languages including Swahili and Urdu. It shows both that deep learning models keep getting bigger, and that they're running up against serious resource constraints in existing computing systems.