GeForce Unlocked: modders double the computing performance of the RTX 20, RTX 30 and RTX 40

NVIDIA’s GeForce RTX 30 and RTX 40 graphics cards are based on the same chips as their professional-grade GPUs. This means that the computing potential of consumer RTX GPUs is limited by NVIDIA’s firmware stack, but modders have managed to unlock an instruction issuance rate equivalent to that of their professional GPU counterparts. Starting with the GeForce RTX 30 series “Ampere,” a researcher developed an unlock that effectively doubles the instruction issuing speed of the FP/BF16 FMA in FP32 operations on GA102, GA103, GA104, and GA106 cards. As a result, cards like the GeForce RTX 3090 Ti benefit from up to 80% faster matrix multiplication in BF16, which is crucial for running almost any neural network. The unlock supports GeForce RTX graphics cards based on GA102, GA103, GA104 and GA106.

Next we have the GeForce RTX 40 series “Ada Lovelace” with another project that demonstrated a massive 2x speedup in FP16 and BF16 tensor throughput with FP32 accumulation. When the GeForce RTX 4090 was tested with FP16 tensor throughput and matrix accumulation, the GPU achieved 165.6 TeraFLOPS, while unlocking pushed that number to 331.6 TeraFLOPS. This is intriguing because these computing performance numbers are usually reserved for the high-end professional segment like the NVIDIA RTX 6000 “Ada Lovelace”. It appears that NVIDIA has artificially limited the AD102 silicon’s performance in compute-aware matrix multiplication and accumulation, thereby affecting the performance of running large language models. For this unlock, AD102, AD103 and AD106 are officially unlocked.

These two separate GPU unlocks appeared almost simultaneously, with one demonstrated for the GeForce RTX 30 “Ampere” series and the other for the GeForce RTX 40 “Ada Lovelace” series. Both unlocks pass NVIDIA driver checks and override PCIe Function Level Reset (FLR), NVIDIA drivers running and CUDA tests are performed normally. For example, the same 450W TGP card power is used before and after unlocking on the GA102 and AD102. For those running compute-intensive tasks such as local LLMs, the additional computing power translates into significantly higher token throughput during pre-fetching and decoding, allowing consumers to experience higher performance. However, it is important to note that any modifications to the GPU will void the warranty and must be enabled at startup so that these mods bind before the NVIDIA drivers bind the card to the kernel.

Updated at 09:20 UTC: NVIDIA’s GeForce RTX 20 series “Turing” are also unlocked by the same technique, increasing performance by 2x.