Tested a 5080 and a 5070 Ti against my 40xx cards. Results:

If we compare RTX 4080 and 5080:

I generated Nvidia assembly code, trying to understand the new Blackwell architecture.

The code is a Radix-4 NTT butterfly using three prime moduli, the most intensively used function of genefer.

GeForce 10 series (Pascal): the multiplier is a 16-bit unit then four 16-bit Multiply and Add are needed to emulate a 32-bit MAD. 394 instructions are generated and the number of cycles is 394.

GeForce 20 series (Turing): each core contains a 32-bit MAD unit. The number of instructions and cycles is 270.

GeForce 30 and 40 series (Ampere & Ada Lovelace): double FP32 performance per SM. Half of FP32 units are paired with integer units (without the MAD instruction). The other half of FP32 units are paired with MAD units. If the number of MAD is equal to the number of other integer instructions, UINT32 performance is doubled (the compiler transforms some add, left shift and move instructions into MAD for load balancing). The number of instructions is 273 (including 131 MAD): counting pairs of cores, the number of cycles is 142.
Ada Lovelace is faster, not because of the core architecture but because of fabrication process (TSMC 4N vs Samsung 8N) and L2 cache size (16x).

GeForce 50 series (Blackwell): cores are all identical with one FP32 unit and one UINT32 unit. It is not faster but load balancing of MAD/others is not needed. 277 instructions are generated (the number of MAD is now 78). It increased a bit because the addressing mode was narrowed. With Ada Lovelace, you can write IADD3 R1, R2, c[0x0][0x180], RZ; where c[0x0][0x180] is a parameter on the stack. Now, two instructions are needed: LDC.64 R3, c[0x0][0x180] and IADD3 R1, PT, PT, R2, R3, RZ. This difference is not sufficient to explain why 5080 is not 15% faster then 4080.
32-bit addressing mode was removed, but genefer is a 64-bit application. Maybe 32-bit memory accesses are now slower because of a different cache organization...?

Thanks for explanation.

GPU-Z shows extrem high Bus Interface Load with RTX5080, did anyone else has the same?

You all know way more than I do but did you download Aida64 Extreme for the free trial to test on the 50xx graphics cards? What does it show for 24-bit/32-bit IOPS for 50xx series graphics cards? I know the software could possibly be inaccurate.

https://www.aida64.com

I ran the trial AIDA64 Extreme GPGPU benchmark on my GeForce RTX 4060 (that runs it alongside my i5-13400 for comparison and hides some values in the trial run).
Here is my screenshot - curious what 50xx series show.

Boost is rated 2460 Mhz for GPU but I have seen 2715/2730/2760 Mhz in 3DMark benchmarks.
(It's a HP desktop with its own HP version GeForce RTX 4060 I presume).

I am totally guessing here based on things I have read but if I understand:

a) 15264 GFLOPS for Single-Precision FLOPS is because 2460 * 1000 * 1000 cycles per second * 3072 shaders * 2 (because two instructions in FMA - fused multiply add) is approximately 15114 GFLOPS that matches techpowerup.com

b) Double-Precision FLOPS is 261.5 GFLOPS because instead of FP64:FP32 ratio of 1:2 consumer GeForce 4060 is 1:64 (since only 2 FP64 for every 1 streaming multiprocessor that has 128 shaders (FP32), unlike datacenter GPUs that would have 64 FP64 for every 1 streaming multiprocessor) hence why Double-Precision FLOPS is 261.5 since 15114/64 is approximately 236.16 GFLOPS that matches techpowerup.com

c) Trial version AIDA64 has 32-bit integer IOPS hidden but looking at images on internet others are same as 24 bit. My 24 bit shows 8414 GIOPS since 2460 x 1000 * 1000 cycles per second * (3072 shaders / 2 --> since Ada Lovelace architecture is half shaders are INT32 or FP32 while other half are only FP32) * 2 (integer multiply add) = is approximately 7557 GIOPS which is half single precision FLOPS and seems to be correct since for RTX 4090 graphics card Nvidia Ada Lovelace document here shows "Peak INT32 TOPS (nonTensor)" as half "Peak FP32 TFLOPS (nonTensor)" value.

https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvidia-ada-gpu-architecture.pdf

Note Yves important comments regarding this Compute Capability 8.9 graphics card:
Compute Capability 8.6 (GeForce 30): SM = 64 MAD32_64/FP32 + 64 INT32/FP32 + 2 FP64. Half of the cores are able to execute a MAD instruction z += x * y, where x, y are 32-bit integers, the result of the multiplication and z are 64-bit integers. The other half of the cores execute other instructions (add, shift, logical operations, etc).
Compute Capability 8.9 (GeForce 40): SM are identical to 8.6 but process size is 5 nm (Ampere was 8 nm) then GPU is operating at higher frequency. More importantly, L2 cache size is 10x: 40x0 are at least 50% faster than 30x0.

d) 64-bit integer IOPS of 2069 GIOPS - I can guess how that value is determined but I leave to you Yves.

Techpowerup page on 4060:
https://www.techpowerup.com/gpu-specs/geforce-rtx-4060.c4107

My other post where I say Nvidia fixed error on website to show 5080, 5090 is Compute Capability 12.0 not 10.0

https://www.primegrid.com/forum_thread.php?id=10836

For Nvidia GPU, there is no 24-bit integer then 24-bit IOPS = 32-bit IOPS. For AMD GPU, a 24-bit multiplication is three times as fast as a 32-bit multiplication and there is no MAD instruction, but a MUL operation.

FLOPS is twice the number of FP operations per second because of FMA.
How IOPS is defined? A priori x2 for IMAD, but there is also a 3-input integer addition (IADD3) which is two additions (genefer code is a list of IMAD and IADD3 and some tests and conditional move for modular addition/substraction).
We should have for
- 40 series: 2 cores can execute 2 FMA or 1 IADD3 and 1 IMAD => 4 FLOP or 4 IOP. Then FLOPS = IOPS.
- 50 series: each core can execute 1 FMA or 1 IADD3 or 1 IMAD => 2 FLOP or 2 IOP. Then FLOPS = IOPS.

There is no 64-bit integer. Then a 64-bit addition is two 32-bit additions and a 64-bit MAD is four 32-bit MAD. Note that every instruction supports carry-in and carry-out.

Here is my Asus PRIME GeForce RTX™ 5070 12GB GDDR7 OC Edition (PRIME-RTX5070-O12G)

Tested ZOTAC GAMING GeForce RTX 5090 AMP Extreme INFINITY 32GB GDDR7 (ZT-B50900B-10P)

Tested ZOTAC GAMING GeForce RTX 5090 AMP Extreme INFINITY 32GB GDDR7 (ZT-B50900B-10P)

+---------------+---------------+---------------+ |Subproject |5090 elapsed |5090 power | +---------------+---------------+---------------+ |GFN-16 |39 |210w | +---------------+---------------+---------------+ |GFN-16 (x2) |60 |207w | +---------------+---------------+---------------+ |GFN-16 (x3) |124 |204w | +---------------+---------------+---------------+ |GFN-17 |98 |279w | +---------------+---------------+---------------+ |GFN-17 (2x) |128 |263w | +---------------+---------------+---------------+ |GFN-17 (3x) |256 |278w | +---------------+---------------+---------------+ |GFN-18 |218 |395w | +---------------+---------------+---------------+ |GFN-18 (2x) |426 |380w | +---------------+---------------+---------------+ |GFN-19 |653 |590w | +---------------+---------------+---------------+ |GFN-19 (2x) |1323 |540w | +---------------+---------------+---------------+ |GFN-20 |2005 |600w | +---------------+---------------+---------------+ |GFN-21 |6969 |600w | +---------------+---------------+---------------+ |DYFL |60592 |600w | +---------------+---------------+---------------+ |AP27 |64 |600w | +---------------+---------------+---------------+ |AP27 (2x) |122 |600w | +---------------+---------------+---------------+

Looks like the Aida64 GPGPU benchmark is currently not supported since Nvidia removed 32-bit OpenCL support for the 5000 series.

https://forums.aida64.com/topic/16903-nvidia-rtx5090d-unsupported-gpgpu-benchmark/

Here is my Asus PRIME GeForce RTX™ 5070 12GB GDDR7 OC Edition (PRIME-RTX5070-O12G)

+---------------+---------------+---------------+ |Subproject |5070 elapsed |5070 power | +---------------+---------------+---------------+ |GFN-16 |70 |90w | +---------------+---------------+---------------+ |GFN-16 (x2) |121 |105w | +---------------+---------------+---------------+ |GFN-16 (x3) |192 |111w | +---------------+---------------+---------------+ |GFN-17 |163 |124w | +---------------+---------------+---------------+ |GFN-17 (2x) |347 |151w | +---------------+---------------+---------------+ |GFN-17 (3x) |472 |158w | +---------------+---------------+---------------+ |GFN-18 |459 |202w | +---------------+---------------+---------------+ |GFN-18 (2x) |896 |196w | +---------------+---------------+---------------+ |GFN-19 |1364 |238w | +---------------+---------------+---------------+ |GFN-19 (2x) |2743 |239w | +---------------+---------------+---------------+ |GFN-20 |4463 |246w | +---------------+---------------+---------------+ |GFN-21 |18131 |247w | +---------------+---------------+---------------+ |DYFL |205857 |246w | +---------------+---------------+---------------+ |AP27 |219 |201w | +---------------+---------------+---------------+ |AP27 (2x) |421 |205w | +---------------+---------------+---------------+

Yves already above described about the architecture.

Just for awareness, on March 20, I put a Nvidia support ticket in regarding page 11-12 of the Nvidia RTX Blackwell GPU Architecture Document.

I asked “Please advise for consumer RTX 5000 series graphics cards if they do or do not have double INT32 performance (i.e. can use all shaders for INT32 rather than half) for Blackwell consumer Geforce 5000 series graphics cards compared to previous Ada Lovelace consumer Geforce 4000 series graphics (which can use half the shaders for INT32)”.

On April 9 Nvidia replied “There is no error in the Blackwell architecture document. Yes, the consumer GeForce RTX 5000 series graphics cards do have double INT32 performance compared to the Ada Lovelace-based RTX 4000 series”

On April 9 Nvidia replied “There is no error in the Blackwell architecture document. Yes, the consumer GeForce RTX 5000 series graphics cards do have double INT32 performance compared to the Ada Lovelace-based RTX 4000 series”

Thanks Yves.

FYI - I have been quoting you (giving you credit) in the Nvidia Blackwell integer thread here:

https://forums.developer.nvidia.com/t/blackwell-integer/320578

Also Yves, in that thread it sounds like for cases where the instructions are not mixed they are not seeing the 2x INT32 increase for Blackwell versus Ada Lovelace. (genefer does a IADD3/IMAD mix for Ada Lovelace so can't be used as a comparison since it negates any "theoretical 2x" increase).

Also Yves, in that thread it sounds like for cases where the instructions are not mixed they are not seeing the 2x INT32 increase for Blackwell versus Ada Lovelace. (genefer does a IADD3/IMAD mix for Ada Lovelace so can't be used as a comparison since it negates any "theoretical 2x" increase).

Latest driver 576.02 has been reported to give performance increases in (some?) gaming related uses. On my own system, relative to early drivers, I saw between 4 to 11% increases in Steel Nomad, Monster Hunter Wilds, Black Myth Wukong, and FFXIV: Dawntrail benchmarks. For Wukong specifically, I noticed the boost clocks were going higher but don't know if that is the only thing going on. Power may be a little higher, but that fluctuated a lot so I'm less certain.

Only compute load I tried is Blender 4.3.0. 7339.73 with latest driver, 7121.3 with older, or just over 3% increase.

I have not done any BOINC type testing on it so far.

CUDA 12.9 provides new insights into Blackwell architecture.

https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#compute-capability-12-x
Compute Capability 12.x (Blackwell 2.0). A Streaming Multiprocessor (SM) consists of:
- 128 FP32 cores for single-precision arithmetic operations,
- 2 FP64 cores for double-precision arithmetic operations,
- 64 INT32 cores for integer math.

More details are available at https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#arithmetic-instructions
Compute Capability of Ada Lovelace is 8.9. Then what's new (or not) with Blackwell architecture?
The throughput of 32-bit integer add, sub, logic is still 64 (and not 128 as indicated in Blackwell GPU Architecture Whitepaper).
But now the throughput of 32-bit integer multiply-add is 128. A surprising architecture: 128 additions can be computed using the MAD instruction, but 64 only using the ADD instruction. It is not a real improvement compared to Ada Lovelace because the throughput was already 64 ADD/SUB/AND/... + 64 MAD = 128 instructions.
It is interesting to note that there are now 128 64-bit integer add, multiply units. Because the 32-bit support was removed and addressing modes were reduced, 64-bit operations are needed for address calculation. These instructions are not as powerful as 32-bit variants (carry-in, carry-out and MAD are not available) but it is sufficient for evaluating a 64-bit address. This may indicate that the next Nvidia architectures will be based on 64-bit cores. It would be a major improvement for primality tests.

64-bit operations are not sufficient to counteract the narrowed addressing mode. The size of the code increased by 5-10%.
RTX 5080 (10752 cores, 360 W) is about as fast as RTX 4080 (9728 cores, 320 W) for GFN-21. The L2 cache size is 64MB and data size is 36MB. GFN-23 is faster on RTX 5080 because the data size is 96MB and memory bandwidth is 960.0GB/s (vs 716.8GB/s).

CUDA Toolkit 12.9 (NVIDIA Driver 575.51.03)
Genefer v4.10
AP27 Search v2.11

Tested ZOTAC GAMING GeForce RTX 5090 AMP Extreme INFINITY 32GB GDDR7 (ZT-B50900B-10P)
Fedora 41 Workstation

Genefer v4.10 AP27 Search v2.11 Tested ZOTAC GAMING GeForce RTX 5090 AMP Extreme INFINITY 32GB GDDR7 (ZT-B50900B-10P)

Memory bus width 256 vs 512?
L1 cache 10 vs 21.25MB?

5090 is about 2x 5080 (number of cores and memory bandwidth).
5090 is twice as fast as 5080 (44806/89351 seconds) but the power consumption is 600/370 W (x1.62).

5080 uses 30Gbps VRAM vs 28Gbps on 5090. I don't know if they use a higher voltage to get the higher speed. If so, that will affect power efficiency beyond just clock scaling.

5080 has higher default clocks, roughly 2.3 GHz base, 2.6 boost, compared to 5090 2.0 base, 2.4 boost. Power limit of 5080 is 360W vs 600W of 5090.

Basically 5080 is pushed harder and works in a less efficient area. If the 5090 didn't have that power limit it likely could go faster still.

I am computing with this kind of setup.

Just 91% utilization of the GPU?
What about the values running two tasks in parallel?
____________
Greetings, Jens

147433824^131072+1

Just 91% utilization of the GPU?
What about the values running two tasks in parallel?

This is what the data I have on hand looks like.

RTX5090
GPU utilization

GFN16
x1 73%
x2 74%
x3 75%

GFN17
x1 81%
x2 81%
x3 82%

GFN18
x1 83%
x2 85%

GFN19
x1 83%
x2 84%

GFN20
x1 90-91%

GFN21
x1 75-96%

DYFL
x1 98%

CUDA 12.9 provides new insights into Blackwell architecture.

But now the throughput of 32-bit integer multiply-add is 128.

It is interesting to note that there are now 128 64-bit integer add, multiply units.

Nvidia documentation last Monday (see https://web.archive.org/web/20250505133403/https://docs.nvidia.com/cuda/cuda-c-programming-guide/#maximize-instruction-throughput

32-bit integer add, extended-precision add, subtract, extended-precision subtract: 64
32-bit integer multiply, multiply-add, extended-precision multiply-add: 128
64-bit integer add, multiply: 128

And Nvidia documentation on Friday (https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#maximize-instruction-throughput)

32-bit integer add, extended-precision add, subtract, extended-precision subtract: 64
32-bit integer multiply, multiply-add, extended-precision multiply-add: 64
64-bit integer add: 64

On Ada Lovelace, the compiler generates about 50% of integer multiply-add and 50% of integer add, shift, compare such that the throughput is 64 + 64 = 128.
On Blackwell, multiply and others are not balanced. But the throughput of integer operations is similar to Ada Lovelace's throughput.
It is inconsistent with the table... the real architecture remains unknown.
Are integer add unit, madd unit, shift unit and compare unit independent such that the throughputs can be added?

This is what the data I have on hand looks like.
RTX5090 GPU utilization [...]

Nvidia has finally responded in the Blackwell Integer thread.

From employee mjoux

​

Hi, I’m part of the team responsible for the instruction throughput table in the CUDA programming guide, and I wanted to give you an update on this thread from my point of view:

First, thank you all for raising these issues!
With CUDA 13.0, we will make significant changes to the table: (1) it will be moved to the CUDA best practices guide as it is less relevant to the programming model and (2) it will be re-structured, and will have example PTX instructions which will hopefully provide a little more clarity, although it is obviously far from perfect.

Concerning Blackwell integer instruction throughput specifically:
As pointed out in the thread already, some instructions have not been improved/changed: this applies to IMAD, LOP3, PRMT for example, as well as IADD3.
The main improved instructions relevant to this thread are IADD, IMNMX/VIMNMX, FSETP/ISETP: addition of 2 operands and min/max/compare.
Concerning integer addition specifically, it is even more complicated: previous architectures already had the possibility to achieve 2x throughput by combining e.g. IADD3 with IMAD.IADD or VIADD (for 9.0/10.0). Blackwell 12.0 now allows achieving this 2x throughput with a single instruction: IADD. But note that it can be difficult to get a sequence of instructions achieving higher throughput: for previous architectures, this is because of constraints in the instructions as well as compiler, which we cannot disclose publicly. For Blackwell 12.0, the compiler often outputs IADD3 instead of IADD: this should be improved soon.

Because of the above, it is unlikely that current benchmarks are able to achieve this 2x throughput, unless they are hand-crafted without relying on the compiler.
Note that this is the case for some other entries in the instruction throughput table: it only lists the theoretical maximum throughput, but we cannot always disclose how to precisely achieve this in practice.

If you have further questions, I can try answering them.