Problem with multi-threading and app

Join PrimeGrid

Read our rules and policies.
Download, install and run the BOINC software used by PrimeGrid.
When prompted, enter the URL:
http://www.primegrid.com/
Select which subprojects to run via the PrimeGrid Preferences.

If you are not using graphics enabled BOINC client, you may use manual registration form.

Author

Message

Corla99 [Lombardia]
Send message
Joined: 26 Jun 16
Posts: 2
ID: 450493
Credit: 101,573,914
RAC: 97

Hi, today I moved my pc with 2xE5-2699 on this project. Before doing this, I've disabled the hyper-threading and created an app_config:

<app_config> <app> <name>llr321</name> <fraction_done_exact/> </app> <app_version> <app_name>llr321</app_name> <cmdline>-t 36</cmdline> <avg_ncpus>36</avg_ncpus> </app_version> </app_config>

The wu uses all 36 core, but on htop the usage is almost 45% on each core.

Does anyone know what the problem is?
____________
Sorry for my english

ID: 130035 | Reply Quote

stream
Volunteer developer
Volunteer tester
Send message
Joined: 1 Mar 14
Posts: 581
ID: 301928
Credit: 451,697,040
RAC: 86

The wu uses all 36 core, but on htop the usage is almost 45% on each core.

Does anyone know what the problem is?

It seems that LLR multi-threading is not perfect - additional cores should sync with main thread and each other, so they stay idle for significant amount of time. For example, with 4 cores you'll see one core using 100% CPU and 3 at ~75%.

You can benchmark LLR on your setup and find optimal number of cores and LLR instances (it can be done from command line).

Note that LLR multi-threading helps only on CPUs with sufficient amount of internal cache. Since all cores are working on same data, more data could stay in the cache and will be accessed faster, this improves performance so much that it's compensating loss of performance due to partial CPU usage. One some CPUs size of internal cache still may be too small for given project. In this case, enabling multi-threading will make things worse - memory accesses are still missing the cache (so no improvement of speed), but second and following cores are not working at 100%.

ID: 130036 | Reply Quote

Keith

Send message
Joined: 8 Dec 13
Posts: 371
ID: 284516
Credit: 316,935,541
RAC: 198,145

I always recommend testing to see what best suits your hardware.

That said, you have two CPU's with 36 physical cores. Those two CPUs have a link between them to transmit data. If you run a single task on all 36 cores then the task is split between the two CPU's and data will get hung up transferring between the two CPU's.
A better configuration would be running two tasks at a time on 18 cores each so that each task sits on a single CPU.

Given your observation about sub-50% CPU usage I can only guess that you would be able to do two units on 18 cores each in the same time as you can do 1 unit on 36 cores.

After that you can test various combinations of number of tasks versus number of cores to see what gives the most throughput of tasks or gives the fastest of individual tasks.
____________
My Primes
Badge Score: 2*1 + 4*2 + 5*1 + 6*12 + 9*2 + 10*1 = 115

ID: 130040 | Reply Quote

Corla99 [Lombardia]
Send message
Joined: 26 Jun 16
Posts: 2
ID: 450493
Credit: 101,573,914
RAC: 97

Thanks for your help.
At the moment 4 wu are in progress, each with 9 dedicated cores. The cpu usage is around 90%

I tried 2 wu with 18 cores each, but the usage was between 65-75%

Is there a way to know how much cache is in use?
____________
Sorry for my english

ID: 130207 | Reply Quote

Toggle Menu

Join PrimeGrid

Returning Participants

Community

Leader Boards

Results

Other