PrimeGrid
Please visit donation page to help the project cover running costs for this month

Toggle Menu

Join PrimeGrid

Returning Participants

Community

Leader Boards

Results

Other

drummers-lowrise

Advanced search

Message boards : Problems and Help : Problem with multi-threading and app_config

Author Message
Profile Corla99 [Lombardia]
Send message
Joined: 26 Jun 16
Posts: 2
ID: 450493
Credit: 101,573,914
RAC: 97
Found 1 prime in the 2019 Tour de Primes321 LLR Amethyst: Earned 1,000,000 credits (1,035,071)Cullen LLR Amethyst: Earned 1,000,000 credits (1,223,962)ESP LLR Amethyst: Earned 1,000,000 credits (1,318,807)Generalized Cullen/Woodall LLR Gold: Earned 500,000 credits (549,624)PPS LLR Gold: Earned 500,000 credits (505,573)PSP LLR Gold: Earned 500,000 credits (638,353)SoB LLR Gold: Earned 500,000 credits (671,709)SR5 LLR Gold: Earned 500,000 credits (507,688)SGS LLR Gold: Earned 500,000 credits (557,712)TRP LLR Gold: Earned 500,000 credits (508,378)Woodall LLR Silver: Earned 100,000 credits (499,988)Generalized Cullen/Woodall Sieve (suspended) Gold: Earned 500,000 credits (528,628)PPS Sieve Emerald: Earned 50,000,000 credits (50,177,335)AP 26/27 Turquoise: Earned 5,000,000 credits (6,735,638)GFN Jade: Earned 10,000,000 credits (15,274,614)PSA Sapphire: Earned 20,000,000 credits (20,832,790)
Message 130035 - Posted: 31 May 2019 | 21:11:54 UTC
Last modified: 31 May 2019 | 21:12:21 UTC

Hi, today I moved my pc with 2xE5-2699 on this project. Before doing this, I've disabled the hyper-threading and created an app_config:

<app_config> <app> <name>llr321</name> <fraction_done_exact/> </app> <app_version> <app_name>llr321</app_name> <cmdline>-t 36</cmdline> <avg_ncpus>36</avg_ncpus> </app_version> </app_config>


The wu uses all 36 core, but on htop the usage is almost 45% on each core.

Does anyone know what the problem is?
____________
Sorry for my english

stream
Volunteer developer
Volunteer tester
Send message
Joined: 1 Mar 14
Posts: 581
ID: 301928
Credit: 451,697,040
RAC: 86
Discovered 1 mega primeFound 1 prime in the 2018 Tour de PrimesFound 1 prime in the 2019 Tour de Primes321 LLR Turquoise: Earned 5,000,000 credits (9,919,609)Cullen LLR Turquoise: Earned 5,000,000 credits (9,934,320)ESP LLR Turquoise: Earned 5,000,000 credits (9,909,084)Generalized Cullen/Woodall LLR Turquoise: Earned 5,000,000 credits (5,921,052)PPS LLR Turquoise: Earned 5,000,000 credits (7,262,900)PSP LLR Turquoise: Earned 5,000,000 credits (5,089,560)SoB LLR Turquoise: Earned 5,000,000 credits (5,824,522)SR5 LLR Turquoise: Earned 5,000,000 credits (5,399,087)SGS LLR Turquoise: Earned 5,000,000 credits (5,436,940)TRP LLR Turquoise: Earned 5,000,000 credits (9,911,706)Woodall LLR Turquoise: Earned 5,000,000 credits (5,011,851)321 Sieve Sapphire: Earned 20,000,000 credits (20,004,228)Generalized Cullen/Woodall Sieve (suspended) Sapphire: Earned 20,000,000 credits (20,047,667)PPS Sieve Sapphire: Earned 20,000,000 credits (20,866,490)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Sapphire: Earned 20,000,000 credits (20,043,271)TRP Sieve (suspended) Sapphire: Earned 20,000,000 credits (20,015,177)AP 26/27 Sapphire: Earned 20,000,000 credits (20,045,194)GFN Emerald: Earned 50,000,000 credits (50,752,940)PSA Double Silver: Earned 200,000,000 credits (200,301,443)
Message 130036 - Posted: 31 May 2019 | 21:38:53 UTC - in response to Message 130035.

The wu uses all 36 core, but on htop the usage is almost 45% on each core.

Does anyone know what the problem is?

It seems that LLR multi-threading is not perfect - additional cores should sync with main thread and each other, so they stay idle for significant amount of time. For example, with 4 cores you'll see one core using 100% CPU and 3 at ~75%.

You can benchmark LLR on your setup and find optimal number of cores and LLR instances (it can be done from command line).

Note that LLR multi-threading helps only on CPUs with sufficient amount of internal cache. Since all cores are working on same data, more data could stay in the cache and will be accessed faster, this improves performance so much that it's compensating loss of performance due to partial CPU usage. One some CPUs size of internal cache still may be too small for given project. In this case, enabling multi-threading will make things worse - memory accesses are still missing the cache (so no improvement of speed), but second and following cores are not working at 100%.

KeithProject donor
Avatar
Send message
Joined: 8 Dec 13
Posts: 371
ID: 284516
Credit: 316,935,541
RAC: 198,145
Discovered 1 mega primeFound 2 primes in the 2018 Tour de PrimesFound 3 primes in the 2019 Tour de Primes321 LLR Turquoise: Earned 5,000,000 credits (5,010,954)Cullen LLR Turquoise: Earned 5,000,000 credits (5,009,911)ESP LLR Turquoise: Earned 5,000,000 credits (5,005,551)Generalized Cullen/Woodall LLR Turquoise: Earned 5,000,000 credits (5,002,124)PPS LLR Turquoise: Earned 5,000,000 credits (6,011,297)PSP LLR Ruby: Earned 2,000,000 credits (4,876,647)SoB LLR Turquoise: Earned 5,000,000 credits (6,751,021)SR5 LLR Turquoise: Earned 5,000,000 credits (5,401,121)SGS LLR Turquoise: Earned 5,000,000 credits (5,009,588)TRP LLR Turquoise: Earned 5,000,000 credits (5,759,074)Woodall LLR Turquoise: Earned 5,000,000 credits (5,053,050)321 Sieve Turquoise: Earned 5,000,000 credits (5,002,887)Generalized Cullen/Woodall Sieve (suspended) Turquoise: Earned 5,000,000 credits (5,004,494)PPS Sieve Double Bronze: Earned 100,000,000 credits (100,111,958)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Amethyst: Earned 1,000,000 credits (1,002,980)TRP Sieve (suspended) Amethyst: Earned 1,000,000 credits (1,010,755)AP 26/27 Emerald: Earned 50,000,000 credits (63,758,110)GFN Emerald: Earned 50,000,000 credits (81,711,176)PSA Silver: Earned 100,000 credits (442,843)
Message 130040 - Posted: 31 May 2019 | 23:11:02 UTC

I always recommend testing to see what best suits your hardware.

That said, you have two CPU's with 36 physical cores. Those two CPUs have a link between them to transmit data. If you run a single task on all 36 cores then the task is split between the two CPU's and data will get hung up transferring between the two CPU's.
A better configuration would be running two tasks at a time on 18 cores each so that each task sits on a single CPU.

Given your observation about sub-50% CPU usage I can only guess that you would be able to do two units on 18 cores each in the same time as you can do 1 unit on 36 cores.

After that you can test various combinations of number of tasks versus number of cores to see what gives the most throughput of tasks or gives the fastest of individual tasks.
____________
My Primes
Badge Score: 2*1 + 4*2 + 5*1 + 6*12 + 9*2 + 10*1 = 115

Profile Corla99 [Lombardia]
Send message
Joined: 26 Jun 16
Posts: 2
ID: 450493
Credit: 101,573,914
RAC: 97
Found 1 prime in the 2019 Tour de Primes321 LLR Amethyst: Earned 1,000,000 credits (1,035,071)Cullen LLR Amethyst: Earned 1,000,000 credits (1,223,962)ESP LLR Amethyst: Earned 1,000,000 credits (1,318,807)Generalized Cullen/Woodall LLR Gold: Earned 500,000 credits (549,624)PPS LLR Gold: Earned 500,000 credits (505,573)PSP LLR Gold: Earned 500,000 credits (638,353)SoB LLR Gold: Earned 500,000 credits (671,709)SR5 LLR Gold: Earned 500,000 credits (507,688)SGS LLR Gold: Earned 500,000 credits (557,712)TRP LLR Gold: Earned 500,000 credits (508,378)Woodall LLR Silver: Earned 100,000 credits (499,988)Generalized Cullen/Woodall Sieve (suspended) Gold: Earned 500,000 credits (528,628)PPS Sieve Emerald: Earned 50,000,000 credits (50,177,335)AP 26/27 Turquoise: Earned 5,000,000 credits (6,735,638)GFN Jade: Earned 10,000,000 credits (15,274,614)PSA Sapphire: Earned 20,000,000 credits (20,832,790)
Message 130207 - Posted: 6 Jun 2019 | 22:16:45 UTC

Thanks for your help.
At the moment 4 wu are in progress, each with 9 dedicated cores. The cpu usage is around 90%

I tried 2 wu with 18 cores each, but the usage was between 65-75%


Is there a way to know how much cache is in use?
____________
Sorry for my english

Post to thread

Message boards : Problems and Help : Problem with multi-threading and app_config

[Return to PrimeGrid main page]
DNS Powered by DNSEXIT.COM
Copyright © 2005 - 2019 Rytis Slatkevičius (contact) and PrimeGrid community. Server load 0.71, 1.14, 1.27
Generated 15 Dec 2019 | 13:16:03 UTC