Author |
Message |
Reggie Volunteer moderator Volunteer tester Send message
Joined: 10 May 14 Posts: 32 ID: 311759 Credit: 80,487,785 RAC: 657
                   
|
I have been running a GFN-WR task for over a day now, but the CPU usage has not dropped below 97%. I am using Gentoo Linux. I installed the closed-source Nvidia drivers instead of the open source ones. Is there any reason for the high CPU usage? |
|
|
Yves GallotVolunteer developer Project scientist Send message
Joined: 19 Aug 12 Posts: 524 ID: 164101 Credit: 304,715,793 RAC: 5,160

|
I have been running a GFN-WR task for over a day now, but the CPU usage has not dropped below 97%. I am using Gentoo Linux. I installed the closed-source Nvidia drivers instead of the open source ones. Is there any reason for the high CPU usage?
See the thread http://www.primegrid.com/forum_thread.php?id=7364
Linux driver is the reason.
You can try this
https://www.reddit.com/r/EtherMining/comments/600lpx/high_cpu_load_with_genoil/ |
|
|
Reggie Volunteer moderator Volunteer tester Send message
Joined: 10 May 14 Posts: 32 ID: 311759 Credit: 80,487,785 RAC: 657
                   
|
Do you mean telling Xserver to run on integrated graphics? I'm not even running an Xserver right now and there's no display connected to the computer. |
|
|
Yves GallotVolunteer developer Project scientist Send message
Joined: 19 Aug 12 Posts: 524 ID: 164101 Credit: 304,715,793 RAC: 5,160

|
Do you mean telling Xserver to run on integrated graphics? I'm not even running an Xserver right now and there's no display connected to the computer.
Yes but if there's no display connected this can't resolve the problem.
The CPU usage is close to zero with the same code running on Windows... why?
I found https://bitcointalk.org/index.php?topic=181328.0
but I don't have access to a Linux box for testing.
|
|
|
Reggie Volunteer moderator Volunteer tester Send message
Joined: 10 May 14 Posts: 32 ID: 311759 Credit: 80,487,785 RAC: 657
                   
|
I think that fixed it, though my way is really hacky. I ended up having to export the environment variable mentioned in the post you linked before starting boinc, meaning everything boinc runs will be started with that variable.
Now, is there a way I can tell boinc to only start the gfn applications like that? I know about command line options in an app_config file, but does that work with environment variables? |
|
|
compositeVolunteer tester Send message
Joined: 16 Feb 10 Posts: 733 ID: 55391 Credit: 575,991,255 RAC: 320,921
                     
|
I think that fixed it, though my way is really hacky. I ended up having to export the environment variable mentioned in the post you linked before starting boinc, meaning everything boinc runs will be started with that variable.
Now, is there a way I can tell boinc to only start the gfn applications like that? I know about command line options in an app_config file, but does that work with environment variables? You don't need to use the YIELD_SLEEP_TIME environment variable. The default value is hardcoded, so you can change that when you compile the preload library. The lib_init function is just there for flexibility, so you can omit the lib_init function if you want. |
|
|
compositeVolunteer tester Send message
Joined: 16 Feb 10 Posts: 733 ID: 55391 Credit: 575,991,255 RAC: 320,921
                     
|
Do you mean telling Xserver to run on integrated graphics? I'm not even running an Xserver right now and there's no display connected to the computer.
Yes but if there's no display connected this can't resolve the problem.
The CPU usage is close to zero with the same code running on Windows... why?
I found https://bitcointalk.org/index.php?topic=181328.0
but I don't have access to a Linux box for testing.
I can test this on my system, I've been messing with GFN code and I have the Makefile set up the way I want. What branch should I checkout? |
|
|
Yves GallotVolunteer developer Project scientist Send message
Joined: 19 Aug 12 Posts: 524 ID: 164101 Credit: 304,715,793 RAC: 5,160

|
I can test this on my system, I've been messing with GFN code and I have the Makefile set up the way I want. What branch should I checkout?
The trunk is stable. |
|
|
Reggie Volunteer moderator Volunteer tester Send message
Joined: 10 May 14 Posts: 32 ID: 311759 Credit: 80,487,785 RAC: 657
                   
|
I think that fixed it, though my way is really hacky. I ended up having to export the environment variable mentioned in the post you linked before starting boinc, meaning everything boinc runs will be started with that variable.
Now, is there a way I can tell boinc to only start the gfn applications like that? I know about command line options in an app_config file, but does that work with environment variables? You don't need to use the YIELD_SLEEP_TIME environment variable. The default value is hardcoded, so you can change that when you compile the preload library. The lib_init function is just there for flexibility, so you can omit the lib_init function if you want.
I still have to have the LD_PRELOAD variable though, unless there's another way of doing it I don't know. |
|
|
compositeVolunteer tester Send message
Joined: 16 Feb 10 Posts: 733 ID: 55391 Credit: 575,991,255 RAC: 320,921
                     
|
I think that fixed it, though my way is really hacky. I ended up having to export the environment variable mentioned in the post you linked before starting boinc, meaning everything boinc runs will be started with that variable.
Now, is there a way I can tell boinc to only start the gfn applications like that? I know about command line options in an app_config file, but does that work with environment variables? You don't need to use the YIELD_SLEEP_TIME environment variable. The default value is hardcoded, so you can change that when you compile the preload library. The lib_init function is just there for flexibility, so you can omit the lib_init function if you want.
I still have to have the LD_PRELOAD variable though, unless there's another way of doing it I don't know.
The function you are replacing is used in OpenCL so it has to be resolved before loading libraries. If only OpenCL is calling sched_yield() then there there is nothing to be concerned about in the other apps. |
|
|
compositeVolunteer tester Send message
Joined: 16 Feb 10 Posts: 733 ID: 55391 Credit: 575,991,255 RAC: 320,921
                     
|
On Linux, it doesn't look good for small GFN, but there is tunable potential to drive the GPU to 100% with low CPU utilization for large GFN.
Testing was with a 3.3 GHz 5820K with a GTX 760. In all cases, without a microsecond sleep, a GFN task takes 100% of a CPU thread.
Then I added a microsecond sleep in HostProgram.h after each call to clEnqueueNDRangeKernel(). So:
- one right after that function call in Execute(), and
- another after the error check following the function call in ExecuteProfiling().
I ran tests with 272042236^(2^N)+1 and this selected OCL2 in every case. These are the estimated runtimes reported by the program.
The CPU has a hard time feeding the GPU when GFN workunits are small, so introducing any sleep for GFN17, or smaller, lengthens the GPU computation time while reducing GPU and CPU utilization.
Microsecond sleeps cause interesting things to happen to the estimated runtime and the % of CPU and GPU for GFN18 and above. With a 1 microsecond sleep, we start to see significant reduction in %CPU utilization.
runtimes
task no sleep sleep=1 %CPU %GPU
GFN18 1:27:00 1:29:00 50 99
GFN19 5:52:00 5:52:00 75 100
GFN20 22:50:00 23:10:00 88 100
GFN21 96:10:00 96:00:00 94 100
Two microseconds of sleep had the same results as 1 microsecond.
With a 1000 microsecond sleep:
task runtime %CPU %GPU
GFN18 16:00:00 5 10
GFN19 32:00:00 5 20
GFN20 62:10:00 4 38
GFN21 129:10:00 5 75
With a 500 microsecond sleep:
task runtime %CPU %GPU
GFN18 9:01:00 9 37
GFN19 18:00:00 8 34
GFN20 35:10:00 8 63
GFN21 95:50:00 32 100
The sleep for each task can be tuned for a given CPU and GPU combination to get the minimum %CPU that produces 100% GPU. In my case:
task runtime %CPU %GPU sleep
GFN18 1:28:00 30 99 20
GFN19 5:55:00 13 100 135
GFN20 23:00:00 9 100 300
GFN21 96:00:00 18 100 600
There is a range of sleep values that alters %CPU while retaining 100% GPU, where the potential value of extracting a small incremental %CPU savings is offset by increasing the task runtime and GPU residency.
It would desirable to automatically adjust the sleep value to reduce GPU utilization when the computer is in use as a workstation. Compare this with the all-or-nothing choice for GPU crunching given by BOINC. For example, GFN21 could normally run with a sleep value of 600 but change this on-the-fly to 1000 when a user is present.
EDIT: Without much work a tuning parameter for sleep microseconds can be added to the genefer command line which would be used in OpenCL tasks. A different value for this parameter can be added to each app's command line in app_config.xml. |
|
|
Reggie Volunteer moderator Volunteer tester Send message
Joined: 10 May 14 Posts: 32 ID: 311759 Credit: 80,487,785 RAC: 657
                   
|
Thanks for that data. So, from your results, the default value of YIELD_SLEEP_TIME=1000 seems to be pretty good? And the larger the workunit, the higher value YIELD_SLEEP_TIME should be? On a side note, I'm now running AP27 tasks on my gpu with the same library, and it's only using ~2.9% cpu. I don't have data to compare that with right now, but it seems like that would be ideal. |
|
|
compositeVolunteer tester Send message
Joined: 16 Feb 10 Posts: 733 ID: 55391 Credit: 575,991,255 RAC: 320,921
                     
|
Your best value of YIELD_SLEEP would be different than mine. It depends on your hardware and the application you run. In my case 1000 is a not a good value for any GFN task. It's way too much for GFNs smaller than GFN22, and not enough for GFN22 or higher. My "best" value for GFN22 is 1610 (all these are found by trial and error: initial guess then bisection). GFN could be altered to dynamically find it's own best values, doing so with several iterations of tests, by setting a target runtime of (for example) 0.5% over the no-sleep runtime. |
|
|
Reggie Volunteer moderator Volunteer tester Send message
Joined: 10 May 14 Posts: 32 ID: 311759 Credit: 80,487,785 RAC: 657
                   
|
Okay, one last question: BOINC shows CPU usage for each task, but how did you figure out the GPU usage? |
|
|
No.15 Send message
Joined: 11 Jun 16 Posts: 26 ID: 449361 Credit: 889,495,407 RAC: 1,486,582
                 
|
Okay, one last question: BOINC shows CPU usage for each task, but how did you figure out the GPU usage?
On windows you can use gpu-z in linux I use the nvidia xserver app
____________
My Primes :) 3060772^262144+1 and 3673932^262144+1 |
|
|
compositeVolunteer tester Send message
Joined: 16 Feb 10 Posts: 733 ID: 55391 Credit: 575,991,255 RAC: 320,921
                     
|
Okay, one last question: BOINC shows CPU usage for each task, but how did you figure out the GPU usage?
Like No. 15 said. "NVIDIA X Server Settings" is the GUI started by the nvidia-settings command.
I use the "htop" command to see process information.
|
|
|
|
Yeah, it worked for me!
My 750 Ti is fine with
useconds_t yield_sleep_time = 50000;
running GFN16.
CPU usage <=13%
GPU usage = 98% |
|
|
|
Could you let me know where should I put this line? |
|
|
|
Download libsleep.c.
Edit that yield_sleep_time variable.
Save.
Compile by the commented command in libsleep.c.
On Linux, start client by a command like this
cd /your/boinc/path
LD_PRELOAD="your/libsleep/path/libsleep.so" ./boinc for standalone BOINC
and
LD_PRELOAD="your/libsleep/path/libsleep.so" boinc for installed BOINC.
Or
Do not edit libsleep.c and simply compile it.
Then you can use a bash variable to set your customized time.
YIELD_SLEEP_TIME="500"
LD_PRELOAD="your/libsleep/path/libsleep.so" boinc |
|
|
|
Correction: bash variable should be on the same line, although it does not read it.
YIELD_SLEEP_TIME="500" LD_PRELOAD="/your/libsleep/path/libsleep.so" boinc |
|
|
dthonon Volunteer tester Send message
Joined: 6 Dec 17 Posts: 373 ID: 957147 Credit: 1,112,640,203 RAC: 542,672
                         
|
And, if it is running as a service, edit /lib/systemd/system/boinc-client.service and modify the following line
ExecStart=/bin/sh -c 'YIELD_SLEEP_TIME="500" LD_PRELOAD="/your/libsleep/path/libsleep.so" /usr/bin/boinc --dir /var/lib/boinc-client >/var/log/boinc.log 2>/var/log/boincerr.log'
CPU reduced from 100% to less than 20%. |
|
|