Hi all,

There are some binaries of the new 3.2.5-dev version available on assembla:

Mac: https://www.assembla.com/code/genefer/subversion/nodes/746/trunk/bin/mac
Windows: https://www.assembla.com/code/genefer/subversion/nodes/746/trunk/bin/windows
Linux: coming soon

This version includes an improved OpenCL transform - should up to 5% faster on some cards, and also has fixed the bug which caused very high MaxErr on Mac. Optimised transforms for AMD CPUs (FMA4 and AVX) are also included. For Intel and pre-Bulldozer AMD CPUs the code will automatically select the fastest transform without having to run benchmarks.

Please go ahead and test the new versions (standalone, on PRPnet or BOINC with app_info.xml) and let us know how it goes.

Thanks

- Iain
____________
Twitter: IainBethune
Proud member of team "Aggie The Pew". Go Aggie!
3073428256125*2^1290000-1 is Prime!

Does not work on my R9-280X (Catalyst 14.4, Win7 x64):

>geneferocl-windows.exe -q "360204^4194304+1"
geneferocl 3.2.5-dev (Windows/OpenCL/32-bit)
Copyright 2001-2014, Yves Gallot
Copyright 2009, Mark Rodenkirch, David Underbakke
Copyright 2010-2012, Shoichiro Yamada, Ken Brazier
Copyright 2011-2014, Iain Bethune, Michael Goetz, Ronald Schneider

Command line: geneferocl-windows.exe -q 360204^4194304+1

Priority change succeeded.

Testing 360204^4194304+1...
Using OCL transform

Running on platform 'AMD Accelerated Parallel Processing', device 'Tahiti', version 'OpenCL 1.2 AMD-APP (1445.5)' and driver '1445.5 (VM)'.

Starting initialization...
Initialization complete (91.940 seconds).
Error: OpenCL error detected: CL_INVALID_KERNEL_ARGS.
An error (2948) occured.

Also crashes immediately on my GTX 645 (Win Vista x64, driver 331.82)

The good news is geneferCUDA seems to work. That's pretty much the only good news, unfortunately.

The Genefer64 (the CPU version) successfully runs the -V command. Anything beyond printing the display banner and it just starts using CPU time but nothing seems to happen. Using "-t -x fma3" (or any other transform) just sits there and doesn't output anything.

GeneferOCL (on a GTX580) does this when running -t:

C:\Temp\GFN\Win\3.2.5>geneferocl-windows_3.2.5.exe -t
geneferocl 3.2.5-dev (Windows/OpenCL/32-bit)
Copyright 2001-2014, Yves Gallot
Copyright 2009, Mark Rodenkirch, David Underbakke
Copyright 2010-2012, Shoichiro Yamada, Ken Brazier
Copyright 2011-2014, Iain Bethune, Michael Goetz, Ronald Schneider

Command line: geneferocl-windows_3.2.5.exe -t

Priority change succeeded.
Running tests for transform implementation "OCL"

Testing 100234^64+1...
Using OCL transform

Running on platform 'NVIDIA CUDA', device 'GeForce GTX 580', version 'OpenCL 1.1 CUDA' and driver '334.89'.


The checkpoint doesn't match current test: 100234^64+1 != 1277444^32768+1. Current test will be restarted
Starting initialization...
Initialization complete (0.001 seconds).
Estimated time remaining for 100234^64+1 is 0:00:00

maxErr exceeded for 100234^64+1, 0.5000 > 0.4500 during final check

AMD HD7970 GPU, Windows 7 64 bit

>geneferocl-windows.exe -q "75898^524288+1"
geneferocl 3.2.5-dev (Windows/OpenCL/32-bit)
Copyright 2001-2014, Yves Gallot
Copyright 2009, Mark Rodenkirch, David Underbakke
Copyright 2010-2012, Shoichiro Yamada, Ken Brazier
Copyright 2011-2014, Iain Bethune, Michael Goetz, Ronald Schneider

Command line: geneferocl-windows.exe -q 75898^524288+1

Priority change succeeded.

Testing 75898^524288+1...
Using OCL transform

Running on platform 'AMD Accelerated Parallel Processing', device 'Tahiti', vers
ion 'OpenCL 1.2 AMD-APP (1445.5)' and driver '1445.5 (VM)'.

Starting initialization...
Initialization complete (3.984 seconds).
Error: OpenCL error detected: CL_INVALID_KERNEL_ARGS.
An error (2948) occured.

>genefer_windows64.exe -q "6960350^32768+1"
genefer 3.2.5-dev (Windows/CPU/64-bit)
Supported transform implementations: sse2 default x87
Copyright 2001-2014, Yves Gallot
Copyright 2009, Mark Rodenkirch, David Underbakke
Copyright 2010-2012, Shoichiro Yamada, Ken Brazier
Copyright 2011-2014, Iain Bethune, Michael Goetz, Ronald Schneider

Command line: genefer_windows64.exe -q 6960350^32768+1

Priority change succeeded.

Testing 6960350^32768+1...
Using SSE2 transform
Starting initialization...
Initialization complete (0.039 seconds).
Testing 6960350^32768+1... 744839 steps to go
maxErr exceeded for 6960350^32768+1, 0.5000 > 0.4500
maxErr exceeded while using SSE2; switching to Default.

Testing 6960350^32768+1...
Using Default transform
Resuming 6960350^32768+1 from a checkpoint (744839 iterations left)

maxErr exceeded for 6960350^32768+1, 0.5000 > 0.4500
maxErr exceeded while using Default; switching to x87 (80-bit).

Testing 6960350^32768+1...
Using x87 (80-bit) transform
Resuming 6960350^32768+1 from a checkpoint (744839 iterations left)
Estimated time remaining for 6960350^32768+1 is 0:32:38
Testing 6960350^32768+1... 741376 steps to go (0:32:06 remaining)
Successful computation progress with x87 (80-bit); switching back to SSE2.

Testing 6960350^32768+1...
Using SSE2 transform
Resuming 6960350^32768+1 from a checkpoint (741375 iterations left)

maxErr exceeded for 6960350^32768+1, 0.5000 > 0.4500
maxErr exceeded while using SSE2; switching to Default.
Too many errors with SSE2; Calculation will proceed using only more accurate tra
nsforms.

Testing 6960350^32768+1...
Using Default transform
Resuming 6960350^32768+1 from a checkpoint (741375 iterations left)

maxErr exceeded for 6960350^32768+1, 1.0000 > 0.4500
maxErr exceeded while using Default; switching to x87 (80-bit).
Too many errors with Default; Calculation will proceed using only more accurate
transforms.

Testing 6960350^32768+1...
Using x87 (80-bit) transform
Resuming 6960350^32768+1 from a checkpoint (741375 iterations left)
Estimated time remaining for 6960350^32768+1 is 0:32:10
6960350^32768+1 is a probable prime. (224220 digits) (err = 0.0067) (time = 0:36:05) 09:05:30

I just committed a new geneferocl-windows.exe binary.
The previous build was not correct: the OpenCL src was an old version, not compatible with CPU API.

I just committed a new geneferocl-windows.exe binary.
The previous build was not correct: the OpenCL src was an old version, not compatible with CPU API.

OpenCL runs now on the GTX 645. However, it is slower than the current app version on PRPnet in many cases:

Current PRPnet version

EDIT: Of course, the new app is faster in the ranges that we can still do GPU work on. :)

Benchmark results for R9-280X (Catalyst 14.4, Win7 x64):

geneferocl 3.2.3

AMD HD7970GHz GPU, AMD X6 1100T CPU, Windows 7 64 bit.

Benchies show an average 7% speedup for the high N's we care about.
N=1048576 even shows a 10.5% speedup:

>primegrid_genefer_3_2_2_0_3.01_windows_intelx86__atiGFN.exe -b
geneferocl 3.2.2 (Windows/OpenCL/32-bit)
Copyright 2001-2014, Yves Gallot
Copyright 2009, Mark Rodenkirch, David Underbakke
Copyright 2010-2012, Shoichiro Yamada, Ken Brazier
Copyright 2011-2014, Iain Bethune, Michael Goetz, Ronald Schneider

Command line: primegrid_genefer_3_2_2_0_3.01_windows_intelx86__atiGFN.exe -b

Priority change succeeded.

Running on platform 'AMD Accelerated Parallel Processing', device 'Tahiti', vers
ion 'OpenCL 1.2 AMD-APP (1445.5)' and driver '1445.5 (VM)'.

Priority change succeeded.
Generalized Fermat Number Bench
Running benchmarks for transform implementation "OCL"
2199064^8192+1 Time: 55.3 us/mul. Err: 0.1934 51956 digits
1798620^16384+1 Time: 55.5 us/mul. Err: 0.1875 102481 digits
1471094^32768+1 Time: 62 us/mul. Err: 0.2031 202102 digits
1203210^65536+1 Time: 90.8 us/mul. Err: 0.1875 398482 digits
984108^131072+1 Time: 136 us/mul. Err: 0.1953 785521 digits
804904^262144+1 Time: 282 us/mul. Err: 0.1953 1548156 digits
658332^524288+1 Time: 518 us/mul. Err: 0.1875 3050541 digits
538452^1048576+1 Time: 1.04 ms/mul. Err: 0.1953 6009544 digits
440400^2097152+1 Time: 2.01 ms/mul. Err: 0.1719 11836006 digits
360204^4194304+1 Time: 4 ms/mul. Err: 0.1719 23305854 digits
294612^8388608+1 Time: 8.28 ms/mul. Err: 0.1641 45879398 digits
Genefer Mark = 101.
Priority change succeeded.

>geneferocl-windows_rev747.exe -b
geneferocl 3.2.5-dev (Windows/OpenCL/32-bit)
Copyright 2001-2014, Yves Gallot
Copyright 2009, Mark Rodenkirch, David Underbakke
Copyright 2010-2012, Shoichiro Yamada, Ken Brazier
Copyright 2011-2014, Iain Bethune, Michael Goetz, Ronald Schneider

Command line: geneferocl-windows_rev747.exe -b

Priority change succeeded.
Priority change succeeded.
Generalized Fermat Number Bench
Running benchmarks for transform implementation "OCL"

Running on platform 'AMD Accelerated Parallel Processing', device 'Tahiti', vers
ion 'OpenCL 1.2 AMD-APP (1445.5)' and driver '1445.5 (VM)'.

2199064^8192+1 Time: 55.1 us/mul. Err: 0.1875 51956 digits
1798620^16384+1 Time: 54.5 us/mul. Err: 0.1875 102481 digits
1471094^32768+1 Time: 59.6 us/mul. Err: 0.1875 202102 digits
1203210^65536+1 Time: 88.9 us/mul. Err: 0.1875 398482 digits
984108^131072+1 Time: 132 us/mul. Err: 0.1797 785521 digits
804904^262144+1 Time: 272 us/mul. Err: 0.1719 1548156 digits
658332^524288+1 Time: 494 us/mul. Err: 0.1797 3050541 digits
538452^1048576+1 Time: 930 us/mul. Err: 0.1758 6009544 digits
440400^2097152+1 Time: 1.88 ms/mul. Err: 0.1719 11836006 digits
360204^4194304+1 Time: 3.78 ms/mul. Err: 0.1621 23305854 digits
294612^8388608+1 Time: 7.69 ms/mul. Err: 0.1621 45879398 digits
Genefer Mark = 109.
Priority change succeeded.

geneferocl 3.2.2 (Windows/OpenCL/32-bit)
...
The upper bound m = 524288, b = 895000, Err = 0.2813
The upper bound m = 1048576, b = 750000, Err = 0.3086
The upper bound m = 2097152, b = 600000, Err = 0.2813
The upper bound m = 4194304, b = 505000, Err = 0.2896
The upper bound m = 8388608, b = 405000, Err = 0.3125

geneferocl 3.2.5-dev (Windows/OpenCL/32-bit)
...
The upper bound m = 524288, b = 905000, Err = 0.2969
The upper bound m = 1048576, b = 750000, Err = 0.2969
The upper bound m = 2097152, b = 625000, Err = 0.3086
The upper bound m = 4194304, b = 505000, Err = 0.3047
The upper bound m = 8388608, b = 405000, Err = 0.2813

Could you try AMD_OCL_BUILD_OPTIONS=-cl-opt-disable
http://www.primegrid.com/forum_thread.php?id=5255&nowrap=true#78922
on ATI cards?
If it is faster on all ATI GPU, it will be the default value of the program.
Thanks.

Could you try AMD_OCL_BUILD_OPTIONS=-cl-opt-disable
http://www.primegrid.com/forum_thread.php?id=5255&nowrap=true#78922
on ATI cards?
If it is faster on all ATI GPU, it will be the default value of the program.
Thanks.

AMD_OCL_BUILD_OPTIONS=-cl-opt-disable is slower for me:

Hi all,

There are some binaries of the new 3.2.5-dev version available on assembla:

Mac: https://www.assembla.com/code/genefer/subversion/nodes/746/trunk/bin/mac
Windows: https://www.assembla.com/code/genefer/subversion/nodes/746/trunk/bin/windows
Linux: coming soon

This version includes an improved OpenCL transform - should up to 5% faster on some cards, and also has fixed the bug which caused very high MaxErr on Mac. Optimised transforms for AMD CPUs (FMA4 and AVX) are also included. For Intel and pre-Bulldozer AMD CPUs the code will automatically select the fastest transform without having to run benchmarks.

Please go ahead and test the new versions (standalone, on PRPnet or BOINC with app_info.xml) and let us know how it goes.

Thanks

- Iain

Verified faster times on different GTX 645s in Win 7 environment using slightly different driver (332.21).

However, using same driver and Win 7 environment, the new app is actually slower or no different on an OEM GTX 660 (only 524288 and 4194304 tests are slightly faster).

Could you try AMD_OCL_BUILD_OPTIONS=-cl-opt-disable
http://www.primegrid.com/forum_thread.php?id=5255&nowrap=true#78922
on ATI cards?

Maybe this is a compile time only option. Seems to make no difference to my system.
I tried "-cl-opt-disable", "-g -O0" and "-O0".

Verified faster times on different GTX 645s in Win 7 environment using slightly different driver (332.21).

However, using same driver and Win 7 environment, the new app is actually slower or no different on an OEM GTX 660 (only 524288 and 4194304 tests are slightly faster).

Hi all,

A new set of binaries has just been posted that should address all the problems reported in the this thread:

Mac: https://www.assembla.com/code/genefer/subversion/nodes/763/trunk/bin/mac
Windows: https://www.assembla.com/code/genefer/subversion/nodes/763/trunk/bin/windows
Linux: https://www.assembla.com/code/genefer/subversion/nodes/763/trunk/bin/linux

Also, it contains updates to the FMA transform code which should be faster, but at least on some processors has shown to slower. If anyone has an FMA-capable CPU please try it out and report your results here. Bug reports, suggestions for improvement etc. are all welcome as usual.

Cheers

- Iain
____________
Twitter: IainBethune
Proud member of team "Aggie The Pew". Go Aggie!
3073428256125*2^1290000-1 is Prime!

Mac: https://www.assembla.com/code/genefer/subversion/nodes/763/trunk/bin/mac
Windows: https://www.assembla.com/code/genefer/subversion/nodes/763/trunk/bin/windows
Linux: https://www.assembla.com/code/genefer/subversion/nodes/763/trunk/bin/linux

Anyone had a chance to test yet?

The Mac and Windows CPU versions of 3.2.5 are live as BOINC version 3.05. Please let me know if you encounter any problems.

The OpenCL versions for Mac and Windows were briefly online, but at least with the Windows versions there seems to have been a problem, so if your computer downloaded GFN or GFN-WR version 3.05 for your GPU, it may have crashed with a computation error. Right now, we're back to using 3.04 or 3.03 of the OpenCL apps, so any new tasks you download should work. Sorry for any inconvenience that may have caused.

Linux apps will be coming soon. CUDA is unchanged.


3.2.5 contains the following enhancements:

OpenCL:
Improved OpenCL b limit (on Mac) and performance.

CPU:
Optimised FMA4 and AVX transforms for AMD CPUs.
Transform auto-selection for Intel and some AMD CPUs.
Faster FMA transform.


____________
My lucky number is 75898⁵²⁴²⁸⁸+1

The Windows and Mac OpenCL (both ATI and Nvidia) apps are now online. They are BOINC version 3.05 for Mac and 3.06 for Windows. Let me know if you experience any difficulties.

The Linux apps are in the pipeline and coming soon. ("Soon" means hours or days. If I had a better estimate, I'd give it to you.)
____________
My lucky number is 75898⁵²⁴²⁸⁸+1

The new Linux apps are now online. Let me know if there's any difficulties.
____________
My lucky number is 75898⁵²⁴²⁸⁸+1

I'm encountering a behavior with the GFN-WR app 3.2.5 (3.06 under Windows 8.0 x64) after shutting down BOINC and restarting. The app won't run at full load on my graphics card, normally at 98%, unless I wait about an hour before restarting BOINC. There are no overheating issues with the card and there is never anything in the stderr.txt file to shed any light on what is happening. Here's a sample of the latest text in the file, although it's truncated because it's just more of the same preceding what I've posted during each shutdown and restart of a given task.

I found a memory handling bug in genefer 3.2.5, in Linux using the valgrind memcheck tool.

backend->SetA uses alen, the number of bytes of data read from the checkpoint file, as the number of elements for the new integer array na. When SetA copies the checkpoint data to the new array (correctly) using memcpy with alen as the number of bytes, it leaves the last 3/4 of na untouched (and uninitialized).

Later Norm tries to shrink the array, testing from the end of the array for elements that are zero. There is a good chance that this fails to reduce the size properly because there is no guarantee that new integer[] performed by SetA zeroes the memory that it allocates.

To fix the problem, if allocating 4 times the number of elements that are read from the checkpoint file is intentional, then array na should be cleared in SetA before memcpy is used. Otherwise array na should be created with the proper number of elements and clearing is unnecessary.

Without examining the code in detail to determine the intent (I suspect it's the latter), I just verified that the safe approach of clearing the na array in SetA, in every backend, fixes the problem according to valgrind memcheck.

I found a memory handling bug in genefer 3.2.5, in Linux using the valgrind memcheck tool. [...]

I found a memory handling bug in genefer 3.2.5, in Linux using the valgrind memcheck tool. [...]

You're right and it's a major bug!
The size (number of bytes) of the integer "a" is written to the checkpoint file, but when a checkpoint is restored, this value is used as a length (number of integers).
I will rewrite this part of the code without mixing size and length types.
Thanks, Yves

Does this affect all versions of Genefer (including OCL and CUDA) on all platforms? The backup and restore code should be identical in all of them, but not necessarily the part of the code that allocates the memory, if I remember correctly.

Does this affect all versions of Genefer (including OCL and CUDA) on all platforms? The backup and restore code should be identical in all of them, but not necessarily the part of the code that allocates the memory, if I remember correctly.

All versions, including CUDA. You remember correctly, previous version of CUDA code was correct

Nlen = alen / sizeof(uint32_t)

but the current version uses "__USE_CPU_INIT" flag and common save and restore functions.

I just committed the fix.

Note that because of double-checking, I don't think that a result of a full test can be wrong. If the exponent is not correctly initialized after a restore on one computer, it will take a different value on another computer and the final residue will be different.
Surprisingly there is no more invalid result with this bug...?

It seems to checkpoint and restore correctly...

With resuming from a checkpoint:

C:\PRPNet\prpclient-5.3.1-windows\prpclient-1>genefer64 -q "123456^8192+1"
genefer 3.2.5 (Windows/CPU/64-bit)
Supported transform implementations: fma3 avx-intel sse4 sse2 default x87
Copyright 2001-2014, Yves Gallot
Copyright 2009, Mark Rodenkirch, David Underbakke
Copyright 2010-2012, Shoichiro Yamada, Ken Brazier
Copyright 2011-2014, Iain Bethune, Michael Goetz, Ronald Schneider

Command line: genefer64 -q 123456^8192+1

Priority change succeeded.

Testing 123456^8192+1...
Using FMA3 transform
Starting initialization...
Initialization complete (0.002 seconds).
Estimated time remaining for 123456^8192+1 is 0:00:04
Testing 123456^8192+1... 126976 steps to go (0:00:06 remaining)
^C caught. Writing checkpoint.

C:\PRPNet\prpclient-5.3.1-windows\prpclient-1>genefer64 -q "123456^8192+1"
genefer 3.2.5 (Windows/CPU/64-bit)
Supported transform implementations: fma3 avx-intel sse4 sse2 default x87
Copyright 2001-2014, Yves Gallot
Copyright 2009, Mark Rodenkirch, David Underbakke
Copyright 2010-2012, Shoichiro Yamada, Ken Brazier
Copyright 2011-2014, Iain Bethune, Michael Goetz, Ronald Schneider

Command line: genefer64 -q 123456^8192+1

Priority change succeeded.

Testing 123456^8192+1...
Using FMA3 transform
Resuming 123456^8192+1 from a checkpoint (124666 iterations left)
Estimated time remaining for 123456^8192+1 is 0:00:03
123456^8192+1 is composite. (RES=85a840e32f44031b) (41710 digits) (err = 0.0005) (time = 0:00:08) 08:47:12

C:\PRPNet\prpclient-5.3.1-windows\prpclient-4>genefer64 -q "123456^8192+1"
genefer 3.2.5 (Windows/CPU/64-bit)
Supported transform implementations: fma3 avx-intel sse4 sse2 default x87
Copyright 2001-2014, Yves Gallot
Copyright 2009, Mark Rodenkirch, David Underbakke
Copyright 2010-2012, Shoichiro Yamada, Ken Brazier
Copyright 2011-2014, Iain Bethune, Michael Goetz, Ronald Schneider

Command line: genefer64 -q 123456^8192+1

Priority change succeeded.

Testing 123456^8192+1...
Using FMA3 transform
Starting initialization...
Initialization complete (0.002 seconds).
Estimated time remaining for 123456^8192+1 is 0:00:04
123456^8192+1 is composite. (RES=85a840e32f44031b) (41710 digits) (err = 0.0005) (time = 0:00:07) 08:46:26

Looking in the database, I do not see an unusually high number of invalid or inconclusive tasks. We tend to get a lot of those due to GPU overclocking errors, so the numbers are higher than they are for LLR, but the numbers are in line from what I'm used to seeing with GFN.

If this bug is causing errors, it's not causing them very often.
____________
My lucky number is 75898⁵²⁴²⁸⁸+1

It seems to checkpoint and restore correctly... [...]

Looking in the database, I do not see an unusually high number of invalid or inconclusive tasks. We tend to get a lot of those due to GPU overclocking errors, so the numbers are higher than they are for LLR, but the numbers are in line from what I'm used to seeing with GFN.

If this bug is causing errors, it's not causing them very often.

We will need to thoroughly test all the different builds and apps before putting them into production, especially since we don't currently seem to have a problem in production.

Yves, thanks for the fix. I think the code is much more self-consistend now.

If I followed things correctly both the read and write checkpoint routines would read/write the size (number of bytes) of a, and then read/write that number of bytes of data, so the data in the checkpoint files should always have been OK.

However, when the data read from the checkpoint is used to initialise the GFN 'Integer' in SetA, then we created an array with too many elements (4x as many), and also the 'len' which is stored is now the size rather than the number of elements. The memcpy is safe as we only copy 'alen' bytes into an array of 'alen*sizeof(unsigned int)' bytes.

So I don't see a correctness issue here, we are just copying around 3x as much junk data as is needed. Also, the problem that valgrind is complaining about is that we do touch uninitialised data. But I don't think that the junk data actually affects anything that goes on in the lower (valid) bytes of the array, right Yves?

If I missed something and there is really a chance of generating incorrect results please let me know!

- Iain
____________
Twitter: IainBethune
Proud member of team "Aggie The Pew". Go Aggie!
3073428256125*2^1290000-1 is Prime!

So I don't see a correctness issue here, we are just copying around 3x as much junk data as is needed. Also, the problem that valgrind is complaining about is that we do touch uninitialised data. But I don't think that the junk data actually affects anything that goes on in the lower (valid) bytes of the array, right Yves?

If I missed something and there is really a chance of generating incorrect results please let me know!

I checked the "last 3/4" that was allocated and not initialized. All these values are zero, on Windows with gcc 4.9.2 or with VS 2013.
I don't see any reason for that but it's lucky and that hides the bug.

I also tested the 3.2.5 app under linux, and it seems to work fine.

Furthermore, I tested both linux and Windows with known PRPs, just in case there's an error in the residue beyond the 16 digits that are displayed. I can't observe any malfunctions.

I can't test the Mac app.

At least under Linux and Windows, it appears that the memory is zeroed out, so the program works despite the error. It probably fails under Valgrind because I imagine Valgrind is initializing the allocated memory to a non-zero pattern.

If my assumptions are correct, then at least on Windows and Linux the app works correctly despite the bug -- unless you're running Valgrind.
____________
My lucky number is 75898⁵²⁴²⁸⁸+1

At least under Linux and Windows, it appears that the memory is zeroed out, so the program works despite the error. It probably fails under Valgrind because I imagine Valgrind is initializing the allocated memory to a non-zero pattern.

If my assumptions are correct, then at least on Windows and Linux the app works correctly despite the bug -- unless you're running Valgrind.

According to my tests on Mac, the new 'na' array always contains zeros. However, as has already been correctly stated, this is not guaranteed, it only happens to be the case because this is likely to be new memory which has never before been allocated by the program. Since we only call read_checkpoint / SetA once (or not at all, for a first time run), it is unlikely but not impossible that we would get non-zeroed memory. This likely explains why we never saw the bug manifest itself (except when valgrind initialises directly with non-zero values).

I don't know how confident to be that the bug never occurred in practice. In theory, all tests which made a checkpoint restart (not including tests which were 'suspended in memory' in BOINC) are potentially exposed. The bug was introduced when the checkpoint file was modified to include the 'a' array in genefer version 3.1.0 (2013-03-04). Thoughts?

- Iain
____________
Twitter: IainBethune
Proud member of team "Aggie The Pew". Go Aggie!
3073428256125*2^1290000-1 is Prime!

Thoughts?

- Iain

1) If the calculation gets corrupted, it's likely to exceed maxErr and error out. We don't see that happening at an accelerated rate. If anything, error rates seem to have gone down.

2) As mentioned before, we're not seeing an unusual rate of inconclusive tests on BOINC. It seems nearly impossible for two to computers both get a corrupted result, not trip maxErr, and somehow get the same exact corrupted result that was caused by random contents of uninitialized memory.

I'm not denying that the potential for errors doesn't exist. Show me a single, repeatable scenario (not involving Valgrind or other artificial modifications of the run time environment) where two different runs of Genefer produce identical erroneous results. For that matter, I have yet to see a single instance where this bug has produced an erroneous result. At the same time, there's plenty of evidence to that Genefer works correctly most of the time -- and possibly all of the time.

Show me a real failure scenario. Then we'll talk about what kind of remediation is necessary, up to and including temporarily shutting down all GFN testing and restesting prior work.

Unless someone can demonstrate that we have a real problem as opposed to a theoretical problem, the plan is to fix genefer, test the new version, and put it into production as quickly as possible. But not before it's thoroughly tested. Putting insufficiently tested software into production is a mistake that's bitten us several times before. I don't want to trade a theoretical problem for a real problem. The fact that this software has already been in production for a while also has affects the decision: If there's a mess we need to clean up, another month won't significantly change the magnitude of the cleanup.
____________
My lucky number is 75898⁵²⁴²⁸⁸+1

Has anyone checked the b limit?

The main problem with shared libraries is that they may not be found on the system.
I often see this kind of error with the linux versions:
http://www.primegrid.com/result.php?resultid=592581943
The versions for Windows use some static versions of the run-time libraries to solve this problem.

Digressing somewhat... how much would using shared libraries conserve CPU cache used by program code, especially from the system libraries?

Those are really good reasons for sticking with statically linked executables for BOINC apps. As said about many things, when you decide to use BOINC to solve a problem, "now you have two problems".

I used to have BOINC problems with GPU libraries too. Mine disappeared when Mike posted 64-bit versions of BOINC apps for Linux. You can see remnants of what didn't work in the 32-bit app:

You can see remnants of what didn't work in the 32-bit app:
[...] it also drove me away from Ubuntu to Linux Mint.

I had a pure 64-bit Ubuntu system, and added the 32-bit Linux compatibility libraries because BOINC demanded it (at the time). But that didn't work with the 64-bit video driver installation. I haven't done the housekeeping to clean out the 32-bit packages yet.

EDIT: The PimeGrid apps demanded it, not BOINC itself.

I fetched the update, and I see the x87 transform SetA method remained unchanged. Is that a mistake, or is there a reason for that?

I fetched the update, and I see the x87 transform SetA method remained unchanged. Is that a mistake, or is there a reason for that?

Is __USE_CPU_INIT now the only approved method for 3.2.6? The alternative doesn't work in CUDASetA because now alen is not defined.

Is __USE_CPU_INIT now the only approved method for 3.2.6? The alternative doesn't work in CUDASetA because now alen is not defined.

Where do we stand with this? Should we be doing production builds and starting testing?
____________
My lucky number is 75898⁵²⁴²⁸⁸+1

Where do we stand with this? Should we be doing production builds and starting testing?

I've posted 3.2.6-dev builds on the SVN (waiting for Linux, hopefully not long).

Mac: https://www.assembla.com/code/genefer/subversion/nodes/776/trunk/bin/mac

Windows: https://www.assembla.com/code/genefer/subversion/nodes/776/trunk/bin/windows

Linux: https://www.assembla.com/code/genefer/subversion/nodes/777/trunk/bin/linux

Testing protocol to come shortly.

- Iain
____________
Twitter: IainBethune
Proud member of team "Aggie The Pew". Go Aggie!
3073428256125*2^1290000-1 is Prime!

I've downloaded your Linux 3.2.6 executables (rev 777) (cuda 5.5, 6.5, and OCL, all 64-bit) if I can help. LMK. Ubuntu 14.04, on a gtx770.

G

I have posted a google sheet containing details of all the candidates and combinations of hardware that we want to test :

https://docs.google.com/spreadsheets/d/1StsmFX5Vomr5i6OCYxdvIlriD09gS55pKFPXSBTT4oU

There are quite a lot of tests in total, so anyone who is able to help - please visit the google sheet, reserve yourself some tests and start running! Please note the instruction to do at least one checkpoint/restart during every test.

The binaries for testing are available from:

Mac: https://www.assembla.com/code/genefer/subversion/nodes/776/trunk/bin/mac
Windows: https://www.assembla.com/code/genefer/subversion/nodes/776/trunk/bin/windows
Linux: https://www.assembla.com/code/genefer/subversion/nodes/777/trunk/bin/linux

Thanks in advance for your help! Hopefully it won't take to long before we can make the public release - the code should be slightly faster than the previous version.

- Iain
____________
Twitter: IainBethune
Proud member of team "Aggie The Pew". Go Aggie!
3073428256125*2^1290000-1 is Prime!

Just to be sure, is it mandatory to run - for example - sse2 test on sse2 hardware?
____________
My stats

Just to be sure, is it mandatory to run - for example - sse2 test on sse2 hardware?

Please note, I have also added columns to note whether you have tested using the 32 bit or 64 bit executable. Can those who already ran tests please go back and fill these in. We do not need to run every test with both binaries, but want to cover both on every platform.

Cheers

- Iain

____________
Twitter: IainBethune
Proud member of team "Aggie The Pew". Go Aggie!
3073428256125*2^1290000-1 is Prime!

Also added a grid for testing integration with PRPNet and BOINC (with app_info.xml).

Thanks everyone for all your help so far!

Cheers

- Iain
____________
Twitter: IainBethune
Proud member of team "Aggie The Pew". Go Aggie!
3073428256125*2^1290000-1 is Prime!

Updated the BOINC/PRPNet testing table to make sure all the apps that we will use are tested.
____________
Twitter: IainBethune
Proud member of team "Aggie The Pew". Go Aggie!
3073428256125*2^1290000-1 is Prime!

This task has the expected result but the calculation of the remaining time seems to be buggy.

Hi Artist, was that timing problem repeatable? I don't really know what to make of it...

- Iain
____________
Twitter: IainBethune
Proud member of team "Aggie The Pew". Go Aggie!
3073428256125*2^1290000-1 is Prime!

We are still in need of testers who have:

CUDA and/or OpenCL devices on Linux
CUDA devices on Mac

If you can help with testing, please post here or PM me. It's not too hard to run the tests, and we can help!

- Iain
____________
Twitter: IainBethune
Proud member of team "Aggie The Pew". Go Aggie!
3073428256125*2^1290000-1 is Prime!

I will attempt the Linux CUDA tests this afternoon. I can give the BOINC CUDA/OCL a shot (both on an Nvidia GTX580) if the initial tests work out, but I don't know how to set up an appinfo for Primegrid. Help, please?
____________
Eating more cheese on Thursdays.

After finally getting 32bit Linux and nVidia/CUDA/OpenCL drivers to install (I now remember why I prefer ATI/AMD graphics on my linux systems...), the final test is in progress and I'll check it in the morning. I am using the cuda55 version to test with, I tried getting cuda32 to run, but the necessary files aren't included in the latest cuda dev packages. I can try to get this working tomorrow or Saturday if needed.

I will note that on my GTX 580, the OpenCL version takes roughly 2/3 the time of the CUDA version to complete the first 2 tests.
____________
Eating more cheese on Thursdays.

Hi Grebuloner,

Thank for running the Linux tests! If it's possible for you also to do the tests under BOINC that would be great.

To set up an app_info, the basics that you need are as follows:

1) Make sure you have no outstanding tasks running, they will be abandonded.
2) Shut down your BOINC client
3) in your BOINC_DIR/projects/www.primegrid.com put the following app_info.xml file:

Thank you, Iain! The first BOINC task is just starting, and when I get home from work tonight I'll change the app over to OCL and get a second one in.
____________
Eating more cheese on Thursdays.

Thanks, that's been a great help!

- Iain
____________
Twitter: IainBethune
Proud member of team "Aggie The Pew". Go Aggie!
3073428256125*2^1290000-1 is Prime!

I just put 3.2.6 into production on BOINC as 3.07. Let us know if anything's not working correctly.

____________
My lucky number is 75898⁵²⁴²⁸⁸+1

I just put 3.2.6 into production on BOINC as 3.07. Let us know if anything's not working correctly.

Strange slowdown on I have a mac pro 4,1 with an r9 280x in it (http://www.primegrid.com/show_host_detail.php?hostid=475557), and an old quad core i7 running windows, also with a 280x in it (http://www.primegrid.com/show_host_detail.php?hostid=478715).

The i7 seems to have done ok with the updated version:

609776452 417260915 24 Feb 2015 | 7:39:51 UTC 27 Feb 2015 | 0:21:33 UTC Completed, waiting for validation 207,446.93 8,330.11 pending Genefer (World Record) v3.07 (atiGFNWR)
609075192 417249133 21 Feb 2015 | 17:50:43 UTC 24 Feb 2015 | 5:24:17 UTC Completed, waiting for validation 205,887.41 8,262.92 pending Genefer (World Record) v3.07 (atiGFNWR)
608174925 417221652 19 Feb 2015 | 4:37:34 UTC 21 Feb 2015 | 15:31:48 UTC Completed, waiting for validation 206,113.43 8,435.72 pending Genefer (World Record) v3.07 (atiGFNWR)
607132296 417221584 15 Feb 2015 | 15:48:11 UTC 19 Feb 2015 | 6:16:36 UTC Completed, waiting for validation 205,349.62 9,116.70 pending Genefer (World Record) v3.07 (atiGFNWR)
606735461 417201233 13 Feb 2015 | 18:38:40 UTC 16 Feb 2015 | 16:18:35 UTC Completed and validated 208,006.03 8,183.44 629,966.70 Genefer (World Record) v3.07 (atiGFNWR)
606239148 417107778 11 Feb 2015 | 12:41:19 UTC 14 Feb 2015 | 6:22:02 UTC Completed and validated 207,590.63 7,933.66 628,719.65 Genefer (World Record) v3.06 (atiGFNWR)
605035091 417147631 8 Feb 2015 | 7:26:56 UTC 11 Feb 2015 | 20:37:05 UTC Completed and validated 207,792.49 6,757.72 629,336.28 Genefer (World Record) v3.06 (atiGFNWR)


Not so with the mac:
606684846 417201244 14 Feb 2015 | 9:05:04 UTC 21 Feb 2015 | 23:00:40 UTC Completed and validated 649,986.67 27,239.87 629,969.57 Genefer (World Record) v3.07 (openclGFNWRMAC)
606227125 416913061 11 Feb 2015 | 9:42:35 UTC 14 Feb 2015 | 9:20:47 UTC Completed and validated 256,539.17 24,888.08 627,232.32 Genefer (World Record) v3.05 (openclGFNWRMAC)
605173845 417154291 8 Feb 2015 | 8:51:59 UTC 11 Feb 2015 | 10:03:12 UTC Completed and validated 257,357.95 24,750.17 629,371.15 Genefer (World Record) v3.05 (openclGFNWRMAC)
602710163 417000286 30 Jan 2015 | 11:14:30 UTC 2 Feb 2015 | 11:23:17 UTC Completed and validated 258,671.68 24,908.65 627,781.02 Genefer (World Record) v3.05 (openclGFNWRMAC)
602005390 416498084 27 Jan 2015 | 10:00:56 UTC 30 Jan 2015 | 11:32:05 UTC Completed and validated 261,088.38 25,024.37 625,347.09 Genefer (World Record) v3.05 (openclGFNWRMAC)
596750227 416315557 2 Jan 2015 | 7:55:58 UTC 5 Jan 2015 | 11:41:03 UTC Completed and validated 268,710.61 21,861.26 624,665.28 Genefer (World Record) v3.05 (openclGFNWRMAC)
595992868 417263805 21 Feb 2015 | 23:51:07 UTC 15 Mar 2015 | 0:51:07 UTC In progress --- --- --- Genefer (World Record) v3.07 (openclGFNWRMAC)

Any ideas, with completion time more than doubling?

Strange slowdown [...]
Any ideas, with completion time more than doubling?

Van Zimmerman wrote:

Strange slowdown [...]
Any ideas, with completion time more than doubling?

Did the FFT size increase in the slower tests?