PrimeGrid
Please visit donation page to help the project cover running costs for this month

Toggle Menu

Join PrimeGrid

Returning Participants

Community

Leader Boards

Results

Other

drummers-lowrise
1) Message boards : Sieving : DC on sieving using CPU is Only waste of TIME. (Message 78785)
Posted 1990 days ago by Profile Lennart SM5YMTProject donor
I find the reasoning made earlier in this thread regarding detecting errors in the software to be compelling. Sure, if we only miss 1% of the factors and therefore needlessly LLR them, OK, I can see where sieving double-check in that case is overkill and a waste. But what if we're missing 50% of the factors. Or 90%. We don't know what that ratio is without a reasonable mechanism to notice it.

Do we put untested software out there? No, hopefully not. I've tested here myself quite a bit. But software is written by humans, and tested by humans, and humans make mistakes... no matter how much "beta" test is done. Maybe the double-check is like a "peer review" prior to publication in academic journals (that's far from a perfect analogy, I know).

As for the "cheating" aspect, well, detecting cheating is good, just on a moral basis, and that makes me feel good and want to continue to contribute. As for *why* someone would want to cheat, I believe I understand the reasons, but they are better discussed by a psychologist than a computer programmer, and is off-topic, I guess.

Perhaps some guidance can be provided by the number of "invalid" tasks that come out of the current TRP Sieve challenge.
Cheers,
--Gary



Valid 527333
Invalid 2

Lennart
2) Message boards : Sieving : DC on sieving using CPU is Only waste of TIME. (Message 78771)
Posted 1991 days ago by Profile Lennart SM5YMTProject donor
Double check everything, without exception.

--G



I think you need to explain why you think that way.

I also think we need to ask our self " why are we sieving"


It seems that many don't know the purpose of sieving.

What do we gain with sieving.

Lennart
3) Message boards : Sieving : DC on sieving using CPU is Only waste of TIME. (Message 78721)
Posted 1992 days ago by Profile Lennart SM5YMTProject donor
1) Do all sieves drop off the way GFN does?
2) Anyone want to try to come up with a formula for how many fewer factors we'll find because of DC? It's clearly less than 50%, and my gut feeling is it's a lot less than 50%. The question is how much less.

1. Yes-ish.
2. I'll leave out the calculation for GFN sieve, but for the others, use this simple formula. Number of candidates sieved out by sieving from p1 to p2 is N*(1-log(p1)/log(p2)), where N is the number of candidates we have after sieving to p1.

If we take p1=1P (10^15), and p2=2P. That amounts to N*0.019 or about 1.9% candidates sieved out. Feel free to plugin different values for p1 and p2.

Incidentally, due to the existence of this formula, we can pretty accurately estimate how much factors we're supposed to find for specific (large-ish) sieve ranges.

EDIT:- Formula not applicable for SGS where we do triple or quad sieve. It is not part of this discussion, but just thought everyone should know.


Thanks, axn.

That's lower than I thought. Cutting the sieve duration in half reduces the number of factors found by less than 2 percent. Playing with the numbers a bit, it looks like the percentage goes down as p1 and p2 get higher.

That's very, very helpful. Thanks!



That is not changing anything .
If the sieve to 1P takes a year it will be 2 year with DC.

Lennart
4) Message boards : Sieving : DC on sieving using CPU is Only waste of TIME. (Message 78706)
Posted 1993 days ago by Profile Lennart SM5YMTProject donor
I want to put a few misconceptions to rest.

First of all, this idea isn't up for a vote, and I'm not soliciting opinions. This is a decision that was made a while ago, for reasons I'll explain (but will not debate). Feel free to express your opinions if you wish, but please do so with the understanding that this is not something we're looking to change in the near future.

There's three reasons for the DC, with the common thread being that without it we can't *detect* a problem. There could be no problem, a little problem, or a huge 100% problem, but without the checking we have no way of knowing. Here's the events that led up to the decision:

1) The PPS Sieve cheating went on for many months completely undetected. It was a fluke that we caught it. In the end we had to resieve months of work and we're not at all sure that there's not huge numbers of factors that were missed in other ranges that weren't resieved. We honestly don't even know the scope of the damage. Jim tabulated the number of extra factors found when we redid the sieve. It was around 7 or 10 million (I forget which.)

2) The PPS-Sieve ATI app was broken, and it was broken the beginning. It's been missing factors all the way back to 2010 or 2011, whenever it came online. Based on a limited number of data points, some 10% of all the factors that should have been found on ATI tasks were missed. There's no way of getting those back unless we resieve everything done in the last 4 years.

3) We discovered earlier this year that the Android sieve app, used for TRP-Sieve (and potentially for ESP-Sieve) was completely broken. It **ALWAYS** returned "no factors". 100% of the time. So every task done on Android missed 100% of the factors.

Those last two events have also led to an increased testing regimen for apps, as well as the increased focus on double checking.

In short, the real reason for double checking isn't so much to catch the occasional missed factor. It's to detect problems, such as cheating or bad software, or any other problem. Without the DC, we don't know when something has broken. The loss of efficiency is the cost necessary to improve quality by having an early warning system for problems.

Based on 2 and 3 above, I'm also considering whether we should adopt a policy of temporarily doing full DC on any project for a period of time (perhaps 1 to 3 months) following the installation of new versions of any app. At the present time, only PPS, PPSE, and SGS aren't already full DC, so this would only apply to those apps when a new version of LLR comes out. With the last LLR release for FMA, we didn't do this, but we did several months worth of testing of the LLR app.

We may reconsider the DC decision for the sieves at some future time if the conditions warrant, but for now it's a done deal.


What you are saying is that you did put a untested new app in production ??

And a second app. ATI app also untested ??


Do you think you could test the app's before releasing them ??


Lennart




5) Message boards : Sieving : DC on sieving using CPU is Only waste of TIME. (Message 78605)
Posted 1997 days ago by Profile Lennart SM5YMTProject donor
I thought the DC on the sieves was brought about due to a problem with some form of cheating. Am I not remembering that correctly? If I am remembering correctly, is that no longer a concern?


That's one of the problems. The other problem is that the lack of double checking allowed a serious flaw in the PPS sieve program to go unnoticed for many years. Since the ATI version of the PPS-Sieve program was introduced (in 2010?) it's been missing factors and we didn't discover this until recently. It's only once we started the partial double checking and made the validator really picky about matching the results that we noticed something was wrong.

The question boils down to which should we consider to be more important:

1) Getting the maximum amount of sieving done as quickly as possible, with the goal of therefore reducing the number of LLR candidates as quickly as possible, or

2) Making sure the sieve file is as accurate as we can, i.e., providing the maximum protection against both hardware and software errors, while at the same time providing the most protection against cheaters.



PPS sieve have nothing to do with CPU sieving.

If the goal is to get a accurate sievefile then I must have misunderstood the goal of finding primes.


Lennart
6) Message boards : Sieving : DC on sieving using CPU is Only waste of TIME. (Message 78602)
Posted 1997 days ago by Profile Lennart SM5YMTProject donor
I thought the DC on the sieves was brought about due to a problem with some form of cheating. Am I not remembering that correctly? If I am remembering correctly, is that no longer a concern?


I don't see that as a problem. If we have a expected time in validator
such cheating should not be a big problem.

Lennart
7) Message boards : Sieving : DC on sieving using CPU is Only waste of TIME. (Message 78600)
Posted 1997 days ago by Profile Lennart SM5YMTProject donor
Just a thought...does it make sense use AR?
Only small overhead but still high reliability.


Yes it make sense. I think that's a good idea if we only find out a good way to find as many errors we can on a easy way.

Lennart

8) Message boards : Sieving : DC on sieving using CPU is Only waste of TIME. (Message 78597)
Posted 1997 days ago by Profile Lennart SM5YMTProject donor
There are no reason to DC sieve WU !!!

We use to sieve to we get sec/factors about 100% of LLR time on a candidate about 75-80% of high n.level in the range.

Now with DC we get that sec/factors multiplied by 2 ( or more).

That means that there are no reason to sieve as deep as we should have done if we only did a single test.

If we miss 10% factors on a single tests we save time compare to DC.

Sieve is only to save time when running LLR !!!

If we not lose 50% or more without DC it is just a waste of time.

My goal is to minimize the time to find the primes. so I would like to stop all DC'ing on sieve wu.

Any thoughts ??

Lennart
9) Message boards : PrimeSearchTeam message board (Message 78513)
Posted 2001 days ago by Profile Lennart SM5YMTProject donor


SETI.Germany invites all BOINC teams to the SETI@Home Wow!-Event 2014. It takes place from Thu 15th August, 16.00 UTC, until Thu 29th August 2014, 16.00 UTC.
We cordially ask all team leaders to forward the invitation to their team. Information about the Wow! Race 2014, an exciting race concept and a registration form can be found here:
http://www.seti-germany.de/Wow/anmeldung.php?&lang=en
SETI.Germany wishes you and your team members much fun with the Wow! Event 2014.

Many Greatings: Terminator


--------------------


Lennart
10) Message boards : Project Staging Area : Megaprime search (Message 77138)
Posted 2054 days ago by Profile Lennart SM5YMTProject donor
http://primes.utm.edu/primes/page.php?id=118004
http://primes.utm.edu/primes/page.php?id=118003



Primes submitted


Next 10 posts
[Return to PrimeGrid main page]
DNS Powered by DNSEXIT.COM
Copyright © 2005 - 2020 Rytis Slatkevičius (contact) and PrimeGrid community. Server load 1.80, 2.30, 2.10
Generated 26 Jan 2020 | 8:08:16 UTC