PrimeGrid
Please visit donation page to help the project cover running costs for this month

Toggle Menu

Join PrimeGrid

Returning Participants

Community

Leader Boards

Results

Other

drummers-lowrise
1) Message boards : Primegen : Compressed (Message 56204)
Posted 3273 days ago by suicid
new pc(s)... more power...and not that much time to use them...
40 bit should be (almost) done or even done... 100% tested, not only 99,999999...% by Miller-Rabin...

but I guess I will stop here because it's more than I need...
less than Diethelms DB, more than primegrid DB... :D
2) Message boards : Primegen : What is the purpose of primegen? (Message 56201)
Posted 3273 days ago by suicid
The lists could be used for stuff like quad. sieve(s)...
you can never have enough numbers to do that... ;)

instead of using (2 and) all odd numbers you could use that list... less division to do...
instead of saving e.g. 15 = 3*5 = {3,5} you could save 15 = {index(3), index(5)} = {1,2} <-- no space between "neighbours"...

with a list based on 32 bit entries:
instead of ~8.000.000.000 (only the odd) numbers
you could save more than 100.000.000.000 IDs...


with a new pc the lists can be created @ home quite fast...
got ~48.000.000.000 primes by using 1 or 2 i7 PCs for ~3 weekends (~200 hours)...
(Linux, C++, 64 bit, programs almost @ optimum, CUDA was not used (yet))
~45 GB --> with extra index-files it should fit on 1 DL-BR (50 GB)
zipped version would be ~30 GB... could be saved on a large SD - as backup...

would be overkill for QS@home (~36 bit to save 1 entry)
--> I guess primegen results would be quite useless today...
3) Message boards : Primegen : Compressed (Message 11013)
Posted 4643 days ago by suicid
Stopped crunshing a long time ago (pc crashed, no time to fix it, then needed new hardware, .... no pc to run the program...only laptop...)

I wrote a program to get the numbers myself...
2^35 is already done
2^36 is about to be finished (less than a week)
2^37 will need some time (~ 1 month)
...

size
now: 2.2 GB ~ 2.2 GP
@2^37: >=5 GB, but less than 6 GB

[2^37 ~ 137.438.953.472 ~ DVD 1-4]
[x GP = x GigaPrimes = x * 1024^3 primes
too lazy to write large numbers and 1GB ~ 1GP ;) ]

will publish the results "when it's done"...
index-generator and viewer needed to use the files...
need to write 2 programs and the pc should finish the first 4.5 GB before I publish the files... and I want to run a test on another pc...

the programm will stop between 2^37 and 2^39
reason: time needed to generate the files
up to: time for new area | sum complete area | size of file
2^36: ~14 days | left: 0 ("done")|<2.7 GB
2^37: >30 days | left: >30 days | <6 GB
2^38: >60 days | left: >90 days | <11 GB
2^39: >120 days | left: >210 days | <20GB
2^40: >240 days | left: >450 days | <40 GB

4) Message boards : Primegen : Compressed (Message 8522)
Posted 4844 days ago by suicid
Still busy... gave the address to the programs to Rytis...
the reader is not finished yet cause the memory of my normal pc is damaged...
--> compress should work - could not verify this 'cause some of the strings (from the original files) looked like this "23456$2424" or "%&/$""§SDF24"...

will try it on the laptop again... with 1-10 MB files instead of 1 GB and only 0.1 disk instead of 6 DVDs...
the procedures for using the files will be the same... I hope so... ;)
5) Message boards : Primegen : Compressed (Message 8517)
Posted 4847 days ago by suicid
OK.... got the program... it should do the job...
I could not verify that it is perfect becuse my memory has 21 errors (found in one hour and everything in the 512 MB - 2 GB Area)
§$%&//$&//§/&%&&/%&)"§&"§ BEEEEEEEP $%&/"(§"/(§"§

--> I will not give you the program
--> you will have to compile the sourcecode yourself (it's written in Delphi 7)


Algorithm:
Is based on storing the difference of prime on position x and position x-1
Pos(0) = 1 (needed a start value even if it is not prime; in PGR (PrimeGrid Reader) it will be Pos(0) = 2)
Pos(1) = 3
Pos(2) = 5
.....
Pos(x) = prime on position x

Do for all primes (x = 1 to max):
current = Pos (x) - Pos (x - 1);
current = (current shr 1) - 1;
if (current < 254)
write (resultfile, byte(current));
else if (current < 253 + (2 shl 16)) then begin
write (resultfile, byte(254));
write (resultfile, word(current - 254));
end
else
write (resultfile, byte(255));
write (resultfile, longword (current - 254 - (2 shl 16)));
//triplebyte = last 3 byte of current
//longword is the easiest way to handle this...
//triplebyte --> max. difference ~ 33M
//longword --> max. difference ~ 8000M
end;


One buffer for input and one for output should be used...



My program:
Will send you the address via PM...
PGR (the reader) is not ready yet... got a little design...

correct order of the input needed
(3,5,7,11,13,..... is ok
5,3,13,7,11 is not ok)

Disks will look like this:
4 resultfiles (each about 1 GB)
1 index
some special files (filesize, ids,....)

e.g.:
Disk#1 content:
PG00.dat,PG01.dat,PG02.dat,PG03.dat,index.dat,....
Disk#2 content:
PG04.dat,PG05.dat,PG06.dat,PG07.dat,index.dat,....

if you want me to rename the files then just give me the name...




PGR:
PGR will use bisection on the index to find the highest value / id (=position)
that is stored in the loaded index... (index = 8 byte position + 8 byte value
(index will contain 72 MB or 144 MB --> there is an option to load only one half / third / ...
by loading every / every second / every third / ......)
Runtime: Only O(log (size(index / 16))) to find the position to start...

1. chose the right file by firstvalue/lastvalue or firstid/lastid #1
2. bisection on index - fast search --> position#2
3. from position#2 to the right primenumber (by value or by id)
....

(C) and idea by me... ;)

have to go... have to upload the data and send you the link...
6) Message boards : Primegen : Compressed (Message 8427)
Posted 4850 days ago by suicid
If you got a windows pc (guess you have) then I can give you the Delphi-program and the sourcecode... (I will make some changes...(*))
if not I have to rewrite compress in C(++)
should be better than sending you the files with max. 100kb/s via azureus (1d when I use the complete line) and you will need the program to compress the other disks...

640G --> ~23.542.625.051 primes ( 640G / ln(640G) )
index every 50.000th number: 7,184631348 MB index file
every 25.000th number: 15MB
every 10.000th number: 36MB

size of resdata will be needed on every disk... (to access the primes
each "Res*.dat"-file 4 byte

last number of each file --> 8 byte / file

primes:
min: 22 GB
max: 88 GB (disk 1-6 = 8 GB --> max should be smaller than 64 GB)
I guess maximum will be not more than 30 GB... (~7 DVDs)

(*) Autodeletion (with deactivate-option) for finished (original) files...
and some kind of resume... should be some GB to be compressed...
--> possibility to compress disk by disk or file by file or everything at once...
(you will see... ;) )
7) Message boards : Problems and Help : DVD6 (Message 8420)
Posted 4851 days ago by suicid
I need about 300 MB to finish the download...
but I am not able to get the missing data...

--> please help me, so I can compress the disk, too, and finish the second DVD... (I need it to check my compress system... if it is not valid then I have to redesign some lines... THEN I can publish the files and write a tool to access the data...)
8) Message boards : Problems and Help : a compact file format for prime lists (Message 8419)
Posted 4851 days ago by suicid
The plaintext format has been chosen just because it's plaintext - you can open it with a text editor and immediately see the numbers.

About your format, we are currently processing (although not exporting yet) the numbers that are above 2^31. If we go for some other format than plaintext, it must be able to support huge numbers (millions of digits).


ROTFLBTC ;)

I tried to open the uncompressed files with notepad & wordpad & OO & ... (Windows 2k, XP,... with >=2GB RAM) and gedit, vi, hexedit,........ (Linux, 512 MB RAM)
--> pc started working for some minutes... then the program / the whole system crashed...
a 500 MB textfile is a little bit tooooooooooooo biiiiiiiiiiig to read it with "notepad"...

so I wrote a "5 minute (not finished, but it works)" - Delphi-program to get the X-th line out of file Y...
9) Message boards : Primegen : Compressed (Message 8418)
Posted 4851 days ago by suicid
I compressed DVD 1,2,3,4 and 5 (download of DVD6 is not completed yet)
Disks 1-6 should be smaller than 8GB (only 2 DVDs)
--> Am I allowed to publish the files? If not I can publish only the 36 Bit version because that I got before I found primegrid... <--

The algorithm is not that complex... can be used without decompressing the data with a special tool/algorithm (like I had to do with the original and 80 - 100 GB are not funny if the files would fit on 2 DVDs and decompress took some hours, too)

I splitted (some files, each around 1 GB) and indexed (every 50.000th prime is in the index) the files... by this way it's possible to search for the x-th prime or to check if y is prime pretty fast...

MfG SuiCid
PS: I will have to check the algorithm again by decompressing the file and comparing the data with the original... (maybe I have to set some constants)
[Return to PrimeGrid main page]
DNS Powered by DNSEXIT.COM
Copyright © 2005 - 2021 Rytis Slatkevičius (contact) and PrimeGrid community. Server load 2.12, 2.18, 2.27
Generated 24 Jun 2021 | 20:10:00 UTC