PrimeGrid
Please visit donation page to help the project cover running costs for this month

Toggle Menu

Join PrimeGrid

Returning Participants

Community

Leader Boards

Results

Other

drummers-lowrise

Advanced search

Message boards : Developer blog : 2008 03 22 - Database Issues

Author Message
JohnProject donor
Honorary cruncher
Avatar
Send message
Joined: 21 Feb 06
Posts: 2875
ID: 2449
Credit: 2,681,934
RAC: 0
321 LLR Bronze: Earned 10,000 credits (11,773)Cullen LLR Bronze: Earned 10,000 credits (14,945)ESP LLR Bronze: Earned 10,000 credits (26,855)PPS LLR Bronze: Earned 10,000 credits (84,876)PSP LLR Bronze: Earned 10,000 credits (15,311)SoB LLR Bronze: Earned 10,000 credits (21,440)SR5 LLR Bronze: Earned 10,000 credits (29,270)SGS LLR Bronze: Earned 10,000 credits (26,616)TPS LLR (retired) Bronze: Earned 10,000 credits (36,288)TRP LLR Bronze: Earned 10,000 credits (41,655)Woodall LLR Bronze: Earned 10,000 credits (15,807)321 Sieve Bronze: Earned 10,000 credits (20,014)Cullen/Woodall Sieve (suspended) Bronze: Earned 10,000 credits (23,405)PPS Sieve Bronze: Earned 10,000 credits (36,192)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (20,306)TRP Sieve (suspended) Bronze: Earned 10,000 credits (21,738)GFN Bronze: Earned 10,000 credits (86,217)PSA Ruby: Earned 2,000,000 credits (2,143,756)
Message 8537 - Posted: 22 Mar 2008 | 18:10:24 UTC

Since 20 March 2008 ~14:00 UTC, PrimeGrid's database has been overwhelmed. This is a result of the amazing participation in the Ides of March Challenge. While the server was able to handle the one day onslaught, the aftermath has proven too much for the database. It has been up and down for the past two days. Conditions now are starting to improve as more and more users reduce their load on the server. Work is being sent and results are being accepted but at a very limited level.

Rytis will be away until at least 26 March so a complete repair will not be available until then. In the meantime, please consider switching your project selections to any of the following: Cullen LLR, Woodall LLR, 321 LLR, PSP LLR. This will further help reduce the load as these projects have 10-30 hour WU completion times.

Also, you may mind find interest in our "Staging Area". This non-BOINC process utilizes LLRNet on different servers. There's plenty of LLR work there with much faster WU completion times. Additionally, you'll be contributing a great deal to the preparation of PrimeGrid's future projects. This will also give you a "sneak peek" into our upcoming Sophie Germain and 32+1 Prime Search projects.

Lastly, we do not wish for your computers to become idle. If you only wish to do BOINC work, please consider attaching to an additional project while we resolve the database issues.

We apologize for this inconvenience and appreciate your participation and patience.

Thank you!
John


____________

JohnProject donor
Honorary cruncher
Avatar
Send message
Joined: 21 Feb 06
Posts: 2875
ID: 2449
Credit: 2,681,934
RAC: 0
321 LLR Bronze: Earned 10,000 credits (11,773)Cullen LLR Bronze: Earned 10,000 credits (14,945)ESP LLR Bronze: Earned 10,000 credits (26,855)PPS LLR Bronze: Earned 10,000 credits (84,876)PSP LLR Bronze: Earned 10,000 credits (15,311)SoB LLR Bronze: Earned 10,000 credits (21,440)SR5 LLR Bronze: Earned 10,000 credits (29,270)SGS LLR Bronze: Earned 10,000 credits (26,616)TPS LLR (retired) Bronze: Earned 10,000 credits (36,288)TRP LLR Bronze: Earned 10,000 credits (41,655)Woodall LLR Bronze: Earned 10,000 credits (15,807)321 Sieve Bronze: Earned 10,000 credits (20,014)Cullen/Woodall Sieve (suspended) Bronze: Earned 10,000 credits (23,405)PPS Sieve Bronze: Earned 10,000 credits (36,192)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (20,306)TRP Sieve (suspended) Bronze: Earned 10,000 credits (21,738)GFN Bronze: Earned 10,000 credits (86,217)PSA Ruby: Earned 2,000,000 credits (2,143,756)
Message 8541 - Posted: 24 Mar 2008 | 18:30:11 UTC

Database Status

The database is still limping along. Work is going out and results are being returned, but it continues at a very limited level.

We are awaiting the return of Rytis which will be after 26 March. Hopefully the database and server will be functioning normally shortly afterwards. More details will be provided then.

Thank you again for your participation and patience. We apologize for this inconvenience. Please see previous post for alternatives while we resolve this issue.

John

____________

Rytis
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 22 Jun 05
Posts: 2648
ID: 1
Credit: 21,592,315
RAC: 88
321 LLR Silver: Earned 100,000 credits (104,475)Cullen LLR Silver: Earned 100,000 credits (291,372)ESP LLR Bronze: Earned 10,000 credits (18,156)Generalized Cullen/Woodall LLR Bronze: Earned 10,000 credits (15,259)PPS LLR Silver: Earned 100,000 credits (113,978)PSP LLR Silver: Earned 100,000 credits (116,517)SoB LLR Silver: Earned 100,000 credits (282,655)SR5 LLR Bronze: Earned 10,000 credits (14,071)SGS LLR Silver: Earned 100,000 credits (100,082)TPS LLR (retired) Silver: Earned 100,000 credits (111,607)TRP LLR Jade: Earned 10,000,000 credits (16,076,365)Woodall LLR Silver: Earned 100,000 credits (101,463)321 Sieve Silver: Earned 100,000 credits (201,501)Cullen/Woodall Sieve (suspended) Silver: Earned 100,000 credits (214,653)Generalized Cullen/Woodall Sieve (suspended) Bronze: Earned 10,000 credits (14,200)PPS Sieve Silver: Earned 100,000 credits (363,095)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Silver: Earned 100,000 credits (200,232)TRP Sieve (suspended) Ruby: Earned 2,000,000 credits (2,453,872)AP 26/27 Silver: Earned 100,000 credits (473,058)GFN Silver: Earned 100,000 credits (184,089)PSA Bronze: Earned 10,000 credits (97,541)
Message 8542 - Posted: 25 Mar 2008 | 19:47:32 UTC

I just got home and kickstarted things. No promises that everything will keep running though - I can't monitor everything, I'm very tired and must get some sleep now.

Note: validation is heavily throttled down now, but at least you should be able to get work and report it back.
____________

Profile VatoProject donor
Volunteer tester
Avatar
Send message
Joined: 2 Feb 08
Posts: 750
ID: 18447
Credit: 197,414,529
RAC: 224,288
Found 1 prime in the 2020 Tour de Primes321 LLR Ruby: Earned 2,000,000 credits (2,419,655)Cullen LLR Ruby: Earned 2,000,000 credits (2,309,957)ESP LLR Ruby: Earned 2,000,000 credits (2,617,477)Generalized Cullen/Woodall LLR Ruby: Earned 2,000,000 credits (2,076,479)PPS LLR Turquoise: Earned 5,000,000 credits (7,130,181)PSP LLR Ruby: Earned 2,000,000 credits (3,564,856)SoB LLR Ruby: Earned 2,000,000 credits (3,254,352)SR5 LLR Ruby: Earned 2,000,000 credits (2,982,194)SGS LLR Ruby: Earned 2,000,000 credits (2,885,788)TPS LLR (retired) Silver: Earned 100,000 credits (103,523)TRP LLR Ruby: Earned 2,000,000 credits (3,450,469)Woodall LLR Ruby: Earned 2,000,000 credits (2,147,072)321 Sieve Sapphire: Earned 20,000,000 credits (20,593,192)Cullen/Woodall Sieve (suspended) Ruby: Earned 2,000,000 credits (4,119,699)Generalized Cullen/Woodall Sieve (suspended) Jade: Earned 10,000,000 credits (10,278,995)PPS Sieve Sapphire: Earned 20,000,000 credits (32,663,237)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Ruby: Earned 2,000,000 credits (4,080,177)TRP Sieve (suspended) Turquoise: Earned 5,000,000 credits (5,221,054)AP 26/27 Sapphire: Earned 20,000,000 credits (20,020,744)GFN Sapphire: Earned 20,000,000 credits (31,284,162)PSA Sapphire: Earned 20,000,000 credits (34,211,039)
Message 8545 - Posted: 26 Mar 2008 | 1:32:24 UTC

Being able to do work is the most important thing IMHO.

Thanks for this, and I hope you get a good nights sleep!

[C@B] Terminal
Send message
Joined: 22 Feb 07
Posts: 88
ID: 5940
Credit: 219,397
RAC: 0
PPS LLR Bronze: Earned 10,000 credits (23,447)TPS LLR (retired) Bronze: Earned 10,000 credits (18,644)321 Sieve Bronze: Earned 10,000 credits (13,155)PPS Sieve Bronze: Earned 10,000 credits (53,936)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Silver: Earned 100,000 credits (104,066)
Message 8546 - Posted: 26 Mar 2008 | 8:12:41 UTC - in response to Message 8542.

I just got home and kickstarted things. No promises that everything will keep running though - I can't monitor everything, I'm very tired and must get some sleep now.

Note: validation is heavily throttled down now, but at least you should be able to get work and report it back.


Could you explain what happened and what did you do to have this improvement on the 25th? Thx. It's just to understand a little bit more.

Profile S@NL - FilmFreakProject donor
Avatar
Send message
Joined: 29 Nov 05
Posts: 58
ID: 1587
Credit: 3,290,496
RAC: 0
321 LLR Silver: Earned 100,000 credits (235,773)PPS LLR Gold: Earned 500,000 credits (512,812)SR5 LLR Gold: Earned 500,000 credits (508,431)SGS LLR Bronze: Earned 10,000 credits (71,624)TRP LLR Bronze: Earned 10,000 credits (82,218)Woodall LLR Bronze: Earned 10,000 credits (10,077)PPS Sieve Amethyst: Earned 1,000,000 credits (1,755,489)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (90,725)
Message 8547 - Posted: 26 Mar 2008 | 12:45:57 UTC - in response to Message 8546.

I'd also like to know if this was an incident, or if it's likely to happen again with the next challenge.
____________
Life is short and meaningless, unless you make the best of it.
http://server.by014.net/~weezepoe

Rytis
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 22 Jun 05
Posts: 2648
ID: 1
Credit: 21,592,315
RAC: 88
321 LLR Silver: Earned 100,000 credits (104,475)Cullen LLR Silver: Earned 100,000 credits (291,372)ESP LLR Bronze: Earned 10,000 credits (18,156)Generalized Cullen/Woodall LLR Bronze: Earned 10,000 credits (15,259)PPS LLR Silver: Earned 100,000 credits (113,978)PSP LLR Silver: Earned 100,000 credits (116,517)SoB LLR Silver: Earned 100,000 credits (282,655)SR5 LLR Bronze: Earned 10,000 credits (14,071)SGS LLR Silver: Earned 100,000 credits (100,082)TPS LLR (retired) Silver: Earned 100,000 credits (111,607)TRP LLR Jade: Earned 10,000,000 credits (16,076,365)Woodall LLR Silver: Earned 100,000 credits (101,463)321 Sieve Silver: Earned 100,000 credits (201,501)Cullen/Woodall Sieve (suspended) Silver: Earned 100,000 credits (214,653)Generalized Cullen/Woodall Sieve (suspended) Bronze: Earned 10,000 credits (14,200)PPS Sieve Silver: Earned 100,000 credits (363,095)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Silver: Earned 100,000 credits (200,232)TRP Sieve (suspended) Ruby: Earned 2,000,000 credits (2,453,872)AP 26/27 Silver: Earned 100,000 credits (473,058)GFN Silver: Earned 100,000 credits (184,089)PSA Bronze: Earned 10,000 credits (97,541)
Message 8549 - Posted: 26 Mar 2008 | 18:45:47 UTC - in response to Message 8546.

Could you explain what happened and what did you do to have this improvement on the 25th? Thx. It's just to understand a little bit more.

PrimeGrid validator/assimilator/dbpurge are actually scripts and not standard BOINC server daemons. They are started every 10 minutes by a cron job.

Something caused heavy load on the server on the 20th, and validator did not finish running in the designated 10 minute period. Because there was no protection from running multiple copies at once, a new copy was started, slowing down the first copy and itself. Because of that it did not finish in the 10 minute period again, and one more copy was launched. The things started cascading.

At the same time, database size exceeded available memory, which caused MySQL caching not be efficient enough, causing a lot of disk accesses, slowing everything even further.

Right now I've modified the validator from starting multiple copies, will do the same for other scripts in the near future.
____________

Profile Starless
Avatar
Send message
Joined: 21 May 07
Posts: 5
ID: 8582
Credit: 11,804,879
RAC: 0
PPS LLR Bronze: Earned 10,000 credits (58,729)TPS LLR (retired) Bronze: Earned 10,000 credits (48,197)PPS Sieve Jade: Earned 10,000,000 credits (10,568,085)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Amethyst: Earned 1,000,000 credits (1,110,169)AP 26/27 Bronze: Earned 10,000 credits (11,473)
Message 8554 - Posted: 27 Mar 2008 | 0:05:30 UTC

What i thought would be a bad idea, the Ides of March, met my expectations of bad ideas completely. All i can say for now :)
____________
what is so hard to understand about 3.1415926535897932384626433832795 ??

Profile Emma LaurijssensProject donor
Avatar
Send message
Joined: 14 May 07
Posts: 42
ID: 8350
Credit: 1,540,757
RAC: 120
321 LLR Bronze: Earned 10,000 credits (63,098)Cullen LLR Silver: Earned 100,000 credits (123,449)ESP LLR Bronze: Earned 10,000 credits (27,335)PPS LLR Bronze: Earned 10,000 credits (33,401)PSP LLR Silver: Earned 100,000 credits (104,432)SoB LLR Bronze: Earned 10,000 credits (43,614)TPS LLR (retired) Bronze: Earned 10,000 credits (17,277)TRP LLR Bronze: Earned 10,000 credits (10,353)Woodall LLR Silver: Earned 100,000 credits (290,543)Cullen/Woodall Sieve (suspended) Bronze: Earned 10,000 credits (79,086)Generalized Cullen/Woodall Sieve (suspended) Bronze: Earned 10,000 credits (37,681)PPS Sieve Silver: Earned 100,000 credits (462,086)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (78,205)AP 26/27 Silver: Earned 100,000 credits (141,505)
Message 8556 - Posted: 27 Mar 2008 | 7:29:01 UTC - in response to Message 8554.

What i thought would be a bad idea, the Ides of March, met my expectations of bad ideas completely. All i can say for now :)


In that respect, you might call it a good idea as well, because it gave a clue about the robustness of the server. This was only a common, minor issue. The next challenge will probably reveal some more issues but all in all, I'm impressed.

____________
?SYNTAX ERROR
READY.

Profile VatoProject donor
Volunteer tester
Avatar
Send message
Joined: 2 Feb 08
Posts: 750
ID: 18447
Credit: 197,414,529
RAC: 224,288
Found 1 prime in the 2020 Tour de Primes321 LLR Ruby: Earned 2,000,000 credits (2,419,655)Cullen LLR Ruby: Earned 2,000,000 credits (2,309,957)ESP LLR Ruby: Earned 2,000,000 credits (2,617,477)Generalized Cullen/Woodall LLR Ruby: Earned 2,000,000 credits (2,076,479)PPS LLR Turquoise: Earned 5,000,000 credits (7,130,181)PSP LLR Ruby: Earned 2,000,000 credits (3,564,856)SoB LLR Ruby: Earned 2,000,000 credits (3,254,352)SR5 LLR Ruby: Earned 2,000,000 credits (2,982,194)SGS LLR Ruby: Earned 2,000,000 credits (2,885,788)TPS LLR (retired) Silver: Earned 100,000 credits (103,523)TRP LLR Ruby: Earned 2,000,000 credits (3,450,469)Woodall LLR Ruby: Earned 2,000,000 credits (2,147,072)321 Sieve Sapphire: Earned 20,000,000 credits (20,593,192)Cullen/Woodall Sieve (suspended) Ruby: Earned 2,000,000 credits (4,119,699)Generalized Cullen/Woodall Sieve (suspended) Jade: Earned 10,000,000 credits (10,278,995)PPS Sieve Sapphire: Earned 20,000,000 credits (32,663,237)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Ruby: Earned 2,000,000 credits (4,080,177)TRP Sieve (suspended) Turquoise: Earned 5,000,000 credits (5,221,054)AP 26/27 Sapphire: Earned 20,000,000 credits (20,020,744)GFN Sapphire: Earned 20,000,000 credits (31,284,162)PSA Sapphire: Earned 20,000,000 credits (34,211,039)
Message 8557 - Posted: 27 Mar 2008 | 10:22:58 UTC

I think running challenges and stressing the server is fine.
It might be a good idea to time the next one so that Rytis is available for the aftermath... :-)

Profile Lonely
Avatar
Send message
Joined: 27 Feb 08
Posts: 2
ID: 19560
Credit: 358,396
RAC: 0
Woodall LLR Bronze: Earned 10,000 credits (10,744)PPS Sieve Silver: Earned 100,000 credits (225,857)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (45,933)GFN Bronze: Earned 10,000 credits (56,950)
Message 8559 - Posted: 27 Mar 2008 | 14:10:49 UTC

:shock: Hope he has broad shoulders and keeps taking his daily dose of porridge. If not, we will have to clone him many times over.:lol:
____________

JohnProject donor
Honorary cruncher
Avatar
Send message
Joined: 21 Feb 06
Posts: 2875
ID: 2449
Credit: 2,681,934
RAC: 0
321 LLR Bronze: Earned 10,000 credits (11,773)Cullen LLR Bronze: Earned 10,000 credits (14,945)ESP LLR Bronze: Earned 10,000 credits (26,855)PPS LLR Bronze: Earned 10,000 credits (84,876)PSP LLR Bronze: Earned 10,000 credits (15,311)SoB LLR Bronze: Earned 10,000 credits (21,440)SR5 LLR Bronze: Earned 10,000 credits (29,270)SGS LLR Bronze: Earned 10,000 credits (26,616)TPS LLR (retired) Bronze: Earned 10,000 credits (36,288)TRP LLR Bronze: Earned 10,000 credits (41,655)Woodall LLR Bronze: Earned 10,000 credits (15,807)321 Sieve Bronze: Earned 10,000 credits (20,014)Cullen/Woodall Sieve (suspended) Bronze: Earned 10,000 credits (23,405)PPS Sieve Bronze: Earned 10,000 credits (36,192)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (20,306)TRP Sieve (suspended) Bronze: Earned 10,000 credits (21,738)GFN Bronze: Earned 10,000 credits (86,217)PSA Ruby: Earned 2,000,000 credits (2,143,756)
Message 8560 - Posted: 27 Mar 2008 | 14:20:54 UTC
Last modified: 27 Mar 2008 | 14:21:50 UTC

Thanks for all the feedback. Points well taken. As for the timing…true, in hindsight, it could/should have been better. However, please note that the database event happened four days after the Challenge completed. Additionally, the participation in the challenge far exceeded our expectations so the amount of work completed was unexpected. Lessons learned. :)

We plan to continue with a monthly challenge...hopefully, improving as we go. As each Challenge will be different, we expect to learn lessons each time.

If people thought the Ides of March Challenge was a bad idea, then they'll thoroughly enjoy our next one which has a "Titanic" theme. :D

More details to come next week.
____________

Profile John M. Johnson "Novex"Project donor
Volunteer tester
Avatar
Send message
Joined: 16 Aug 07
Posts: 625
ID: 10876
Credit: 1,066,951
RAC: 0
321 LLR Bronze: Earned 10,000 credits (50,802)Cullen LLR Silver: Earned 100,000 credits (111,106)PPS LLR Silver: Earned 100,000 credits (101,350)PSP LLR Bronze: Earned 10,000 credits (15,210)SGS LLR Bronze: Earned 10,000 credits (27,055)TPS LLR (retired) Bronze: Earned 10,000 credits (67,826)Woodall LLR Bronze: Earned 10,000 credits (10,609)321 Sieve Bronze: Earned 10,000 credits (70,873)Cullen/Woodall Sieve (suspended) Bronze: Earned 10,000 credits (20,185)PPS Sieve Bronze: Earned 10,000 credits (20,034)Sierpinski (ESP/PSP/SoB) Sieve (suspended) Bronze: Earned 10,000 credits (24,551)TRP Sieve (suspended) Bronze: Earned 10,000 credits (40,040)AP 26/27 Silver: Earned 100,000 credits (159,148)PSA Silver: Earned 100,000 credits (344,961)
Message 8565 - Posted: 28 Mar 2008 | 9:43:04 UTC

I just want to thank you John and Rytis for all the hard work you put into the "Ides of March Challenge" and everyone who helped with churning out work units! I look forward to the next Challenge :)

John M. Johnson "Novex"

Message boards : Developer blog : 2008 03 22 - Database Issues

[Return to PrimeGrid main page]
DNS Powered by DNSEXIT.COM
Copyright © 2005 - 2020 Rytis Slatkevičius (contact) and PrimeGrid community. Server load 1.49, 1.57, 1.57
Generated 14 Aug 2020 | 5:17:19 UTC