Beginning October 12th, one week from today, access to the statistics files exported by PrimeGrid will be restricted to authorised statistics sites and other authorised users.

We're still working on the final plan, but if you are currently using our statistics files, please contact me either on the forums, by private message, or at the email address listed on our contacts page.

Michael Goetz
____________
My lucky number is 75898⁵²⁴²⁸⁸+1

What URLs does this affect? Does this include the "subproject status" links and such at the bottom of the left panel of this page, or are these different files hidden elsewhere?
____________
8915 × 2 ^ 1507177 + 1 -- 453,710 digit PPSE
6603 × 2 ^ 1411654 + 1 -- 424,955 digit PPSE (DC)

What URLs does this affect? Does this include the "subproject status" links and such at the bottom of the left panel of this page, or are these different files hidden elsewhere?

I am occasionally downloading the statistics files (the nice XMLs) for my private stats engine.

Would like to be able to continue to download these

Why is this being done? Just curious.

Some of you may find this useful.

One person had been downloading the entire user_work.gz file just so he could read his own sub-project stats.

I've made that easier now. If you want an XML version of your subproject stats, use this URL:

Another thing that's been confusing people.

Only the big stats files in the /stats/ and /stats_work/ directories are affected.

If you're not downloading a massive gzip file, or your the URL you're using doesn't start with https://www.primegrid.com/stats/ or https://www.primegrid.com/stats_work/ then you're not affected by this change.

If you're running a stats site with a webserver for the whole world, you're probably using the big stats files.

If you're running something like gridcoin or charity engine, you're probably using the big stats files.

If you're getting information only for yourself, you're probably not using the big stats files.

If you're getting information for your team, you're probably not using the big stats files.
____________
My lucky number is 75898⁵²⁴²⁸⁸+1

Some of you may find this useful.

One person had been downloading the entire user_work.gz file just so he could read his own sub-project stats.

I've made that easier now. If you want an XML version of your subproject stats, use this URL:

https://www.primegrid.com/home.php?format=xml

I've removed the need to use your account key. This url will now return your subproject and badge information in XML format:

I've removed the need to use your account key. This url will now return your subproject and badge information in XML format:

https://www.primegrid.com/home.php?format=xml

(Earlier posts saying that an account key was needed have been corrected to avoid confusion.)

As of right now, only authorized users can access the stats and stats_work folders.
____________
My lucky number is 75898⁵²⁴²⁸⁸+1

I don't use them, but I'm curious.
How often are these xml files updated ?

I'm guessing some people might be having trouble with the passwords because there are symbols in there. If you're using something like wget, there's two things you need to be aware of:

1) Although it's not the most secure method you can put the username and password directly in the URL as such:

wget https://username:password@www.primegrid.com/stats/

You can also use the --http-user=user and --http-password=password parameters. This isn't any more secure than putting it in the URL, however. (Both would be visible via ps, for example.)

More secure would be to put it in .wgetrc or .netrc. You would use http-user=user and http-password=password.

2) Bash is probably going to mangle the symbols in the password, so replace them with their hex equivalent:

Assume your username is user and your password is !password!.

"!" is hex 21, so !password! becomes %21password%21

wget https://user:%21password%21@www.primegrid.com/stats/
____________
My lucky number is 75898⁵²⁴²⁸⁸+1

Thanks for that info!!
Helps me a lot! :D

Just a note for anyone using home.php?format=xml from a script:

I messed up the installation of the code on www, so it wasn't working before. It is now.

Also, in my instructions yesterday I said to change the "&" to "%26". That's incorrect. Change the "&" to "\&".


____________
My lucky number is 75898⁵²⁴²⁸⁸+1

Hi Michael,

Have you noticed a drop off in the Gridcoin team stats since the restricted access went in?

Gridcoin is no longer accepting PrimeGrid as a 'whitelisted' project

Dad

In early September I noticed that PrimeGrid was virtually unreachable for a few minutes. It turns out there was a DDOS attack against us, but it was a short lived attack. It only lasted a few minutes. Some investigation determined that this had been happening about once a day for a while, but since it was short in duration nobody really noticed.

By now you probably have guessed where this story is going.

Gridcoin needs our statistics, and collects them by having a large number of host computers each collect our stats files. At pretty much the exact same time. And these are really large files. While we have a fairly hefty server, and a decent pipe to the Internet, Gridcoin *is* a significant player at PrimeGrid, and all those hosts (over a thousand) downloading those large files at the same time was completely saturating our Internet connection.

PrimeGrid isn't the largest BOINC project, but we're up there, and our servers are more powerful than most. I'm not saying that to brag. The point is that if Gridcoin can effectively, albeit unintentionally, mount a successful DDOS attack against PrimeGrid's servers, imagine what Gridcoin is doing to all the other BOINC sites with much smaller hardware. Their only saving grace is that their stats files are almost certainly smaller than ours, and they may have fewer Gridcoiners (is that a word?) on their system. Nevertheless, this method of gathering the statistics is undoubtedly affecting many BOINC sites adversely. What is certain is that it has been making PrimeGrid's server unreachable for periods of time. That's unacceptable. Except for the fact that this is clearly unintentional, it would literally be criminal.

Back in September I reached out to Gridcoin. There's a fix in the pipeline. Things will hopefully be back to normal soon, minus the DDOS part.

We're not the bad guys here. Nor is Gridcoin. Everyone is working together to get this working.

Hi Michael,

Any news on how this is travelling with the Gridcoin team?

Thanx

No need, the Gridcoin team replied to me to say by end of next week they hope to have finished testing a new DLL