OT (of sorts): stats masters... How do you do it?

[Fullmetal Chocobo]

For those that do the stats for any project, how do you do your stats? I would really like to learn the methods used for doing the stats. I thoroughly enjoyed doing the stats for "please delete", but I'm lost on how ya'll do the real stats.

 

[biodoc]

Yeah, I'd like some tips and tricks too from the experts.

GLeeM has already helped me.

I found a stats site with a "format" that I like which is updated close to the time I do the stats.

1) copy table & then "paste special then choose unicode text" into a spreadsheet.
2) reformat data to "taylor" it to the TeAm.
3) copy and paste sections into post box.
4) format to make it look acceptable (very tedious)

It takes me over an hour to do malaria.

Wish list for forum: "TAB" function; non-kerned font like "Courier New"!

 

[IJump]

I filled in doing stats at one point for some project.. I am too old to remember how I did it or even what project it was..

All the stats gurus are great guys.... Thanks to all of you that take the time to do it...

 

[GLeeM]

I'm no expert!

I do pretty much like biodoc:

sort page
select and copy what I want
paste into Notepad
get rid of spaces - just leave tabs (spaces screw up Excel importing data)
save
import external data into Excel (column separator is tabs)
cut name column and paste at end column
insert columns between data for "_____" underscores for spacing
copy and paste back into blank Notepad to delete or add underscores to get right length for correct spacing - this works better for me than adjusting them in thread post window

I "quote" yesterdays thread and delete stuff that I will replace with new
paste this into new thread post window
paste todays stuff into correct places

I have been meaning to make a program that will do all this and more
F@H has txt files updated every two or three hours for each team
the program would use a database to store stats over time to keep track of Milestones
hopefully I'll be able to figure out the needed code!

Takes about a half hour

 

[BlackMountainCow]

When I did the Rosetta stats, I used a special script that was written for me, which basically gathered the stats, formated them and gave me some options about milestones. I just had to fill in the new members, overtakes and general info and news about Rosetta. Pretty convenient I'd say.

 

[Bok7575]

If you could give me a standard format I could probably write a php script which generated the stats for you...

Bok

 

[Fullmetal Chocobo]

Are there any specific sites used to do the stats? Also, are they used for the majority of projects, or does each project have it's own best site for grabbing them from?

 

[The Borg]

I like checking out BOINC Stats, but for your use FMC, maybe something like boincsynergy may give you a better format to work with. They have stacks of ways of giving you the stats - country, team, indivicual, project, ...

Check out here and scroll to the bottom.

 

[BlackMountainCow]

Originally posted by: Bok7575
If you could give me a standard format I could probably write a php script which generated the stats for you...

Bok

Hey FMC, if the head behind the Free-DC stats offers you a customized script - what more can you ask for?! :beer::beer:

 

[Spacehead]

I just copy/paste the info from the stats page into a spreadsheet, then remove the columns i don't need & add in the proper spacing. I do all this inside the spreadsheet.

For the SoB stats i use Free-DC's site. For the weekly stats thread i do i use BOINCStats for the BOINC projects & Free-DC for all others. Not really sure why... i guess it's because Free-DC doesn't do SETI at the moment & just want to keep everything looking the same when i grab the info.

I keep all the URL's i use in WordPad & also keep the format for the headings in there too.


The daily SoB stats take maybe 10 minutes to do each day & the weekly thread(which i forgot to do again this past week) is taking less time each time i do it... maybe around 1/2 hour.

 

[Fullmetal Chocobo]

Originally posted by: BlackMountainCow

Originally posted by: Bok7575
If you could give me a standard format I could probably write a php script which generated the stats for you...

Bok

Hey FMC, if the head behind the Free-DC stats offers you a customized script - what more can you ask for?! :beer::beer:

I don't know what to ask him for though... I don't know enough about doing the stats. Would it be possible to agree on a standard format for all of the projects, making stats making a lot easier and less time consuming for everyone?

 

[Bok7575]

Originally posted by: Spacehead
I just copy/paste the info from the stats page into a spreadsheet, then remove the columns i don't need & add in the proper spacing. I do all this inside the spreadsheet.

For the SoB stats i use Free-DC's site. For the weekly stats thread i do i use BOINCStats for the BOINC projects & Free-DC for all others. Not really sure why... i guess it's because Free-DC doesn't do SETI at the moment & just want to keep everything looking the same when i grab the info.

I keep all the URL's i use in WordPad & also keep the format for the headings in there too.


The daily SoB stats take maybe 10 minutes to do each day & the weekly thread(which i forgot to do again this past week) is taking less time each time i do it... maybe around 1/2 hour.

They do Seti stats

These will be migrated to live (ie. > stats.free-dc.org) very soon. Still quite a few more features to add, but cross project Boinc stats are done, create date rankings, country rankings etc are all done...

 

[Spacehead]

Originally posted by: Fullmetal Chocobo
Would it be possible to agree on a standard format for all of the projects, making stats making a lot easier and less time consuming for everyone?

Probably hard to have everyone agree on a set format.


Bok, yeah i knew you were going to update your stats to include SETI.
When you're all done will the URL still be the same?
http://stats.free-dc.org/ or will everything start with http://stats3.free-dc.org/

 

[Bok7575]

It will be stats.free-dc.org..

 

[OhioDude]

If you have MS Excel and can write some VBA code, getting data from a web page and loading it into a workbook is insanely easy.

Here's a code snippet that connects to Bok's site and downloads the first page of the TA R@H stats into a workbook. Comments are prefaced with a single quote. (For some reason, I can't get the Attach Code function to work so my code indentations aren't showing up. Hence, this looks pretty messy... )

=======================
Public Sub GetRosettaStats()

Dim qtbl As Excel.QueryTable
Dim ws As Excel.Worksheet

' Set pointer to storage worksheet
Set ws = ThisWorkbook.Sheets(1)

' I always want to start with a fresh sheet so delete any existing QueryTables
For Each qtbl In ws.QueryTables
qtbl.Delete
Set qtbl = Nothing
Next qtbl

' Clear existing data in the worksheet
ws.Cells.ClearContents

' Set the URL for the stats site and the destination for the returned data
Set qtbl = ws.QueryTables.Add(Connection:="URL;http : //stats.free-dc.org/new/teamstats.php?proj=rah&team=TeAm+AnandTech", Destination:=ws.Range("A1"))

' Set some properties
With qtbl
' I want to select which table on Bok's page to load
.WebSelectionType = xlSpecifiedTables
' The data I want is in the 16th html table
.WebTables = "16"
' I want data only. No formatting
.WebFormatting = xlWebFormattingNone
End With

' Get the data!
qtbl.Refresh BackgroundQuery:=False

' Clean up pointers
Set qtbl = Nothing
Set ws = Nothing

End Sub
=====================

I used a similar technique when I did the Seti race stats for S@H Classic. I wrote code to download team stats from Berkeley, perform all of the calculations I needed including milestoners, format the output the way I wanted, create a text file with the formatted output and load it into notepad. I had the whole thing scheduled to run at 5:30 AM. When I got to work in the morning, notepad was open on my desktop with the race stats loaded. All I had to do was add a few embelishments like smilies and any other BS I felt like putting in, copy it to the clipboard and paste it into a forum message.

BTW - IJump: I was the dude you filled in for way back when. Plus, for a while, I was pumping a CSV file from my stats runs to you each day via email. Just goes to show you how quickly old statisticians are forgotten. Find a new stats pusher to satisfy your addictions and to hell with your old supplier. That's life on the DC streets.

 

[IJump]

I remember that. It was a long time ago....

 

[Smoke]

Originally posted by: OhioDude
... is insanely easy.

It also helped that you are one of the "smart ones" on this forum. :laugh:

 

[TallBill]

Originally posted by: biodoc
Yeah, I'd like some tips and tricks too from the experts.

GLeeM has already helped me.

I found a stats site with a "format" that I like which is updated close to the time I do the stats.

1) copy table & then "paste special then choose unicode text" into a spreadsheet.
2) reformat data to "taylor" it to the TeAm.
3) copy and paste sections into post box.
4) format to make it look acceptable (very tedious)

It takes me over an hour to do malaria.

Wish list for forum: "TAB" function; non-kerned font like "Courier New"!

An hour???? I did SoB stats in 4-5 minutes using excel.

 

[biodoc]

Originally posted by: TallBill

Originally posted by: biodoc
Yeah, I'd like some tips and tricks too from the experts.

GLeeM has already helped me.

I found a stats site with a "format" that I like which is updated close to the time I do the stats.

1) copy table & then "paste special then choose unicode text" into a spreadsheet.
2) reformat data to "taylor" it to the TeAm.
3) copy and paste sections into post box.
4) format to make it look acceptable (very tedious)

It takes me over an hour to do malaria.

Wish list for forum: "TAB" function; non-kerned font like "Courier New"!

An hour???? I did SoB stats in 4-5 minutes using excel.

I'm down to about 30 minutes now.

You're a better man than I am Tallbill...no way I could do it in 5 minutes.

 

[Spacehead]

Originally posted by: biodoc

Originally posted by: TallBill

Originally posted by: biodoc
Yeah, I'd like some tips and tricks too from the experts.

GLeeM has already helped me.

I found a stats site with a "format" that I like which is updated close to the time I do the stats.

1) copy table & then "paste special then choose unicode text" into a spreadsheet.
2) reformat data to "taylor" it to the TeAm.
3) copy and paste sections into post box.
4) format to make it look acceptable (very tedious)

It takes me over an hour to do malaria.

Wish list for forum: "TAB" function; non-kerned font like "Courier New"!

An hour???? I did SoB stats in 4-5 minutes using excel.

I'm down to about 30 minutes now.

You're a better man than I am Tallbill...no way I could do it in 5 minutes.

"4) format to make it look acceptable (very tedious)"
I suppose that's the most time consuming?
I also assume that's where you add the spacing and/or underscores? Much easier to do that all in the spreadsheet instead of the post box.

What's the URL for where you gather you data?

 

[biodoc]

Originally posted by: Spacehead

Originally posted by: biodoc

Originally posted by: TallBill

Originally posted by: biodoc
Yeah, I'd like some tips and tricks too from the experts.

GLeeM has already helped me.

I found a stats site with a "format" that I like which is updated close to the time I do the stats.

1) copy table & then "paste special then choose unicode text" into a spreadsheet.
2) reformat data to "taylor" it to the TeAm.
3) copy and paste sections into post box.
4) format to make it look acceptable (very tedious)

It takes me over an hour to do malaria.

Wish list for forum: "TAB" function; non-kerned font like "Courier New"!

An hour???? I did SoB stats in 4-5 minutes using excel.

I'm down to about 30 minutes now.

You're a better man than I am Tallbill...no way I could do it in 5 minutes.

"4) format to make it look acceptable (very tedious)"
I suppose that's the most time consuming?
I also assume that's where you add the spacing and/or underscores? Much easier to do that all in the spreadsheet instead of the post box.

What's the URL for where you gather you data?

http://statsnstones.tswb.org/Project.aspx?projid=6

Add the spacing in the spreadsheet? This would be faster?

I sort the data in the spreadsheet, copy paste into text editor, add the spacing and copy/paste into the post box.

Yeah, I guess I could do it faster but I'm sipping my morning coffee, chasing the cat off my keyboard and I do get distracted by some of the posts in the forum

Maybe tomorrow morning I'll set a stopwatch and ignore the distractions and then report back.

 

[Spacehead]

Originally posted by: biodoc

Originally posted by: Spacehead

Originally posted by: biodoc

Originally posted by: TallBill

Originally posted by: biodoc
Yeah, I'd like some tips and tricks too from the experts.

GLeeM has already helped me.

I found a stats site with a "format" that I like which is updated close to the time I do the stats.

1) copy table & then "paste special then choose unicode text" into a spreadsheet.
2) reformat data to "taylor" it to the TeAm.
3) copy and paste sections into post box.
4) format to make it look acceptable (very tedious)

It takes me over an hour to do malaria.

Wish list for forum: "TAB" function; non-kerned font like "Courier New"!

An hour???? I did SoB stats in 4-5 minutes using excel.

I'm down to about 30 minutes now.

You're a better man than I am Tallbill...no way I could do it in 5 minutes.

"4) format to make it look acceptable (very tedious)"
I suppose that's the most time consuming?
I also assume that's where you add the spacing and/or underscores? Much easier to do that all in the spreadsheet instead of the post box.

What's the URL for where you gather you data?

http://statsnstones.tswb.org/Project.aspx?projid=6

Add the spacing in the spreadsheet? This would be faster?

I sort the data in the spreadsheet, copy paste into text editor, add the spacing and copy/paste into the post box.

Yeah, I guess I could do it faster but I'm sipping my morning coffee, chasing the cat off my keyboard and I do get distracted by some of the posts in the forum

Maybe tomorrow morning I'll set a stopwatch and ignore the distractions and then report back.

Just a suggestion.... it's easier (i think) to use a site that you can use "sort by ..." columns. And yes, it's quicker to add any underscores, dots, whatever kind of spacing you want in the spreadsheet.

But in the end, it's whatever you want to do, however you want to do it. Whatever you feel most comfortably doing.

 

[biodoc]

I see, you put the spacers in a separate column in the spreadsheet and then reuse/adjust them for the next days stats. Yep, that would be faster.

Yeah, I'd prefer to use the FreeDC Stats site for malaria but the update timings aren't right when I do the stats early in the morning so I looked for another stats site to use that fits my schedule.

thanks for the advice spacehead!

 

[Spacehead]

Originally posted by: biodoc
I see, you put the spacers in a separate column in the spreadsheet and then reuse/adjust them for the next days stats. Yep, that would be faster.

Yeah, I'd prefer to use the FreeDC Stats site for malaria but the update timings aren't right when I do the stats early in the morning so I looked for another stats site to use that fits my schedule.

thanks for the advice spacehead!

Yep, spacers get their own column.

 

[Fullmetal Chocobo]

Originally posted by: biodoc

Originally posted by: TallBill

Originally posted by: biodoc
Yeah, I'd like some tips and tricks too from the experts.

GLeeM has already helped me.

I found a stats site with a "format" that I like which is updated close to the time I do the stats.

1) copy table & then "paste special then choose unicode text" into a spreadsheet.
2) reformat data to "taylor" it to the TeAm.
3) copy and paste sections into post box.
4) format to make it look acceptable (very tedious)

It takes me over an hour to do malaria.

Wish list for forum: "TAB" function; non-kerned font like "Courier New"!

An hour???? I did SoB stats in 4-5 minutes using excel.

I'm down to about 30 minutes now.

You're a better man than I am Tallbill...no way I could do it in 5 minutes.

That is just incredible. The only time I've done stats, I had to count the individual posts, increment my counter in Excel, then generate the reports from there. It took betwee 1-3 hours each time. But damn was it fun. :laugh:

I definitely need to learn more VBScript.