Technology
 

Talk:Stats/ua script

From Mozilla Community - Bringing Mozilla Together

Tip for everybody:

Hit preview. It's a pain to backtrack quite a few edits to ensure that everything's honkey dorey (meaning, good, not vandalism or editing code to make the server vulnerable, or accidentally deleting something someone else said, or whatever). 4 edits within such short time, heh, "Show Preview" is useful. --Tom talk/Bliki 13:59, 1 Apr 2005 (EST)

Table of contents


[edit] Anyone mind?

Does anybody mind if I do a little bit of reformatting of the page so that it is actually structured? Here's what I'm thinking:

  1. Proposed Scripts
    1. Kincera's Script UA script
    2. A-dead-trousers stat display script
    3. someone else's script
  2. Proposed Additions
    1. Ghedipunk's Image Script
    2. someone else's additions...
  3. Proposed Changes
    1. Ghedipunk's change $database variable
    2. codyrockx's change if to switch
    3. someone else's changes...
  4. Proposed Alternatives
    1. Ghedipunk's file access instead of database use
    2. someone else's alternatives...

If anyone has any objections or alternatives, speak up... I won't do anything unless at least someone says go ahead, and I deffinitely won't do anything if someone says no. I'd like to make the changes some time on Monday, though, if that's alright. Also, don't worry, I won't be changing any of the content. It will be all copy and paste. Ghedipunk 11:23, 2 Apr 2005 (EST)

Absolutely, feel free. At the moment things are looking a tad disorganised. Thats what wiki's are all about from what I can tell.--awrowe 11:53, 2 Apr 2005 (EST)

[edit] Nice code change

Nice change me at work, I still have no idea how you got the code looking like that, I'm going to have a look now.

I like the idea of a wiki, I just have no idea how to use it. --awrowe 12:40, 31 Mar 2005 (EST)

I threaded your thread so it looks nicer. Also, I wrapped it in pres and used nowiki just incase there was some code in there that might turn into links or whatever. --Tom talk/Bliki 15:14, 31 Mar 2005 (EST)

[edit] Original author

I think it is important for me to note that I didn't write all of the code. I had forgotten where I got it since I deleted the original copyright notice when I put it on my webserver since it woulnd't be required there. But I found a link to the original code with the original author's signature on it. Thank him:

http://www.apptools.com/phptools/browser/source.php

--Kincera 13:08, 31 Mar 2005 (EST)

Does he seem to be the kind of guy who would mind if we used it? I'll go have a look at his site and shoot him a line. --awrowe 13:18, 31 Mar 2005 (EST)
It is GPL'ed. So no worries :)--Kincera 13:19, 31 Mar 2005 (EST)
Yeah, just saw that. Even still, damn good spot there, its practically written to do what we want it to.
Have to get some reliable hosting with a database now, you think mozilla.org will float our boat for us? --awrowe 13:21, 31 Mar 2005 (EST)
I dunno. Have you sent them a request or proposal on it?--Kincera 13:34, 31 Mar 2005 (EST)
Actually in discussion about getting something set up right now. Would be excellent if it came off. --awrowe 13:44, 31 Mar 2005 (EST)
I will help in any way I can.--Kincera 14:02, 31 Mar 2005 (EST)
Your help will be gratefully received too.
To throw something of a spanner in the works here, Chris Messina mentioned that it would be a good idea to have a cross platform script to do something like this as well.
The original suggestion was to perhaps create a javascript and (I thought maybe) an asp script which will allow webmasters to choose their flavour of reporting. If you can, or know someone who can do this, lets try and get these happening as well.
Also, ghedipunk created an add on to the original script which allows webmasters to use whatever image they want to be the 'face' of the script. I think this is fantastic and we should integrate that.
I'll get back to this later in the morning, its ten to one am on April fools day here. --awrowe 18:00, 31 Mar 2005 (EST)
I've done ASP scripting before. I can probably get it working, but it will take me a little while. If someone out there has more experience, they can probably get it done faster.--Kincera 07:43, 1 Apr 2005 (EST)

[edit] Errors in this script

Right, I've been working on getting this script going on my home server.

I have managed to get rid of most of the problems, but I am still getting an error: 

Notice: Undefined index: aol in C:\www\webroot\includes\ua_script.php on line 197

I am just about to cut the line about aol completely out of the script. I'd like to get a reference there for the aol browser if possible though, so we might have to work on that.

I'll see how the edit goes.

I haven't finished working out the aol problem yet, because I noticed another error which has come up with using this script.
I am running mambo on both my development server and my main site and one of the global variables used by mambo is the variable $database.
In particular, this script breaks my pathway (breadcumbs) when it runs, because the pathway includes a reference and a select to $database.
I am trying hard to dredge up my php skills to turn the ua_script.php reference to $database into a local reference and prevent confusion with mambo's global reference, or whether we can rename this variable. It's kind of pathetic, but I can't remember how to do this.

--awrowe 06:34, 1 Apr 2005 (EST)

Right, I feel silly now, I just changed the variable $database to $db and my testing server miraculously worked again. This is a minor issue we will have to think about, there are probably heaps of different variable names being used in heaps of different sites, so we will need a unique name for things like this. awrowe

[edit] Script data

The script will only provide one entry for me, whether I access the testing server using IE or Firefox (the only browsers on my computer at the moment)

The results are here in comma seperated format: 

,2005-04-01,00000000000000,Firefox 1.0.2,Windows,127.0.0.1,

As you can see we are starting to get what we are looking for, however there are still some flaws here.

You are getting errors in the script I posted? It works fine on my webserver. Try echoing your useragent string to see what it is reporting. --Kincera 07:43, 1 Apr 2005 (EST)
Did you set up the database table correctly? Here is what I used to create my table (well, actually, it's what PHPMyAdmin used, but same difference. ;-) ) 
CREATE TABLE `my_table` (
`id` INT NOT NULL AUTO_INCREMENT ,
`date` DATE NOT NULL ,
`time` TIME NOT NULL ,
`browser` CHAR( 50 ) NOT NULL ,
`os` CHAR( 50 ) NOT NULL ,
`ip` CHAR( 20 ) NOT NULL ,
UNIQUE (
`id`
)
);
Here is what you would enter into a PHP script in order to create the same table... Note that you need to have already connected to your database. 
 $sql = 'CREATE TABLE `my_table` ( `id` INT NOT NULL AUTO_INCREMENT ,'
        . ' `date` DATE NOT NULL ,'
        . ' `time` TIME NOT NULL ,'
        . ' `browser` CHAR( 50 ) NOT NULL ,'
        . ' `os` CHAR( 50 ) NOT NULL ,'
        . ' `ip` CHAR( 20 ) NOT NULL ,'
        . ' UNIQUE ( `id` ) );'
        . ' ';
Ghedipunk 08:38, 1 Apr 2005 (EST)
Here is what I have for my stats table 
CREATE TABLE `stats` (
  `id` int(10) unsigned NOT NULL auto_increment,
  `date` date NOT NULL default '0000-00-00',
  `time` time NOT NULL default '00:00:00',
  `browser` varchar(50) NOT NULL default '',
  `os` varchar(50) NOT NULL default '',
  `ip` varchar(15) NOT NULL default '',
  PRIMARY KEY  (`id`)
) TYPE=MyISAM PACK_KEYS=0 COMMENT='Track Visitor Stats';
--Kincera 09:28, 1 Apr 2005 (EST)
Thanks for that. I think we have now established why I am not chasing a career as a databse technician or php programmer. *blush*
That was my fault, I created the table based on the sql in the script and didn't think about how to structure it properly. Its working properly now. Sorry bout that.
--awrowe 09:37, 1 Apr 2005 (EST)


The other thing is, I haven't managed to sort the problem with the aol error I was getting. I ended up commenting it out, but I'll uncomment it again and see what the problem is. 

Notice: Undefined index: aol in <br>C:\www\webroot\includes\ua_script.php on line 197

Thats whats coming up unless I comment out all the aol detection. --awrowe 09:48, 1 Apr 2005 (EST)

I don't think it is necessary to properly detect AOL. I think their latest browser version is a bastardized version of IE (despite them owning Netscape) so hopefully the useragent string will reflect that. --Kincera 09:53, 1 Apr 2005 (EST)
I agree, I think I'll uncomment any references to AOL in the script. Would be nice to have it there though, since showing the figures would also illustrate exactly how big AOL's marlket share is among the average user.--awrowe 10:02, 1 Apr 2005 (EST)
Being that they are publicly traded, I would expect that their subscriber numbers aren't state secrets. We probably don't need web stats to track that. --Kincera 10:06, 1 Apr 2005 (EST)
Thats true again. hmm.--awrowe 10:11, 1 Apr 2005 (EST)
I suspect the AOL tracking would come in handy for advertisers and such. Since we are focusing in on browsers being used and if there is some old and obscure AOL browser that doesn't report itself as IE, I suspect it will be statistically insignificant. Remember, it will be logged as an unknown. So long as unknown doesn't exceed a percentage point or two, it isn't that big of a deal IMO. --Kincera 10:14, 1 Apr 2005 (EST)
Fair enough. That works well then. Brilliant!

[edit] Image Script

Hey have you tried integrating ghedipunk's image script into this yet? I haven't yet, I've had to try and make some living money today, but I want to try to associate it with the logo on the page.

[edit] Server Hit and Proposals

I'm thinking about the server hit we might be creating if the script is associated with every page on a site, for example in the template. I'm not so much worried about the hit on the contributors server as I am about the hit on the server hosting the script and entering the database.

At the moment we are exploring two options, one of which was suggested by Chris Messina and one which is the way I would personally prefer to see it happen, which is the script we have been talking about. I'm sending an email to Chris now to ask if we could set up www.stats.spreadfirefox.com, with a database established for it as well. What we need now is an equivalent script in asp and one in javascript as well, so webmasters will have different flavours to choose from. I am thinking the options we should offer webmasters for participation in this project are as follows: A script to link to an image and a script which doesn't link to an image. These two variants of scripts would be offered in 3 flavours and if necessary we can ask me at work for buttons and so on to use. ghedipunk and a dead trousers have both been working on scripts for this as well, they may be able to help.

Comments?

--awrowe 10:27, 1 Apr 2005 (EST)

Perhaps we could keep the stats on the various web servers for a short period of time and dump the information to the main server on a regular basis. This way, the web servers could also display their own individual stats if they choose to, and the main server would only recieve large hits during non-peak times, such as at midninght on the first day of the month, or 2am on Thursdays. We could probably do this by using email. The biggest consideration with this approach is that we won't be able to configure the script to work with only one database type, or even assume that all of the webmasters will be using a database. This approach would deffinitely be a long term project.
Ghedipunk 18:00, 1 Apr 2005 (EST)
Perhaps the stats could be dumped after every 1,000 entries, instead of at a specific time. This would keep the main server from being swamped by constant HTTP requests (like what would happen if the web servers didn't hold their own stats), and keep the server from going down while it's processing the information from several sites dumping their database at once. Ghedipunk 19:59, 1 Apr 2005 (EST)


These are both good ideas. I'm not sure how to implement them though, is there a way we can come up with instructions for webmasters which will allow them to set it all off automatically? If not, then I think we will just have to accept that the server is going to have to take te data as it comes in.--awrowe 09:56, 2 Apr 2005 (EST)
Actually, PHP allows you to send mail via the script... We could just add a function that counts how many entries are in the database already, send an email to the central server if we have enough, then clear out the database and get it ready for the next thousand hits. Someone else would have to figure out how to parse the data from the email into the central server's database, though... I do know that it would work, though.


It seems to me that embedding a script to query the row count and the computational logic required to decide when to send another e-mail are a bit too complicated. It would seem that having set times to tabulate would make so much more sense. The query to pull the data would be very simple as it would be time based and not logic based.--Kincera 14:47, 2 Apr 2005 (EST)

[edit] Use of script as a php include?

I may have completely the wrong idea about how this is working and if so, please tell me.

The way I am seeing this working is to have the script hosted on a remote server along with the database.

The webmaster of the website who is contributing the data uses an include like 
<?php include_once ('http://www.example.com/ua_script.php'); ?>
which is then placed on the index or major entry pages of the website.

The website calls the script from our site and the script logs the user_agent of the browser which accessed the page. Does that make sense?

Working with images and so on is great and will certainly spread the word of what we are doing, but is this how we are focusing our ideas? I need to make sure we are all aiming at the same target.--awrowe 12:03, 3 Apr 2005 (EDT)

[edit] Data under GPL?

GPL is for programs, things like 'data' or 'documentation' go better under a CC license or GNU Free Documentation License (GFDL). I suppose the php code to create the data would be GPL though. Hrm... --Tom talk/Bliki 14:04, 1 Apr 2005 (EST)

The way I would envision presenting the data is to make the final processed data available to anyone and everyone who wants to see it. I'm not sure about the various licences available, but I think probably the GNU Free Documentation Licence is the one we want, I'll go and have a look at it.
I don't think we can open the raw data up to the general public without creating some confidentiality issues with the websites participating. Because of the way we are considering working with it, it would be simple for competing sites to extrapolate visitor statistics and possible turnover and so on from the raw data. I think a lot of sites would see that as releasing confidential company data to the public and would be dead against it.
As I proposed earlier, we could certainly release raw data from sites who don't have a problem with the public seeing it, but the consent would have to be express and they would have to place the link themselves. I think that would probably save us from any legal ramifications.
I'll get to work on a information sheet for participating webmasters which will outline all these details. I'll put it on this wiki for review, while we try to organise the database and hosting. I have to get on to Chris about that. --awrowe 05:12, 2 Apr 2005 (EST)
Retrieved from "http://mozilla.wikia.com/wiki/Talk:Stats/ua_script"