You are an unregistered user, you can register here
Navigation

Information

Site

Donations
If you wish to make a donation you can by clicking the image below.


 
Go Back   The Unreal Admins Page > Forums > Hosted Forums > UTStats > General Chat

Reply
Thread Tools Display Modes
  #1  
Unread 3rd March, 2005, 02:30 PM
Rush's Avatar
Rush Rush is offline
Holy Shit!!
 
Join Date: Apr 2003
Location: Texas
Posts: 1,157
Default Automating UTStats Import

Here are a few ways to make it automated @ GNU/Linux:

1) UTServer and UTStats @ the same BOX, using Cron

Let's prepare the import.php first, open it and comment the 6th line, it should look now like:
Code:
// Get key from web browser
//$adminkey = $_GET[key];
Then uncomment the 13th line:
Code:
$adminkey = $argv[1];
* don't forget about setting the $akey in the 3rd line to somewhat like "mypassword"
*php command may require a full path to the binary for example /usr/local/php5/bin/php

You should try running the script manually first to find out if everything is ok.
Code:
cd /your/utstats/main/path
php import.php "mypassword"
Now you should see some crappy html output and import procedure if there were any logs.

Ok, we're on the good way to make it working. In your /home directory create a script "utstatsimport"
Code:
#!/bin/bash
for file in `ls /your/path/ut-server/Logs/Unreal.ngLog*.log 2> /dev/null`; do
mv $file /your/utstats/main/path/logs > /dev/null
done
cd /your/utstats/main/path
php import.php "mypassword" > /dev/null
*CHANGE THE PATHS TO UTSERVER AND UTSTATS!! don't forget about the password !
.. and make it executable
Code:
chmod +x utstatsimport
*php command may require a full path to the binary for example /usr/local/php5/bin/php

Better check this script on few logs now to avoid searching the mistakes in the future.

Now setting the cron. Create a file called "mycrontab" with this content
Code:
SHELL=/bin/bash
HOME=/home/me

# run-parts
* * * * * ./utstatsimport
*change the home to your own !!!

now the last one is to run a little command in your home:
Code:
crontab ./mycrontab
The script should be runned now about every one minute.
---------
2) UTServer and UTStats @ the same BOX, without Cron(stupid admin)

Use the point's 1 hints to the moment you have to create the utstatsimport script, the script itself has to be different:

Code:
#!/bin/bash
while [ 1 ];do
for file in `ls /your/path/ut-server/Logs/Unreal.ngLog*.log 2> /dev/null`; do
mv $file /your/utstats/main/path/logs > /dev/null
done
cd /your/utstats/main/path
php import.php "mypassword" > /dev/null
sleep 1m
done
*CHANGE THE PATHS TO UTSERVER AND UTSTATS!! don't forget about the password !
*php command may require a full path to the binary for example /usr/local/php5/bin/php

Make it executable
Code:
chmod +x ./utstatsimport
and just run it by
Code:
./utstatsimport &
To stop it use the command
Code:
killall -9 utstatsimport
---------
3) UTServer and UTStats on different boxes .. whatever.

I won't describe how to use the ftp stuff in utstats php part here cause I just don't use it, haha. I'll just tell you how to trigger the import.php script.

3a) With cron ....
Create a script in your home, called "utstatsimport" with content:
Code:
#!/bin/bash
lynx -dump "http://utstats.mydomain.com/import.php?key='mypassword'"  > /dev/null
*Change the mypassword to your own !!!
Make it executable with
Code:
chmod +x ./utstatsimport
Now create a file called "mycrontab" also in your home with this content:
Code:
SHELL=/bin/bash
HOME=/home/me

# run-parts
* * * * * ./utstatsimport
*change the home to your own !!!

now the last thing you should do is to run a little command in your home:
Code:
crontab ./mycrontab
3b)If the admin is lame and you cannot use cron ...

Create a script in your home, called "utstatsimport" with content:
Code:
#!/bin/bash
while [ 1 ];do
lynx -dump "http://utstats.mydomain.com/import.php?key='mypassword'"  > /dev/null
sleep 1m
done
*Change the mypassword to your own !!!
Make it executable with
Code:
chmod +x ./utstatsimport
and just run it by
Code:
./utstatsimport &
. To stop it use the command
Code:
killall -9 utstatsimport
__________________
[email address]

Last edited by Rush : 3rd March, 2005 at 02:40 PM.
Reply With Quote
  #2  
Unread 3rd March, 2005, 07:34 AM
Azazel's Avatar
Azazel Azazel is offline
Administrator
 
Join Date: Jul 2002
Location: UK
Posts: 2,408
Default

Nice work Rush as always
__________________

Phoenix Alliance - The Movie
Revisionistic Movie
The Unreal Admins Page - the resource for unreal admins
Reply With Quote
  #3  
Unread 3rd March, 2005, 07:44 AM
Rush's Avatar
Rush Rush is offline
Holy Shit!!
 
Join Date: Apr 2003
Location: Texas
Posts: 1,157
Default

I just hope it is clear enough and do not contains any bugs.
__________________
[email address]
Reply With Quote
  #4  
Unread 3rd March, 2005, 12:30 PM
toa toa is offline
Unstoppable
 
Join Date: Mar 2005
Posts: 179
Default

A little suggestion for those people who don't have access to cron jobs (and/or lame admins ): WebCron.
You just have to enter "http://utstats.mydomain.com/import.php?key=mypassword" in the URL field. Don't forget to click 'enable' after you created your task.
Reply With Quote
  #5  
Unread 3rd March, 2005, 12:46 PM
PizzaMan's Avatar
PizzaMan PizzaMan is offline
Holy Shit!!
 
Join Date: Jun 2004
Location: Bergen - Norway
Posts: 2,389
Default

Quote:
Originally Posted by toa
A little suggestion for those people who don't have access to cron jobs (and/or lame admins ): WebCron.
You just have to enter "http://utstats.mydomain.com/import.php?key=mypassword" in the URL field. Don't forget to click 'enable' after you created your task.
Nice one
Reply With Quote
  #6  
Unread 9th March, 2005, 01:15 AM
THE_WHO THE_WHO is offline
Forum Newcomer
 
Join Date: Dec 2004
Posts: 5
Default

or use this http://www.cronjob.de
Reply With Quote
  #7  
Unread 25th April, 2005, 02:15 PM
Matthew Matthew is offline
Holy Shit!!
 
Join Date: Dec 2004
Posts: 2,382
Default

hmm I'm having trouble with import.

I had to run webcron. I have my account, and all is well webcron wise. It is enabled, and runs every hour. Problem is, when I go to see RECENT MATCHES, it doesn't appear that they have been processed. I check webcron, and it said all was well. I opened the cron's log. The stats FTPed fine, and then were deleted, as they are supposed to be. Well now for some reason those new games are not getting truely imported. Any suggestions?

I have my cron task pointed at:
http://www.atomicunreal.com/stats/import.php?key=*******
The task is ENBALED and runs fine, no errors

EDIT:

I will mention that:
UTStats, when I manually import, changes the current page automatically about 4 times as it imports the tables. Could webcron just not be allowing the tables to be imported?

Also just after a manual import, a small amount of files were FTPed, but many many matches were just imported and were already up there... so I believe they are uploading correctly.

Last edited by Matthew : 25th April, 2005 at 03:53 PM.
Reply With Quote
  #8  
Unread 25th April, 2005, 05:36 PM
Matthew Matthew is offline
Holy Shit!!
 
Join Date: Dec 2004
Posts: 2,382
Default

Hey I got it... not sure if it is the "right" way to do it but it works so...
In IMPORT.PHP I have changed
Code:
$logdir = opendir('logs');

while (false !== ($filename = readdir($logdir)))
{
//Our (self set) timelimit exceeded => reload the page to prevent srcipt abort
	if (!empty($import_reload_after) and $start_time + $import_reload_after <= time()) {
		if (!$html) die('Time limit exceeded - unable to reload page (no HTML output)' ."\n");

		$elapsed = $elapsed + time() - $start_time;
		$target = $PHP_SELF ."?key=". urlencode($adminkey) ."&amp;".str_rand()."=".str_rand()."&amp;no_ftp=1&amp;debug=$debug&amp;files=$files&amp;elapsed=$elapsed";
		echo '<meta http-equiv="refresh" content="2;URL='. $target .'">';

		echo'<br /><table border="0" cellpadding="1" cellspacing="2" width="720">
		  <tr>
			<td class="heading" align="center" colspan="2">Maximum execution time exeeded; restarting ...</td>
		  </tr>
		  </table>';

		include("includes/footer.php");
		return;
	}

	$oldfilename = $filename;
	$filename = 'logs/' . $filename;
	$backupfilename = 'logs/backup/' . $oldfilename;
To:

Code:
$logdir = opendir('logs');

while (false !== ($filename = readdir($logdir)))
{
//////////////////////////////////////////////////////////////////
// ----------------------- EDIT FOR WEBCRON-----------------------
//////////////////////////////////////////////////////////////////
//Our (self set) timelimit exceeded => reload the page to prevent srcipt abort
//	if (!empty($import_reload_after) and $start_time + $import_reload_after <= time()) {
//		if (!$html) die('Time limit exceeded - unable to reload page (no HTML output)' ."\n");
//
//		$elapsed = $elapsed + time() - $start_time;
//		$target = $PHP_SELF ."?key=". urlencode($adminkey) ."&amp;".str_rand()."=".str_rand()."&amp;no_ftp=1&amp;debug=$debug&amp;files=$files&amp;elapsed=$elapsed";
//		echo '<meta http-equiv="refresh" content="2;URL='. $target .'">';
//
//		echo'<br /><table border="0" cellpadding="1" cellspacing="2" width="720">
//		  <tr>
//			<td class="heading" align="center" colspan="2">Maximum execution time exeeded; restarting ...</td>
//		  </tr>
//		  </table>';
//
//		include("includes/footer.php");
//		return;
//	}
//
//////////////////////////////////////////////////////////////////
	$oldfilename = $filename;
	$filename = 'logs/' . $filename;
	$backupfilename = 'logs/backup/' . $oldfilename;
Reply With Quote
  #9  
Unread 25th April, 2005, 05:41 PM
toa toa is offline
Unstoppable
 
Join Date: Mar 2005
Posts: 179
Default

Quote:
Originally Posted by Matthew
I had to run webcron. I have my account, and all is well webcron wise. It is enabled, and runs every hour. Problem is, when I go to see RECENT MATCHES, it doesn't appear that they have been processed.
Tecnically there is no difference between a connection from your browser and the webcron thinghie. Both request a webpage.
The importer will stop after $import_reload_after seconds (config.php). It asks your browser to reload the same page again until it processed all the files. Webcron does not obey these requests, so your import script won't run long enough to do all the work. You may increase this value depending on your php's max_execution_time (check phpinfo()) or even set it to 0 if safe mode is disabled.
Reply With Quote
  #10  
Unread 19th December, 2005, 08:08 PM
khaytsus khaytsus is offline
Forum Newcomer
 
Join Date: Dec 2005
Posts: 5
Default

What about just putting the script in the LocalBatcherURL and making sure bBatchLocal is on? Works for my server so far, and no cron script.

I rolled my own script to handle the moving of logs and running import, nothing new there though. Pretty crappy bash kludge really.
Reply With Quote
  #11  
Unread 7th July, 2006, 03:52 AM
2399Skillz 2399Skillz is offline
Super Moderator
 
Join Date: Jan 2004
Location: North Carolina
Posts: 2,245
Default

Think we can get an update to this?

Lines 6 and 13 are totally different than what you have up there.
__________________

UT-FILES.COM
Get your files
Reply With Quote
  #12  
Unread 7th July, 2006, 04:03 AM
2399Skillz 2399Skillz is offline
Super Moderator
 
Join Date: Jan 2004
Location: North Carolina
Posts: 2,245
Default

Apparently they aren't needed anyway.

I didn't have to do most of the stuff in that tutorial for the new version to get the stats automated, though your cronjob scripts did do the work just fine.

Editing the import.php file as you mention didn't require anything.

Well I take that back, I did comment out the 21st line.
if (!isset($adminkey)) $adminkey = 'password;

Took me a while to figure it out, I thought it was the method I was working, when come to find out the web server user didn't have access to the Logs directory on the server, once I set those permissions it was able to move the files, and add them to the database just fine.
__________________

UT-FILES.COM
Get your files
Reply With Quote
  #13  
Unread 7th July, 2006, 01:32 PM
2399Skillz 2399Skillz is offline
Super Moderator
 
Join Date: Jan 2004
Location: North Carolina
Posts: 2,245
Default

I take that back, something isn't working.

The cronjob didn't appear to work correctly, so I tested the file the cronjob called for and it gives me two errors. So I ran the manual script (stupid admins) and it did something but I have no idea what.
__________________

UT-FILES.COM
Get your files
Reply With Quote
  #14  
Unread 7th July, 2006, 01:53 PM
Matthew Matthew is offline
Holy Shit!!
 
Join Date: Dec 2004
Posts: 2,382
Default

Why not use crowbar's mutator and save yourself a lot of physical pain?
Reply With Quote
  #15  
Unread 7th July, 2006, 02:15 PM
2399Skillz 2399Skillz is offline
Super Moderator
 
Join Date: Jan 2004
Location: North Carolina
Posts: 2,245
Default

Because I have access to cron jobs on my server and I'd rather do it that way, than adding another serveractor to the game server.
__________________

UT-FILES.COM
Get your files
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT +1. The time now is 04:20 PM.


 

All pages are copyright The Unreal Admins Page.
You may not copy any pages without our express permission.