|
Log files
Finally got some time to try something, after my previous poor attempt at
creating a log file :-)
So, I shouldn`t an array, but the file has got to be read somehow, so I`m
assuming I should use file_get_contents() ?
<?php
// Random test errors
$error = array(`Double Hit`, `Preset Post`, `No Session`, `Spam Detected`,
`Profanity Found`);
$fault = $error[array_rand($error)];
$logfile = `reject.log`;
$record = date(`d-m-Y H:i:s`)." [$fault]n";
error_log($record, 3, $logfile);
// Small file sizes just for testing
if (filesize($logfile) > 1000)
{
$logdata = file_get_contents($logfile);
$nline = strpos($logdata, "n", 500);
$newdata = substr($logdata, $nline);
$fp = fopen($logfile, `w`);
stream_set_write_buffer($fp, 0);
fwrite($fp, $newdata);
fclose($fp);
}
// Readfile just for test
echo `<pre>`;
readfile($logfile);
echo `</pre>`;
?>
Have used stream_set_write_buffer($fp, 0) for the first time.
Is it needed? I`ve tried flock() before and it seemed to screw things up.
Clicking the refresh button repeatedly shows it works, but is it any good, or
should I start again?
Thanks.
Comments
Hi,
You are still reading the whole file into memory.
Can you tell me again what version of php you are running? I will set up
your version of php in a WAMP here for testing.
File locking is only required if more then one script (or thread) can access
the log files.
I assume you are on linux, windows is different for file locking.
Hi,
Yes, linux and php 4.4 but please don`t install another version of php just for
me, but thanks anyway.
My site is a hobby site, but it seems to attract so much abuse.
I`ve had quite a few DOS attacks, but the shared server is DOS protected
(limited max per minute).
My previous log managed to record 3 attempts per second with no trouble.
This stops my visitors seeing the site though, but the max they can afford to
keep this up is usually about 2 hours, or their other spam doesn`t get
delivered.
It`s just a simple log file I`m trying to create, and it`s just to record the
failed attempts to spam me.
It`s not a full blow error log like the server logs provides.
I get one genuine message per month (and did get over a100 spam attempts per
day).
Since I send the "Double Hit" and "Preset Post" messages etc. to a non-existant
page, it`s all more or less stopped now.
This seems to really bugger up their spam list program.
I`m not sure whether eveyone knows what a Double Hit is, but it defeats cookie
or session protection.
No idea how they do it though?
In the past 2 days, I`ve had only 3 Double Hits, 2 Preset Posts, and 3 Profanity
errors.
Sending them to a non-existant site is causing them to get really annoyed and
*try* to swear at me.
I originally created a rolling road log (using a array <yes, bad idea>) to
display to the spammer his errors, with an arrow pointing to his latest attempt.
This drove them crazy and they tried to bring my site down.
If I put my nice CATCHA image back on, it would all stop, but I like the
challenge, he-he!
Re: You are still reading the whole file into memory.
If that`s what $logdata = file_get_contents($logfile); does, then yes.
If I don`t, how can I find half of it, then find the next line break to write to
the updated file?
Re: File locking is only required if more then one script (or thread) can access
the log files.
I didn`t know that, so it isn`t needed in my case.
Hi,
I had already installed php 4.4.4 onto Apache 2.0.63
I have a tested and working script below. I didn`t know if you wanted oldest
first or newest first. It is not hard to reverse it.
flock has issues on some servers that you need to get around by opening the
file in `w` or `w+` to get around. I used a different way of using a
separate lock file altogether.
--
<?php
// Random test errors
$error = array(`Double Hit`, `Preset Post`, `No Session`, `Spam Detected`,
`Profanity Found`);
$fault = $error[array_rand($error)];
$record = date(`d-m-Y H:i:s`)." [$fault]n";
$log_filename = `reject.log`;
$backup_filename = `reject.bak`;
$temp_filename = `reject.tmp`;
$lock_filename = `reject.loc`;
$max_filesize = 1000;
$error_reporting = TRUE;
if(!file_lock($lock_filename))
{
die();
}
if(!file_exists($log_filename))
{
if(file_exists($backup_filename))
{
rename($backup_filename, $log_filename);
}
else
{
$handle = file_open($log_filename, `w`);
fclose($handle);
}
}
if(!($log_file = file_open($log_filename, `r`)))
{
die();
}
if(!($temp_file = file_open($temp_filename, `w`)))
{
die();
}
if(filesize($log_filename) > $max_filesize)
{
fseek($log_file, -$max_filesize, SEEK_END);
$trash = fgets($log_file);
}
file_passthrough($log_file, $temp_file);
fwrite($temp_file, $record);
fclose($log_file);
fclose($temp_file);
@unlink($backup_filename);
rename($log_filename, $backup_filename);
rename($temp_filename, $log_filename);
function file_open($filename, $mode)
{
if(!($handle = @fopen($filename, $mode)))
{
throw_error("Unable to OPEN file [" . $filename . "]");
return FALSE;
}
else
{
return $handle;
}
}
function file_passthrough($source_handle, $destination_handle)
{
while(!feof($source_handle))
{
$buffer = fread($source_handle, 4096);
fwrite($destination_handle, $buffer);
}
}
function file_lock($lock_filename)
{
if(!($handle = file_open($lock_filename, `w`)))
{
return FALSE;
}
$max_lock_time = 10000; // 10 seconds
while($time < $max_lock_time)
{
if(flock($handle, LOCK_EX))
{
return TRUE;
}
$sleep = rand(1, 10000);
$time .= $sleep;
usleep($sleep);
}
throw_error("Unable to LOCK lock file [" . $lock_filename . "]");
return FALSE;
}
function throw_error($string)
{
global $error_reporting;
if(!$error_reporting)
{
return;
}
echo($string . "<br>n");
}
?>
<html>
<head>
<title>Log file test</title>
<meta http-equiv="refresh" content="1">
</head>
<body>
<?php
echo("Log file size = " . filesize($log_filename) . "<br>n");
$log_file = @fopen($log_filename, `r`);
while($one_line = fgets($log_file))
{
echo($one_line . "<br>n");
}
?>
</body>
</html>
Hi,
Wow, I wasn`t expecting you to go to so much trouble, and it`s much appreciated.
There`s certainly a lot for me to learn from this, from file handling to the way
you`ve organised it. I`m still looking at it in amazement!
It works a treat and the layout and use of functions is great, making it easier
to understand. Also, the http-equiv="refresh" is a neat trick for testing it.
I`m slowly realizing not to load a file or array with the data, and work
directly on the files. It`s finally sunk in.
It`s not normally the done thing to give someone the complete code, but it would
have taken a boat load of explaining on how to do it (well in my case, grin!).
This line threw me for I while:
$trash = fgets($log_file);
as $trash isn`t used anywhere.
But, then I realised this is how you were finding the record line ending, which
moves the file pointer to it. That`s a really cool way of doing it!
I can see now why I couldn`t get flock to work correctly, as I`d got it all
wrong.
I noticed you didn`t use flock($handle, LOCK_UN), so is this not needed in this
case?
I`m still trying to understand the use of a reject.loc ?
$sleep = rand(1, 10000);
$time .= $sleep;
usleep($sleep);
I haven`t work out why the random sleep yet?
I think you maybe doing something crafty that I`m missing?
Many thanks for this.
Hi,
I didn`t use flock($handle, LOCK_UN) as this automatically happens when the
script thread ends. However in some cases it may improve access for other
scripts.
I wrote the script so that other scripts can access the files as well if
they first check and lock the lock file.
I haven`t used flock before, normally I just create a lock file and check
for its existence before accessing other files then unlinking it when
finished. The problem with this is that if a script terminated half way
through and doesn`t unlink the lock file then there is a permanent lock. One
solution with smaller scripts that run very quickly is to watch the lock
file for a while longer then the scripts take to run and then access the
files as normal.
I read somewhere that flock has issues on some platforms and has to open the
file as write or append so I decided on a separate lock file. The log file
is opened in read mode.
Also fclose has to be done before renaming files and fclose also unlocks a
locked file.
The random sleep is from the php.net website. I expect that it is because
some scripts can end up synchronised so a random delay will break
synchronism.
Reading the script again I cab see two possible issues that you may want to
check -
1)
function file_lock($lock_filename)
{
if(!($handle = file_open($lock_filename, `w`)))
{
return FALSE;
}
Will opening an existing file in write mode fopen($filename, `w`) release
any previous locks on the file ??
2)
function file_passthrough($source_handle, $destination_handle)
{
while(!feof($source_handle))
{
$buffer = fread($source_handle, 4096);
fwrite($destination_handle, $buffer);
}
}
I should have explained that I am somewhat of a newbe to php!
In some other languages the above will write to a RAM buffer and the buffer
will only be written to the file system when the file stream is closed.
Does php have a way to flush a steam buffer to the file system ?
Here is the fix for the second issue -
function file_passthrough($source_handle, $destination_handle)
{
while(!feof($source_handle))
{
$buffer = fread($source_handle, 4096);
fwrite($destination_handle, $buffer);
fflush($destination_handle);
}
}
Hi,
I started reading about flock() out of interest.
I guess you`ve probably looked at the user notes:
http://uk.php.net/flock
It seems no-one can agree on anything to do with flock().
Would it be suitable to use a mysql database for an error log?
I`m assuming these problems wouldn`t occur with a DB.
Regards.
Would it be suitable to use a mysql database for an error log?
From the very beginning of this thread I thought that a MySQL database table
would be the best way to go. I figured that you were pursuing this because it
was not an option. If it is, use it by all means. Periodically you can delete
old records as long as you have some date-time field to know when an entry was
added.
would be the best way to go. I figured that you were pursuing this because it
was not an option. If it is, use it by all means. Periodically you can delete
old records as long as you have some date-time field to know when an entry was
added.
Hi,
Using a flat file was a temporary thing to start out with.
Then I just got interested in some of Rob`s techniques.
I haven`t done much with flat files previously, so it`s another string to my
bow, as they say.
Not that I have fired many arrows lately :-)
Regards.
|
|
