Support Queries

Python statistics program

I've created a python program to generate some slightly advanced statistics from the cache logs. It's located here: http://zrwr.tuxfamily.org/programming/youtube-cache-stats.tar.gz

It lists video totals, sizes, download dates, times watched, and bandwidth saved. Unfortunately, it's only accurate based on the logs it can read.

Edit: Updated link

13 Answers

Thanks a ton for writing the cache analyzer man!! Did you test it? If so, please share some statistics and also can you write a "How to use stat analyzer?", so that people can get help with different options. Also, put a copyright notice in header after GPL comment like

(C) Copyright 2008

You're welcome! I fixed a condition when a video was found the in logs but not in the cache, and I added options parsing. It seems to work fine on my system. It's now a tarball of a bazaar branch so changes can be tracked in the future.

By default, it works without any options. There are other options to control output but they might change over time so it's best to use: ./youtube-cache-stats.py --help to see all the options.

Since my old logs were being deleted in /var/log/squid (but it seems to not occur in /var/log/youtube_cache after upgrading to 1.5) and I'm the only one using the cache, I'll post up stats later when I watch more videos :)

Thanks man!! I am glad to see that you create a code repo for the same :) I'll try to include it in plugin in next version.

if you get this error:

Traceback (most recent call last):
  File "./videocache-stats.py", line 364, in ?
    run()
  File "./videocache-stats.py", line 357, in run
    analyze_logs(cache_record, log_files, options.verbose)
  File "./videocache-stats.py", line 163, in analyze_logs
    fi = fileinput.hook_compressed(log_path, "r")
AttributeError: 'module' object has no attribute 'hook_compressed'

your python version are < 2.5 !! to solve this.. apply patch attached

patch -p1 < videocache-stats.patch

######## file videocache-stats.patch #########

--- videocache-stats.py 2009-01-07 04:59:50.000000000 -0200
+++ videocache-stats_24.py      2009-01-07 04:59:38.000000000 -0200
@@ -160,7 +160,7 @@

        # For each log file
        for log_path in log_paths:
-               fi = fileinput.hook_compressed(log_path, "r")
+               fi = fileinput.input(log_path, "r")

                if verbose:
                        print "Reading", log_path

############ end ###########

hook_compressed featurea available for python >= 2.5 ... the difference between hook_compressed and input is that script can´t support gzipped/bzipped files... :)

Thanks for the feedback, but that patch just wipes out the videocache logs. The argument "r" gets passed into the "inplace" parameter. I hope you didn't lose any data! Unfortunately, I never backed up my /var directory but it's not a big deal.

I'll find a solution to reading multiple compressed files for python version < 2.5 within a few days.

Link is not working..

Hi!

For me its working fine. You want to try it again.

Thank You!

Hi, I extract your script and localize to my enviroment; but why I get 0 MB of bandwidth was saved in the summary:

Report approximately from Wed Jun 24 17:54:38 2009

CACHE SUMMARY
20 videos cached
318 MB of videos
0 MB of bandwidth was saved

At the detail I got this for the bandwidth saving:

= BANDWIDTH SAVING =
NAME                     SIZE MB DOWNLOAD      LAST WATCHED   WATCHED  SAVED MB
3a1b469c26bcc769              19 none          none                 0         0
088374622da0d8f7               1 3.1 days ago  3.1 days ago         1         0
85c68d67888748cc               5 3.0 days ago  3.0 days ago         1         0
20234911f0a01e7e               9 3.9 days ago  3.9 days ago         1         0
8fc843a666f70096               9 3.0 days ago  3.0 days ago         1         0
3c696f7ced8302bf              10 3.0 days ago  3.0 days ago         1         0
75a9cc8b99b5a314              10 2.9 days ago  2.9 days ago         1         0
71b18d5472628008              10 3.1 days ago  3.1 days ago         1         0
857b1de5ac3ca1ab              11 3.0 days ago  3.0 days ago         1         0
8eab6c2162d2da11              13 3.0 days ago  3.0 days ago         1         0
NAME                     SIZE MB DOWNLOAD      LAST WATCHED   WATCHED  SAVED MB
3a1b469c26bcc769              19 none          none                 0         0
088374622da0d8f7               1 3.1 days ago  3.1 days ago         1         0
85c68d67888748cc               5 3.0 days ago  3.0 days ago         1         0
20234911f0a01e7e               9 3.9 days ago  3.9 days ago         1         0
8fc843a666f70096               9 3.0 days ago  3.0 days ago         1         0
3c696f7ced8302bf              10 3.0 days ago  3.0 days ago         1         0
75a9cc8b99b5a314              10 2.9 days ago  2.9 days ago         1         0
71b18d5472628008              10 3.1 days ago  3.1 days ago         1         0
857b1de5ac3ca1ab              11 3.0 days ago  3.0 days ago         1         0
8eab6c2162d2da11              13 3.0 days ago  3.0 days ago         1         0

Please assist

Mynullvoid,

No video was watched more than once, so no saving.

Thank You!

I had been testing it by opening the same video over and over, and the result still shows 0 MB savings :(

Mynullvoid,

Do you see entries with CACHE_HIT in your videocache.log file?

Thank You!

Hi, where can I download the python script ?

regards

Ryz

help pless

Traceback (most recent call last):
  File "/usr/local/share/videocache/videocache.py", line 847, in <module>
    log(format%(os.getpid(), '-', '-', 'LOG_ROTATE_ERR', '-', 'Could not rotate logfiles.'))
TypeError: 'NoneType' object is not callable
Traceback (most recent call last):
  File "/usr/local/share/videocache/videocache.py", line 847, in <module>
    log(format%(os.getpid(), '-', '-', 'LOG_ROTATE_ERR', '-', 'Could not rotate logfiles.'))
TypeError: 'NoneType' object is not callable

this my but install cache video in squid version 2.7.STABLE7