You are hereForums / Developer / Python statistics program

Videocache 1.9.5 is available now. This version improves support for logging and fixes video caching problems for various sites like Vimeo, Break, Tube8 etc due to change in URL patterns. Also, from this version onwards, UrlGrabber has been removed from dependencies. Check Changelog for details. If you have purchased version 1.9.x previously, you can claim free upgrade by contacting using Contact Link. Please mention the email address using which you purchased Videocache. New users proceed to download page right now, secure your copy and enjoy caching :-)

Python statistics program

Post Reply

12 replies [Last post]

22 November, 2008 - 09:43

chris

Offline

Joined: Nov 22 2008

Python statistics program

I've created a python program to generate some slightly advanced statistics from the cache logs. It's located here: http://zrwr.tuxfamily.org/programming/youtube-cache-stats.tar.gz

It lists video totals, sizes, download dates, times watched, and bandwidth saved. Unfortunately, it's only accurate based on the logs it can read.

Edit: Updated link

22 November, 2008 - 12:07

admin

Offline

Joined: Nov 2 2008

Thanks a ton for writing the cache analyzer man!! Did you test it? If so, please share some statistics and also can you write a "How to use stat analyzer?", so that people can get help with different options. Also, put a copyright notice in header after GPL comment like

23 November, 2008 - 07:39

chris

Offline

Joined: Nov 22 2008

You're welcome! I fixed a condition when a video was found the in logs but not in the cache, and I added options parsing. It seems to work fine on my system. It's now a tarball of a bazaar branch so changes can be tracked in the future.

By default, it works without any options. There are other options to control output but they might change over time so it's best to use: ./youtube-cache-stats.py --help to see all the options.

Since my old logs were being deleted in /var/log/squid (but it seems to not occur in /var/log/youtube_cache after upgrading to 1.5) and I'm the only one using the cache, I'll post up stats later when I watch more videos :)

23 November, 2008 - 11:42

admin

Offline

Joined: Nov 2 2008

Thanks man!! I am glad to see that you create a code repo for the same :) I'll try to include it in plugin in next version.

7 January, 2009 - 12:33

Alexandre Correa (not verified)

if you get this error:

Traceback (most recent call last):
File "./videocache-stats.py", line 364, in ?
run()
File "./videocache-stats.py", line 357, in run
analyze_logs(cache_record, log_files, options.verbose)
File "./videocache-stats.py", line 163, in analyze_logs
fi = fileinput.hook_compressed(log_path, "r")
AttributeError: 'module' object has no attribute 'hook_compressed'

your python version are < 2.5 !! to solve this.. apply patch attached

patch -p1 < videocache-stats.patch

######## file videocache-stats.patch #########

--- videocache-stats.py 2009-01-07 04:59:50.000000000 -0200
+++ videocache-stats_24.py 2009-01-07 04:59:38.000000000 -0200
@@ -160,7 +160,7 @@

# For each log file
for log_path in log_paths:
- fi = fileinput.hook_compressed(log_path, "r")
+ fi = fileinput.input(log_path, "r")

if verbose:
print "Reading", log_path

############ end ###########

hook_compressed featurea available for python >= 2.5 ... the difference between hook_compressed and input is that script can´t support gzipped/bzipped files... :)

9 January, 2009 - 23:31

chris

Offline

Joined: Nov 22 2008

Thanks for the feedback, but that patch just wipes out the videocache logs. The argument "r" gets passed into the "inplace" parameter. I hope you didn't lose any data! Unfortunately, I never backed up my /var directory but it's not a big deal.

I'll find a solution to reading multiple compressed files for python version < 2.5 within a few days.

9 June, 2009 - 09:15

Anonymous (not verified)

Link is not working..

27 June, 2009 - 19:07

admin

Offline

Joined: Nov 2 2008

Hi!

For me its working fine. You want to try it again.

Thank You!

29 June, 2009 - 07:59

mynullvoid (not verified)

Hi, I extract your script and localize to my enviroment; but why I get 0 MB of bandwidth was saved in the summary:

Report approximately from Wed Jun 24 17:54:38 2009

CACHE SUMMARY
20 videos cached
318 MB of videos
0 MB of bandwidth was saved

At the detail I got this for the bandwidth saving:

= BANDWIDTH SAVING =
NAME SIZE MB DOWNLOAD LAST WATCHED 3a1b469c26bcc769 19 none none 088374622da0d8f7 1 3.1 days ago 3.1 days ago 85c68d67888748cc 5 3.0 days ago 3.0 days ago 20234911f0a01e7e 9 3.9 days ago 3.9 days ago 8fc843a666f70096 9 3.0 days ago 3.0 days ago 3c696f7ced8302bf 10 3.0 days ago 3.0 days ago 75a9cc8b99b5a314 10 2.9 days ago 2.9 days ago 71b18d5472628008 10 3.1 days ago 3.1 days ago 857b1de5ac3ca1ab 11 3.0 days ago 3.0 days ago 8eab6c2162d2da11 13 3.0 days ago 3.0 days ago NAME SIZE MB DOWNLOAD LAST WATCHED 3a1b469c26bcc769 19 none none 088374622da0d8f7 1 3.1 days ago 3.1 days ago 85c68d67888748cc 5 3.0 days ago 3.0 days ago 20234911f0a01e7e 9 3.9 days ago 3.9 days ago 8fc843a666f70096 9 3.0 days ago 3.0 days ago 3c696f7ced8302bf 10 3.0 days ago 3.0 days ago 75a9cc8b99b5a314 10 2.9 days ago 2.9 days ago 71b18d5472628008 10 3.1 days ago 3.1 days ago 857b1de5ac3ca1ab 11 3.0 days ago 3.0 days ago 8eab6c2162d2da11 13 3.0 days ago 3.0 days ago WATCHED SAVED MB
0 0
1 0
1 0
1 0
1 0
1 0
1 0
1 0
1 0
1 0
WATCHED SAVED MB
0 0
1 0
1 0
1 0
1 0
1 0
1 0
1 0
1 0
1 0

Please assist

29 June, 2009 - 09:39

admin

Offline

Joined: Nov 2 2008

Mynullvoid,

No video was watched more than once, so no saving.

Thank You!

2 July, 2009 - 13:24

#10

mynullvoid

Offline

Joined: Jul 1 2009

I had been testing it by opening the same video over and over, and the result still shows 0 MB savings :(

2 July, 2009 - 17:38

#11

admin

Offline

Joined: Nov 2 2008

Mynullvoid,

Do you see entries with CACHE_HIT in your videocache.log file?

Thank You!

17 January, 2010 - 02:00

#12

Ryz (not verified)

Hi, where can I download the python script ?

regards

Ryz

Introduction

VideoCache Hacks

Python statistics program

Post new comment

Subcribe to Updates

Buy Now

New forum topics

Recent comments

Spread it

Who's online

Introduction

User login

VideoCache Hacks

Python statistics program

Post new comment

Subcribe to Updates

Buy Now

New forum topics

Recent comments

Spread it

Who's online