VideoCache

Centos 5.4 Squid 2.6 Video Cach does not work :/

by star3am on 19 Mar 2010

Hi, I have heard great things about your application, but have yet to see it in action.

We purchased version 1.9.2 here is my full bug report.

1) the RPM is useless for Centos :/ as the versions needed is not available in any repo,

ERROR with rpm_check_debug vs depsolve:
rpmlib(FileDigests) is needed by videocache-1.9.2-1.noarch
Complete!
(1, [u'Please report this error in http://yum.baseurl.org/report'])

I get the same error with rpm -ivh

I then read your forum and you suggest to install the videocache-1.9.2.tar.gz

I now run python setup.py install which reports sucess, and I can tail your log, but the problem is that the Videos are never fetched, rather I see in

# ls -lah /var/spool/videocache/tmp/
total 8.0K
drwxr-xr-x  2 squid squid 4.0K Mar 19 11:25 .
drwxr-xr-x 17 squid squid 4.0K Mar 18 08:45 ..
-rw-------  1 squid squid    0 Mar 19 11:25 4316732.13460751.11.flv

The size are always 0 ... no matter what I do,

I check the dependencies,

# rpm -qa | grep python
python-elementtree-1.2.6-5
libxml2-python-2.6.26-2.1.2.8
libselinux-python-1.33.4-5.5.el5
python-urlgrabber-3.1.0-5.el5
gamin-python-0.1.7-8.el5
dbus-python-0.70-9.el5_4
python-iniparse-0.2.3-4.el5
audit-libs-python-1.7.13-2.el5
rpm-python-4.4.2.3-18.el5
python-sqlite-1.1.7-1.2.1
python-2.4.3-27.el5

All there, so what can be the problem, reading your forum, I gather that I should be able to access http://PROXY_IP_ADDRESS/videocache/

which i can, working perfectly, my problem is with download part,

here is a part of your videocache log, the error I always see is,
Requested video was not found in cache.

## snip ##
2010-03-19 11:16:20,950 11630 192.168.12.198 - REQUEST - http://v13.lscache2.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dVTlBMUl9FSkNNNV9IRlZF&fexp=902906%2C901801&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1269014400&key=yt1&signature=57C93FECEB4E3A2F9A0E96D8CD7675CD5AED8EC4.5BA3AAAC111A9F7C5AE4334D7D4F67196B9D5B62&factor=1.25&id=e6fd2f97a893942e
2010-03-19 11:16:20,950 11630 192.168.12.198 e6fd2f97a893942e URL_HIT YOUTUBE http://v13.lscache2.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dVTlBMUl9FSkNNNV9IRlZF&fexp=902906%2C901801&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1269014400&key=yt1&signature=57C93FECEB4E3A2F9A0E96D8CD7675CD5AED8EC4.5BA3AAAC111A9F7C5AE4334D7D4F67196B9D5B62&factor=1.25&id=e6fd2f97a893942e
2010-03-19 11:16:20,950 11630 192.168.12.198 e6fd2f97a893942e CACHE_MISS YOUTUBE Requested video was not found in cache.
2010-03-19 11:16:20,953 11630 192.168.12.198 e6fd2f97a893942e NEW_URL YOUTUBE http://v13.lscache2.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dVTlBMUl9FSkNNNV9IRlZF&fexp=902906%2C901801&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1269014400&key=yt1&signature=57C93FECEB4E3A2F9A0E96D8CD7675CD5AED8EC4.5BA3AAAC111A9F7C5AE4334D7D4F67196B9D5B62&factor=1.25&id=e6fd2f97a893942e
2010-03-19 11:16:22,136 11630 192.168.12.198 - REQUEST - http://v13.nonxt2.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dVTlBMUl9FSkNNNV9IRlZF&fexp=902906%2C901801&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1269014400&key=yt1&signature=57C93FECEB4E3A2F9A0E96D8CD7675CD5AED8EC4.5BA3AAAC111A9F7C5AE4334D7D4F67196B9D5B62&factor=1.25&id=e6fd2f97a893942e&redirect_counter=1
2010-03-19 11:16:22,136 11630 192.168.12.198 e6fd2f97a893942e URL_HIT YOUTUBE http://v13.nonxt2.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dVTlBMUl9FSkNNNV9IRlZF&fexp=902906%2C901801&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1269014400&key=yt1&signature=57C93FECEB4E3A2F9A0E96D8CD7675CD5AED8EC4.5BA3AAAC111A9F7C5AE4334D7D4F67196B9D5B62&factor=1.25&id=e6fd2f97a893942e&redirect_counter=1
2010-03-19 11:16:22,137 11630 192.168.12.198 e6fd2f97a893942e CACHE_MISS YOUTUBE Requested video was not found in cache.
2010-03-19 11:16:22,143 11630 192.168.12.198 e6fd2f97a893942e NEW_URL YOUTUBE http://v13.nonxt2.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dVTlBMUl9FSkNNNV9IRlZF&fexp=902906%2C901801&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1269014400&key=yt1&signature=57C93FECEB4E3A2F9A0E96D8CD7675CD5AED8EC4.5BA3AAAC111A9F7C5AE4334D7D4F67196B9D5B62&factor=1.25&id=e6fd2f97a893942e&redirect_counter=1
2010-03-19 11:16:23,464 11630 192.168.12.198 - REQUEST - http://r14.led1.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dVTlBMUl9FSkNNNV9IRlZF&fexp=902906%2C901801&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1269014400&key=yt1&signature=57C93FECEB4E3A2F9A0E96D8CD7675CD5AED8EC4.5BA3AAAC111A9F7C5AE4334D7D4F67196B9D5B62&factor=1.25&id=e6fd2f97a893942e&redirect_counter=1&st=nx
2010-03-19 11:16:23,464 11630 192.168.12.198 e6fd2f97a893942e URL_HIT YOUTUBE http://r14.led1.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dVTlBMUl9FSkNNNV9IRlZF&fexp=902906%2C901801&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1269014400&key=yt1&signature=57C93FECEB4E3A2F9A0E96D8CD7675CD5AED8EC4.5BA3AAAC111A9F7C5AE4334D7D4F67196B9D5B62&factor=1.25&id=e6fd2f97a893942e&redirect_counter=1&st=nx
2010-03-19 11:16:23,464 11630 192.168.12.198 e6fd2f97a893942e CACHE_MISS YOUTUBE Requested video was not found in cache.
## snip ##

Could you please update your package so that it can install via RPM ? alternatively, help me to fix this error ? I think many people have this problem :(

Thanks again, for your effort and software, I have heard many good things, I would love to see it in action :)

ciao/Riaan

6 Answers

by Kulbir Saini on 20 Mar 2010

Hi Riaan,

Thanks for such a detailed bug report.

  1. CentOS people are not keeping up with rpm development and that seems to create problem with installation via RPM. I'll still try to rebuild rpm with older versions of rpmlib.
  2. Did you configure videocache via /etc/videocache.conf properly? Please visit configure page if you have not already.
  3. Your videocache doesn't seem to schedule videos. Can you please dump the whole videocache.log at http://pastebin.com/ and give the url here in comment?

Thank you!

by star3am on 23 Mar 2010

Hallo Admin, thank you very much for coming back to me, much appreciated !

CentOS people are not keeping up with rpm development and that seems to create problem with installation via RPM. I'll still try to rebuild rpm with older versions of rpmlib.

I know :/ coming from Gentoo, bleeding edge software Centos can be a bit of a pain, that said, it's pretty stable, so I can somewhat understand

Did you configure videocache via /etc/videocache.conf properly? Please visit configure page if you have not already.

Yes, at first I tried the default, Squid runs on the same machine, I experienced the same trouble, the files do not get downloaded, size always 0 bytes :/

I then went through your forum to troubleshoot, I have tried various options, but none gave me any luck

here is my config file, as it is currently, http://pastebin.com/2dT23T7B

Your videocache doesn't seem to schedule videos. Can you please dump the whole videocache.log at http://pastebin.com/ and give the url here in comment?

Sure, here it is, http://pastebin.com/WJKicGKg

I first tried to keep everything as default as possible, with some minor tweaks like permissions and proxy address and also videocach dir size as your forum suggested,

here is my squid.conf maybe, that can help too,

acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
# Videocache settings
url_rewrite_program /usr/bin/python /usr/share/videocache/videocache.py
url_rewrite_children 10
acl videocache_allow_url url_regex -i \\.youtube\\.com\\/get_video\\?
acl videocache_allow_url url_regex -i \\.youtube\\.com\\/videoplayback \\.youtube\\.com\\/videoplay \\.youtube\\.com\\/get_video\\?
acl videocache_allow_url url_regex -i \\.youtube\\.com\\/videoplayback
acl videocache_allow_url url_regex -i \\.youtube\\.[a-z][a-z]\\/videoplayback \\.youtube\\.[a-z][a-z]\\/videoplay \\.youtube\\.[a-z][a-z]\\/get_video\\?
acl videocache_allow_url url_regex -i \\.googlevideo\\.com\\/videoplayback \\.googlevideo\\.com\\/videoplay \\.googlevideo\\.com\\/get_video\\?
acl videocache_allow_url url_regex -i \\.google\\.com\\/videoplayback \\.google\\.com\\/videoplay \\.google\\.com\\/get_video\\?
acl videocache_allow_url url_regex -i \\.google\\.[a-z][a-z]\\/videoplayback \\.google\\.[a-z][a-z]\\/videoplay \\.google\\.[a-z][a-z]\\/get_video\\?
acl videocache_allow_url url_regex -i (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\/videoplayback\\?
acl videocache_allow_url url_regex -i (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\/videoplay\\?
acl videocache_allow_url url_regex -i (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\/get_video\\?
acl videocache_allow_url url_regex -i proxy[a-z0-9\\-][a-z0-9][a-z0-9][a-z0-9]?\\.dailymotion\\.com\\/
acl videocache_allow_url url_regex -i vid\\.akm\\.dailymotion\\.com\\/
acl videocache_allow_url url_regex -i [a-z0-9][0-9a-z][0-9a-z]?[0-9a-z]?[0-9a-z]?\\.xtube\\.com\\/(.*)flv
acl videocache_allow_url url_regex -i bitcast\\.vimeo\\.com\\/vimeo\\/videos\\/
acl videocache_allow_url url_regex -i va\\.wrzuta\\.pl\\/wa[0-9][0-9][0-9][0-9]?
acl videocache_allow_url url_regex -i \\.files\\.youporn\\.com\\/(.*)\\/flv\\/
acl videocache_allow_url url_regex -i \\.msn\\.com\\.edgesuite\\.net\\/(.*)\\.flv
acl videocache_allow_url url_regex -i media[a-z0-9]?[a-z0-9]?[a-z0-9]?\\.tube8\\.com\\/ mobile[a-z0-9]?[a-z0-9]?[a-z0-9]?\\.tube8\\.com\\/
acl videocache_allow_url url_regex -i \\.mais\\.uol\\.com\\.br\\/(.*)\\.flv
acl videocache_allow_url url_regex -i \\.video[a-z0-9]?[a-z0-9]?\\.blip\\.tv\\/(.*)\\.(flv|avi|mov|mp3|m4v|mp4|wmv|rm|ram)
acl videocache_allow_url url_regex -i video\\.break\\.com\\/(.*)\\.(flv|mp4)
acl videocache_allow_dom dstdomain .mccont.com .metacafe.com .redtube.com .cdn.dailymotion.com
acl videocache_deny_url url_regex -i http:\\/\\/[a-z][a-z]\\.youtube\\.com http:\\/\\/www\\.youtube\\.com
url_rewrite_access deny videocache_deny_url
url_rewrite_access allow videocache_allow_url
url_rewrite_access allow videocache_allow_dom
redirector_bypass on

Videocache settings and the apache videocache config,

#                                                                            #
# file : /etc/httpd/conf.d/videocache.conf                                   #
#                                                                            #
# videocache is a squid url rewriter to cache videos from various websites.  #
# Check http://cachevideos.com/ for more details.                            #
#                                                                            #
# ----------------------------- Note This ---------------------------------- #
# Don't change this file unless you have good knowledge of how Apache works. #
# Don't forget to reload httpd and squid services if you change this file.   #
#                                                                            #

Alias /videocache /var/spool/videocache
<Directory /var/spool/videocache>
  Options +Indexes
  Order Allow,Deny
  Allow from all
</Directory>

I'd be happy to supply you with any other info you need, I'm pretty keen to get videocache going :)

Again, great project, thank you
Riaan

by star3am on 24 Mar 2010

Hi, I have made a reply, and have posted it, do you have any updates for me ? Squid is now in production, and still no video cache, please could you update me ?

Thank you

by Kulbir Saini on 26 Mar 2010

Hi!

We have a new version of videocache now. Please contact using contact link and I'll send over the new version.

PS : This time I have compiled it with older rpmlib :)

Thank You!

by Ted on 26 Mar 2010

Hello,
after running videocache 1.9.2 for almost a week i got the same problem, i tried everything, but no result. now i turned off videocache and waiting to get the newer version to try it.

Regards;

by star3am on 29 Apr 2010

Hi admin, hope your doing well

Just something I picked up, when you reload squid on Centos 5.4 videocache stops working and generates core dump,

-rw-------  1 squid squid 176M Apr 29 09:34 core.24199
-rw-------  1 squid squid 201M Apr 29 14:40 core.24488

Steps to reproduce,

in squid.conf

coredump_dir /var/log/squid

Reload Squid

service squid reload

Now, no videos will be fetched, /var/log/videocache/videocache.log

2010-04-29 11:02:22,021 18487 - - SCHEDULER - Download Scheduler starting.
2010-04-29 11:38:47,254 22338 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:05:14,383 24986 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:05:14,439 24980 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:05:14,447 24982 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:06:49,822 25119 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:32:29,993 27486 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:34:05,592 27623 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:45:20,718 28581 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:55:00,934 29434 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:59:50,811 29851 - - SCHEDULER - Download Scheduler starting.
2010-04-29 13:01:29,054 30118 - - SCHEDULER - Download Scheduler starting.
2010-04-29 13:07:54,805 30699 - - SCHEDULER - Download Scheduler starting.
2010-04-29 13:33:32,786 605 - - SCHEDULER - Download Scheduler starting.
2010-04-29 13:59:14,822 2878 - - SCHEDULER - Download Scheduler starting.
2010-04-29 14:05:41,849 3590 - - SCHEDULER - Download Scheduler starting.
2010-04-29 14:10:31,131 4168 - - SCHEDULER - Download Scheduler starting.

Check for Video Cache process

# ps aux | grep video
squid     7195  1.4  0.1  12728  6932 ?        Ss   14:39   0:00 (python) /usr/share/videocache/videocache.py
squid     7196  1.5  0.1  12728  6848 ?        Ss   14:39   0:00 (python) /usr/share/videocache/videocache.py
squid     7197  1.5  0.1  12728  6936 ?        Ss   14:39   0:00 (python) /usr/share/videocache/videocache.py
squid     7198  1.5  0.1  12728  6852 ?        Ss   14:39   0:00 (python) /usr/share/videocache/videocache.py
squid     7199  1.5  0.1  12728  6852 ?        Ss   14:39   0:00 (python) /usr/share/videocache/videocache.py
squid     7200  1.5  0.1  12728  6856 ?        Ss   14:39   0:00 (python) /usr/share/videocache/videocache.py
squid     7201  1.3  0.1  12728  6852 ?        Ss   14:39   0:00 (python) /usr/share/videocache/videocache.py
squid     7202  1.5  0.1  12728  6968 ?        Ss   14:39   0:00 (python) /usr/share/videocache/videocache.py
squid     7203  1.3  0.1  12728  6852 ?        Ss   14:39   0:00 (python) /usr/share/videocache/videocache.py
squid     7204  1.5  0.1  12728  6932 ?        Ss   14:39   0:00 (python) /usr/share/videocache/videocache.py
root      7245  0.0  0.0   3920   676 pts/0    S+   14:39   0:00 grep video
squid    13901  0.0  0.1  44600  6324 ?        S    09:53   0:00 (python) /usr/share/videocache/videocache.py
squid    13910  0.0  0.1  12732  6860 ?        Ss   09:54   0:00 (python) /usr/share/videocache/videocache.py
squid    13911  0.0  0.1  12732  6940 ?        Ss   09:54   0:00 (python) /usr/share/videocache/videocache.py
squid    13912  0.0  0.1  12732  6860 ?        Ss   09:54   0:00 (python) /usr/share/videocache/videocache.py
squid    13913  0.0  0.1  12732  6864 ?        Ss   09:54   0:00 (python) /usr/share/videocache/videocache.py
squid    13914  0.0  0.1  12732  6940 ?        Ss   09:54   0:00 (python) /usr/share/videocache/videocache.py
squid    13915  0.0  0.1  12732  6860 ?        Ss   09:54   0:00 (python) /usr/share/videocache/videocache.py
squid    13916  0.0  0.1  12732  6864 ?        Ss   09:54   0:00 (python) /usr/share/videocache/videocache.py
squid    13955  0.0  0.1  12732  6864 ?        Ss   09:54   0:00 (python) /usr/share/videocache/videocache.py
squid    13960  0.0  0.1  12732  6856 ?        Ss   09:54   0:00 (python) /usr/share/videocache/videocache.py
squid    13961  0.0  0.1  12732  6940 ?        Ss   09:54   0:00 (python) /usr/share/videocache/videocache.py
# Kill the processes 
# killall python
# killall python
python: no process killed

killall python && update-vc && service httpd reload && /etc/init.d/squid reload
# works, and I now see messages in the log
2010-04-29 14:40:33,545 7293 - - RELOAD - videocache plugin was reloaded.
2010-04-29 14:40:33,547 7290 - - RELOAD - videocache plugin was reloaded.
2010-04-29 14:40:33,547 7287 - - RELOAD - videocache plugin was reloaded.
2010-04-29 14:40:33,551 7291 - - RELOAD - videocache plugin was reloaded.
2010-04-29 14:40:34,855 7372 - - XMLRPCSERVER - Starting XMLRPCServer on port 9100.
2010-04-29 14:40:34,855 7372 - - SCHEDULER - Download Scheduler starting.
2010-04-29 14:41:00,042 7369 192.168.12.135 11HP72AiXpk URL_HIT YOUTUBE http://www.youtube.com/get_video?el=detailpage&noflv=1&video_id=11HP72AiXpk&asv=3&fmt=5&t=vjVQa1PpcFOgMMOZSvkudxGnfXOO-t-csYnSt0TAv28=
2010-04-29 14:41:00,043 7369 192.168.12.135 11HP72AiXpk CACHE_MISS YOUTUBE Requested video was not found in cache.

Again, great application, thanks for your effort, soon you will have no more bugs :)

cheers, Riaan

You need to sign in. Please sign in to add answer to this question.