Centos 5.4 Squid 2.6 Video Cach does not work :/
by star3am on 19 Mar 2010Hi, I have heard great things about your application, but have yet to see it in action.
We purchased version 1.9.2 here is my full bug report.
1) the RPM is useless for Centos :/ as the versions needed is not available in any repo,
ERROR with rpm_check_debug vs depsolve:
rpmlib(FileDigests) is needed by videocache-1.9.2-1.noarch
Complete!
(1, [u'Please report this error in http://yum.baseurl.org/report'])
I get the same error with rpm -ivh
I then read your forum and you suggest to install the videocache-1.9.2.tar.gz
I now run python setup.py install which reports sucess, and I can tail your log, but the problem is that the Videos are never fetched, rather I see in
# ls -lah /var/spool/videocache/tmp/
total 8.0K
drwxr-xr-x 2 squid squid 4.0K Mar 19 11:25 .
drwxr-xr-x 17 squid squid 4.0K Mar 18 08:45 ..
-rw------- 1 squid squid 0 Mar 19 11:25 4316732.13460751.11.flv
The size are always 0 ... no matter what I do,
I check the dependencies,
# rpm -qa | grep python
python-elementtree-1.2.6-5
libxml2-python-2.6.26-2.1.2.8
libselinux-python-1.33.4-5.5.el5
python-urlgrabber-3.1.0-5.el5
gamin-python-0.1.7-8.el5
dbus-python-0.70-9.el5_4
python-iniparse-0.2.3-4.el5
audit-libs-python-1.7.13-2.el5
rpm-python-4.4.2.3-18.el5
python-sqlite-1.1.7-1.2.1
python-2.4.3-27.el5
All there, so what can be the problem, reading your forum, I gather that I should be able to access http://PROXY_IP_ADDRESS/videocache/
which i can, working perfectly, my problem is with download part,
here is a part of your videocache log, the error I always see is,
Requested video was not found in cache.
## snip ##
2010-03-19 11:16:20,950 11630 192.168.12.198 - REQUEST - http://v13.lscache2.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dVTlBMUl9FSkNNNV9IRlZF&fexp=902906%2C901801&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1269014400&key=yt1&signature=57C93FECEB4E3A2F9A0E96D8CD7675CD5AED8EC4.5BA3AAAC111A9F7C5AE4334D7D4F67196B9D5B62&factor=1.25&id=e6fd2f97a893942e
2010-03-19 11:16:20,950 11630 192.168.12.198 e6fd2f97a893942e URL_HIT YOUTUBE http://v13.lscache2.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dVTlBMUl9FSkNNNV9IRlZF&fexp=902906%2C901801&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1269014400&key=yt1&signature=57C93FECEB4E3A2F9A0E96D8CD7675CD5AED8EC4.5BA3AAAC111A9F7C5AE4334D7D4F67196B9D5B62&factor=1.25&id=e6fd2f97a893942e
2010-03-19 11:16:20,950 11630 192.168.12.198 e6fd2f97a893942e CACHE_MISS YOUTUBE Requested video was not found in cache.
2010-03-19 11:16:20,953 11630 192.168.12.198 e6fd2f97a893942e NEW_URL YOUTUBE http://v13.lscache2.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dVTlBMUl9FSkNNNV9IRlZF&fexp=902906%2C901801&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1269014400&key=yt1&signature=57C93FECEB4E3A2F9A0E96D8CD7675CD5AED8EC4.5BA3AAAC111A9F7C5AE4334D7D4F67196B9D5B62&factor=1.25&id=e6fd2f97a893942e
2010-03-19 11:16:22,136 11630 192.168.12.198 - REQUEST - http://v13.nonxt2.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dVTlBMUl9FSkNNNV9IRlZF&fexp=902906%2C901801&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1269014400&key=yt1&signature=57C93FECEB4E3A2F9A0E96D8CD7675CD5AED8EC4.5BA3AAAC111A9F7C5AE4334D7D4F67196B9D5B62&factor=1.25&id=e6fd2f97a893942e&redirect_counter=1
2010-03-19 11:16:22,136 11630 192.168.12.198 e6fd2f97a893942e URL_HIT YOUTUBE http://v13.nonxt2.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dVTlBMUl9FSkNNNV9IRlZF&fexp=902906%2C901801&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1269014400&key=yt1&signature=57C93FECEB4E3A2F9A0E96D8CD7675CD5AED8EC4.5BA3AAAC111A9F7C5AE4334D7D4F67196B9D5B62&factor=1.25&id=e6fd2f97a893942e&redirect_counter=1
2010-03-19 11:16:22,137 11630 192.168.12.198 e6fd2f97a893942e CACHE_MISS YOUTUBE Requested video was not found in cache.
2010-03-19 11:16:22,143 11630 192.168.12.198 e6fd2f97a893942e NEW_URL YOUTUBE http://v13.nonxt2.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dVTlBMUl9FSkNNNV9IRlZF&fexp=902906%2C901801&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1269014400&key=yt1&signature=57C93FECEB4E3A2F9A0E96D8CD7675CD5AED8EC4.5BA3AAAC111A9F7C5AE4334D7D4F67196B9D5B62&factor=1.25&id=e6fd2f97a893942e&redirect_counter=1
2010-03-19 11:16:23,464 11630 192.168.12.198 - REQUEST - http://r14.led1.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dVTlBMUl9FSkNNNV9IRlZF&fexp=902906%2C901801&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1269014400&key=yt1&signature=57C93FECEB4E3A2F9A0E96D8CD7675CD5AED8EC4.5BA3AAAC111A9F7C5AE4334D7D4F67196B9D5B62&factor=1.25&id=e6fd2f97a893942e&redirect_counter=1&st=nx
2010-03-19 11:16:23,464 11630 192.168.12.198 e6fd2f97a893942e URL_HIT YOUTUBE http://r14.led1.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dVTlBMUl9FSkNNNV9IRlZF&fexp=902906%2C901801&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1269014400&key=yt1&signature=57C93FECEB4E3A2F9A0E96D8CD7675CD5AED8EC4.5BA3AAAC111A9F7C5AE4334D7D4F67196B9D5B62&factor=1.25&id=e6fd2f97a893942e&redirect_counter=1&st=nx
2010-03-19 11:16:23,464 11630 192.168.12.198 e6fd2f97a893942e CACHE_MISS YOUTUBE Requested video was not found in cache.
## snip ##
Could you please update your package so that it can install via RPM ? alternatively, help me to fix this error ? I think many people have this problem :(
Thanks again, for your effort and software, I have heard many good things, I would love to see it in action :)
ciao/Riaan
6 Answers
Hi Riaan,
Thanks for such a detailed bug report.
- CentOS people are not keeping up with rpm development and that seems to create problem with installation via RPM. I'll still try to rebuild rpm with older versions of rpmlib.
- Did you configure videocache via /etc/videocache.conf properly? Please visit configure page if you have not already.
- Your videocache doesn't seem to schedule videos. Can you please dump the whole videocache.log at http://pastebin.com/ and give the url here in comment?
Thank you!
Hallo Admin, thank you very much for coming back to me, much appreciated !
CentOS people are not keeping up with rpm development and that seems to create problem with installation via RPM. I'll still try to rebuild rpm with older versions of rpmlib.
I know :/ coming from Gentoo, bleeding edge software Centos can be a bit of a pain, that said, it's pretty stable, so I can somewhat understand
Did you configure videocache via /etc/videocache.conf properly? Please visit configure page if you have not already.
Yes, at first I tried the default, Squid runs on the same machine, I experienced the same trouble, the files do not get downloaded, size always 0 bytes :/
I then went through your forum to troubleshoot, I have tried various options, but none gave me any luck
here is my config file, as it is currently, http://pastebin.com/2dT23T7B
Your videocache doesn't seem to schedule videos. Can you please dump the whole videocache.log at http://pastebin.com/ and give the url here in comment?
Sure, here it is, http://pastebin.com/WJKicGKg
I first tried to keep everything as default as possible, with some minor tweaks like permissions and proxy address and also videocach dir size as your forum suggested,
here is my squid.conf maybe, that can help too,
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
# Videocache settings
url_rewrite_program /usr/bin/python /usr/share/videocache/videocache.py
url_rewrite_children 10
acl videocache_allow_url url_regex -i \\.youtube\\.com\\/get_video\\?
acl videocache_allow_url url_regex -i \\.youtube\\.com\\/videoplayback \\.youtube\\.com\\/videoplay \\.youtube\\.com\\/get_video\\?
acl videocache_allow_url url_regex -i \\.youtube\\.com\\/videoplayback
acl videocache_allow_url url_regex -i \\.youtube\\.[a-z][a-z]\\/videoplayback \\.youtube\\.[a-z][a-z]\\/videoplay \\.youtube\\.[a-z][a-z]\\/get_video\\?
acl videocache_allow_url url_regex -i \\.googlevideo\\.com\\/videoplayback \\.googlevideo\\.com\\/videoplay \\.googlevideo\\.com\\/get_video\\?
acl videocache_allow_url url_regex -i \\.google\\.com\\/videoplayback \\.google\\.com\\/videoplay \\.google\\.com\\/get_video\\?
acl videocache_allow_url url_regex -i \\.google\\.[a-z][a-z]\\/videoplayback \\.google\\.[a-z][a-z]\\/videoplay \\.google\\.[a-z][a-z]\\/get_video\\?
acl videocache_allow_url url_regex -i (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\/videoplayback\\?
acl videocache_allow_url url_regex -i (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\/videoplay\\?
acl videocache_allow_url url_regex -i (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\/get_video\\?
acl videocache_allow_url url_regex -i proxy[a-z0-9\\-][a-z0-9][a-z0-9][a-z0-9]?\\.dailymotion\\.com\\/
acl videocache_allow_url url_regex -i vid\\.akm\\.dailymotion\\.com\\/
acl videocache_allow_url url_regex -i [a-z0-9][0-9a-z][0-9a-z]?[0-9a-z]?[0-9a-z]?\\.xtube\\.com\\/(.*)flv
acl videocache_allow_url url_regex -i bitcast\\.vimeo\\.com\\/vimeo\\/videos\\/
acl videocache_allow_url url_regex -i va\\.wrzuta\\.pl\\/wa[0-9][0-9][0-9][0-9]?
acl videocache_allow_url url_regex -i \\.files\\.youporn\\.com\\/(.*)\\/flv\\/
acl videocache_allow_url url_regex -i \\.msn\\.com\\.edgesuite\\.net\\/(.*)\\.flv
acl videocache_allow_url url_regex -i media[a-z0-9]?[a-z0-9]?[a-z0-9]?\\.tube8\\.com\\/ mobile[a-z0-9]?[a-z0-9]?[a-z0-9]?\\.tube8\\.com\\/
acl videocache_allow_url url_regex -i \\.mais\\.uol\\.com\\.br\\/(.*)\\.flv
acl videocache_allow_url url_regex -i \\.video[a-z0-9]?[a-z0-9]?\\.blip\\.tv\\/(.*)\\.(flv|avi|mov|mp3|m4v|mp4|wmv|rm|ram)
acl videocache_allow_url url_regex -i video\\.break\\.com\\/(.*)\\.(flv|mp4)
acl videocache_allow_dom dstdomain .mccont.com .metacafe.com .redtube.com .cdn.dailymotion.com
acl videocache_deny_url url_regex -i http:\\/\\/[a-z][a-z]\\.youtube\\.com http:\\/\\/www\\.youtube\\.com
url_rewrite_access deny videocache_deny_url
url_rewrite_access allow videocache_allow_url
url_rewrite_access allow videocache_allow_dom
redirector_bypass on
Videocache settings and the apache videocache config,
# #
# file : /etc/httpd/conf.d/videocache.conf #
# #
# videocache is a squid url rewriter to cache videos from various websites. #
# Check http://cachevideos.com/ for more details. #
# #
# ----------------------------- Note This ---------------------------------- #
# Don't change this file unless you have good knowledge of how Apache works. #
# Don't forget to reload httpd and squid services if you change this file. #
# #
Alias /videocache /var/spool/videocache
<Directory /var/spool/videocache>
Options +Indexes
Order Allow,Deny
Allow from all
</Directory>
I'd be happy to supply you with any other info you need, I'm pretty keen to get videocache going :)
Again, great project, thank you
Riaan
Hi, I have made a reply, and have posted it, do you have any updates for me ? Squid is now in production, and still no video cache, please could you update me ?
Thank you
Hi!
We have a new version of videocache now. Please contact using contact link and I'll send over the new version.
PS : This time I have compiled it with older rpmlib :)
Thank You!
Hello,
after running videocache 1.9.2 for almost a week i got the same problem, i tried everything, but no result. now i turned off videocache and waiting to get the newer version to try it.
Regards;
Hi admin, hope your doing well
Just something I picked up, when you reload squid on Centos 5.4 videocache stops working and generates core dump,
-rw------- 1 squid squid 176M Apr 29 09:34 core.24199
-rw------- 1 squid squid 201M Apr 29 14:40 core.24488
Steps to reproduce,
in squid.conf
coredump_dir /var/log/squid
Reload Squid
service squid reload
Now, no videos will be fetched, /var/log/videocache/videocache.log
2010-04-29 11:02:22,021 18487 - - SCHEDULER - Download Scheduler starting.
2010-04-29 11:38:47,254 22338 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:05:14,383 24986 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:05:14,439 24980 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:05:14,447 24982 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:06:49,822 25119 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:32:29,993 27486 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:34:05,592 27623 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:45:20,718 28581 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:55:00,934 29434 - - SCHEDULER - Download Scheduler starting.
2010-04-29 12:59:50,811 29851 - - SCHEDULER - Download Scheduler starting.
2010-04-29 13:01:29,054 30118 - - SCHEDULER - Download Scheduler starting.
2010-04-29 13:07:54,805 30699 - - SCHEDULER - Download Scheduler starting.
2010-04-29 13:33:32,786 605 - - SCHEDULER - Download Scheduler starting.
2010-04-29 13:59:14,822 2878 - - SCHEDULER - Download Scheduler starting.
2010-04-29 14:05:41,849 3590 - - SCHEDULER - Download Scheduler starting.
2010-04-29 14:10:31,131 4168 - - SCHEDULER - Download Scheduler starting.
Check for Video Cache process
# ps aux | grep video
squid 7195 1.4 0.1 12728 6932 ? Ss 14:39 0:00 (python) /usr/share/videocache/videocache.py
squid 7196 1.5 0.1 12728 6848 ? Ss 14:39 0:00 (python) /usr/share/videocache/videocache.py
squid 7197 1.5 0.1 12728 6936 ? Ss 14:39 0:00 (python) /usr/share/videocache/videocache.py
squid 7198 1.5 0.1 12728 6852 ? Ss 14:39 0:00 (python) /usr/share/videocache/videocache.py
squid 7199 1.5 0.1 12728 6852 ? Ss 14:39 0:00 (python) /usr/share/videocache/videocache.py
squid 7200 1.5 0.1 12728 6856 ? Ss 14:39 0:00 (python) /usr/share/videocache/videocache.py
squid 7201 1.3 0.1 12728 6852 ? Ss 14:39 0:00 (python) /usr/share/videocache/videocache.py
squid 7202 1.5 0.1 12728 6968 ? Ss 14:39 0:00 (python) /usr/share/videocache/videocache.py
squid 7203 1.3 0.1 12728 6852 ? Ss 14:39 0:00 (python) /usr/share/videocache/videocache.py
squid 7204 1.5 0.1 12728 6932 ? Ss 14:39 0:00 (python) /usr/share/videocache/videocache.py
root 7245 0.0 0.0 3920 676 pts/0 S+ 14:39 0:00 grep video
squid 13901 0.0 0.1 44600 6324 ? S 09:53 0:00 (python) /usr/share/videocache/videocache.py
squid 13910 0.0 0.1 12732 6860 ? Ss 09:54 0:00 (python) /usr/share/videocache/videocache.py
squid 13911 0.0 0.1 12732 6940 ? Ss 09:54 0:00 (python) /usr/share/videocache/videocache.py
squid 13912 0.0 0.1 12732 6860 ? Ss 09:54 0:00 (python) /usr/share/videocache/videocache.py
squid 13913 0.0 0.1 12732 6864 ? Ss 09:54 0:00 (python) /usr/share/videocache/videocache.py
squid 13914 0.0 0.1 12732 6940 ? Ss 09:54 0:00 (python) /usr/share/videocache/videocache.py
squid 13915 0.0 0.1 12732 6860 ? Ss 09:54 0:00 (python) /usr/share/videocache/videocache.py
squid 13916 0.0 0.1 12732 6864 ? Ss 09:54 0:00 (python) /usr/share/videocache/videocache.py
squid 13955 0.0 0.1 12732 6864 ? Ss 09:54 0:00 (python) /usr/share/videocache/videocache.py
squid 13960 0.0 0.1 12732 6856 ? Ss 09:54 0:00 (python) /usr/share/videocache/videocache.py
squid 13961 0.0 0.1 12732 6940 ? Ss 09:54 0:00 (python) /usr/share/videocache/videocache.py
# Kill the processes
# killall python
# killall python
python: no process killed
killall python && update-vc && service httpd reload && /etc/init.d/squid reload
# works, and I now see messages in the log
2010-04-29 14:40:33,545 7293 - - RELOAD - videocache plugin was reloaded.
2010-04-29 14:40:33,547 7290 - - RELOAD - videocache plugin was reloaded.
2010-04-29 14:40:33,547 7287 - - RELOAD - videocache plugin was reloaded.
2010-04-29 14:40:33,551 7291 - - RELOAD - videocache plugin was reloaded.
2010-04-29 14:40:34,855 7372 - - XMLRPCSERVER - Starting XMLRPCServer on port 9100.
2010-04-29 14:40:34,855 7372 - - SCHEDULER - Download Scheduler starting.
2010-04-29 14:41:00,042 7369 192.168.12.135 11HP72AiXpk URL_HIT YOUTUBE http://www.youtube.com/get_video?el=detailpage&noflv=1&video_id=11HP72AiXpk&asv=3&fmt=5&t=vjVQa1PpcFOgMMOZSvkudxGnfXOO-t-csYnSt0TAv28=
2010-04-29 14:41:00,043 7369 192.168.12.135 11HP72AiXpk CACHE_MISS YOUTUBE Requested video was not found in cache.
Again, great application, thanks for your effort, soon you will have no more bugs :)
cheers, Riaan