vidoe cache not running properly
by Anonymous on 15 Nov 2008INFO 10.100.100.100 - NEW_URL YOUTUBE Error in parsing the url http://www.youtube.com/get_video?video_id=sq1dlf8nH-0&t=OEgsToPDskLpfqy3rw90Tdqu1V43Lmr6&el=detailpage&ps=&fmt=34
2008-11-15 10:55:47,834 INFO 10.100.110.120- REQUEST - http://v3.cache.googlevideo.com/videoplayback?id=b2ad5d95ff271fed&itag=34&begin=0&len=604800000&ip=x.x.x.x&signature=068C18B55071AED7821CC33617CE0015A55A77BA.3184C28A4C2A8B042C0B34CD47505D2187AA57B7&sver=2&expire=1226748416&key=yt4&ipbits=2
Well getting these types of messeages in youtube.log files ro miss_cache . if it is running then why i can not find any .flv files in /var/spool/squid/video_cache/youtube ? owner of this folder is squid and I also made a softlink of /var/spool/squid/vide_cache to /var/www/html/ but when i access it via browser i got error although ive set the permission to 777
Forbidden
You don't have permission to access /video_cache// on this server
Regards
Net_Spy
31 Answers
Did you configure your /etc/youtube_cache.conf properly? Please see this reply and try the tricks mentioned there. And if they don't work, get back with details on OS, version of youtube_cache etc.
PS : Why did you created a softlink? Was it mentioned in installation step or somewhere else?
Thanks for reply,,
Well Im using fedoracore 6 and I install rpm package of youtube cache and install the apache on fedro installation I've disabled the Selinux im using cutomise firewall . I'm not using the default port of squid im using transparent porxy port 8080, in youtube_cache.conf cache_hosst = ip of machine that have squid and apache installed . ive define the port 80 in youtube_cache.conf .
Regards
Net_Spy
"i've define the port 80 in youtube_cache.conf "
which port are you talking about? rpc_port. If yes, you should not change rpc_port unless it is already in use by some other program. Can you please paste your youtube_cache.conf in some pastebin service like this.
Hi!
Your proxy and cache_host options are not set correctly. What is x.x in 10.100.x.x ? Please replace x.x with actual integer values. For example,
proxy=http://10.100.2.3:3128/ and
cache_host=10.100.2.3 .
After doing this, restart squid service.
well here is the peice of code ive added into my squid.conf file
proxy=http://10.100.50.1:80/
cache_host=10.100.50.1 .
Regards
Net_Spy
Thats right. Now restart squid service and check if your youtube cache is working fine by having a look at youtube_cache.log. I think it should work now. Your squid.conf part is absolutely fine :)
well here is the httpd.log
I've removed all the folders from /var/www/html could you please tell me this procedure to create folder that may accessable via browser , cause i was getting forbiden error .
Regards
Net_Spy
I think you didn't follow the installation steps correctly. Please download the latest version and then try to follow the installation instructions step by step.
Ive installed the 1.4 the latest one and follow the steps too, but still no use.
i have trouble cachevideos
ready config youtube_cache.conf
no use iptables
disabled selinux
use centos 5.2 squid 2.6 (squid-arp)
have access http://10.0.0.6/video_cache/
Index of /video_cache
Icon Name Last modified Size Description[DIR] Parent Directory -
[DIR] dailymotion/ 15-Nov-2008 17:18 -
[DIR] google/ 15-Nov-2008 17:18 -
[DIR] metacafe/ 15-Nov-2008 17:18 -
[DIR] redtube/ 15-Nov-2008 17:18 -
[DIR] tmp/ 16-Nov-2008 13:00 -
[DIR] vimeo/ 15-Nov-2008 17:18 -
[DIR] wrzuta/ 15-Nov-2008 17:18 -
[DIR] xtube/ 15-Nov-2008 17:18 -
[DIR] youtube/ 15-Nov-2008 18:58 -
Apache/2.2.3 (CentOS) Server at apache Port 80
008-11-15 17:52:36,018 INFO - - SCHEDULEDER - Download Scheduler starting.
2008-11-15 17:52:36,061 INFO - - SCHEDULEDER - Download Scheduler starting.
2008-11-15 17:52:40,976 INFO 10.0.0.15 - REQUEST - http://dal-v145.dal.youtube.com/get_video?video_id=SCaWLaimDCQ&ip=201.230.102.164&signature=5E4C851E0C429CBAF66968B23FB982D590BFEB20.94C60C8358F637EAE862AE220D0F456597413986&sver=2&expire=1226810704&key=yt4&ipbits=2
2008-11-15 17:52:40,984 INFO 10.0.0.15 SCaWLaimDCQ URL_HIT YOUTUBE http://dal-v145.dal.youtube.com/get_video?video_id=SCaWLaimDCQ&ip=201.230.102.164&signature=5E4C851E0C429CBAF66968B23FB982D590BFEB20.94C60C8358F637EAE862AE220D0F456597413986&sver=2&expire=1226810704&key=yt4&ipbits=2
2008-11-15 17:52:40,985 INFO 10.0.0.15 SCaWLaimDCQ CACHE_MISS YOUTUBE Requested video was not found in cache.2008-11-15 17:52:36,018 INFO - - SCHEDULEDER - Download Scheduler starting.
2008-11-15 17:52:36,061 INFO - - SCHEDULEDER - Download Scheduler starting.
2008-11-15 17:52:40,976 INFO 10.0.0.15 - REQUEST - http://dal-v145.dal.youtube.com/get_video?video_id=SCaWLaimDCQ&ip=201.230.102.164&signature=5E4C851E0C429CBAF66968B23FB982D590BFEB20.94C60C8358F637EAE862AE220D0F456597413986&sver=2&expire=1226810704&key=yt4&ipbits=2
2008-11-15 17:52:40,984 INFO 10.0.0.15 SCaWLaimDCQ URL_HIT YOUTUBE http://dal-v145.dal.youtube.com/get_video?video_id=SCaWLaimDCQ&ip=201.230.102.164&signature=5E4C851E0C429CBAF66968B23FB982D590BFEB20.94C60C8358F637EAE862AE220D0F456597413986&sver=2&expire=1226810704&key=yt4&ipbits=2
2008-11-15 17:52:40,985 INFO 10.0.0.15 SCaWLaimDCQ CACHE_MISS YOUTUBE Requested video was not found in cache.
well here is my youtubecache.log
2008-11-18 11:14:36,741 INFO 10.100.39.129 f7U6x5It5Fo NEW_URL YOUTUBE http://nyc-v312.nyc.youtube.com/get_video?video_id=f7U6x5It5Fo&ip=202.209.30.100.2&signature=9904A66792A194084C6510E43B4D70AD77DEA31D.9354D4CB676F0F918A79BFF80DED6437AD96661C&sver=2&expire=1227008816&key=yt4&ipbits=0
2008-11-18 11:14:39,676 INFO 10.100.39.129 f7U6x5It5Fo SCHEDULED YOUTUBE Video scheduled for download.
2008-11-18 11:14:39,703 INFO 10.100.39.129 f7U6x5It5Fo DOWNLOAD_ERR YOUTUBE An error occured while retrieving the video.
2008-11-18 11:14:50,150 INFO 10.100.9.88 - REQUEST - http://www.youtube.com/get_video?video_id=1DMUO419RZg&t=OEgsToPDskI_bdV21EWPgk0JkECkgXrf&el=detailpage&ps=
2008-11-18 11:14:50,153 INFO 10.100.9.88 1DMUO419RZg URL_HIT YOUTUBE http://www.youtube.com/get_video?video_id=1DMUO419RZg&t=OEgsToPDskI_bdV21EWPgk0JkECkgXrf&el=detailpage&ps=
2008-11-18 11:14:50,153 INFO 10.100.9.88 1DMUO419RZg CACHE_MISS YOUTUBE Requested video was not found in cache.
2008-11-18 11:14:50,155 INFO 10.100.9.88 1DMUO419RZg NEW_URL YOUTUBE http://www.youtube.com/get_video?video_id=1DMUO419RZg&t=OEgsToPDskI_bdV21EWPgk0JkECkgXrf&el=detailpage&ps=
2008-11-18 11:14:50,730 INFO 10.100.9.88 - REQUEST - http://v22.cache.googlevideo.com/get_video?origin=lax-v56.lax.youtube.com&video_id=1DMUO419RZg&ip=202.209.30.100.2&signature=048FA43B099FC5A00A31B3F87CB4A25FFB5AE0A0.5A002C9943385E9E08C263077D1AF8E4FC15A559&sver=2&expire=1227008830&key=yt4&ipbits=0
2008-11-18 11:14:51,688 INFO 10.100.9.88 1DMUO419RZg SCHEDULED YOUTUBE Video scheduled for download.
2008-11-18 11:14:51,715 INFO 10.100.9.88 1DMUO419RZg DOWNLOAD_ERR YOUTUBE An error occured while retrieving the video.
2008-11-18 11:15:02,988 INFO 10.100.9.88 - REQUEST - http://www.youtube.com/get_video?video_id=gCf6E1K4x8I&t=OEgsToPDskISrl-LSnbtIULDxwPs7BFM&el=detailpage&ps=
2008-11-18 11:15:02,992 INFO 10.100.9.88 gCf6E1K4x8I URL_HIT YOUTUBE http://www.youtube.com/get_video?video_id=gCf6E1K4x8I&t=OEgsToPDskISrl-LSnbtIULDxwPs7BFM&el=detailpage&ps=
2008-11-18 11:15:02,992 INFO 10.100.9.88 gCf6E1K4x8I CACHE_MISS YOUTUBE Requested video was not found in cache.
2008-11-18 11:15:02,994 INFO 10.100.9.88 gCf6E1K4x8I NEW_URL YOUTUBE http://www.youtube.com/get_video?video_id=gCf6E1K4x8I&t=OEgsToPDskISrl-LSnbtIULDxwPs7BFM&el=detailpage&ps=
2008-11-18 11:15:03,556 INFO 10.100.9.88 - REQUEST - http://dal-v124.dal.youtube.com/get_video?video_id=gCf6E1K4x8I&ip=202.209.30.100.2&signature=1C09FCF2B250C7BEB221DA6786CF0111340CBE08.ADFFEC59202CF9B1A97638FE4D2ACE5A27D72B7C&sver=2&expire=1227008843&key=yt4&ipbits=0
2008-11-18 11:15:03,713 INFO 10.100.9.88 gCf6E1K4x8I SCHEDULED YOUTUBE Video scheduled for download.
2008-11-18 11:15:03,742 INFO 10.100.9.88 gCf6E1K4x8I DOWNLOAD_ERR YOUTUBE An error occured while retrieving the video.
Can you paste the following files at pastebin.com .
- /etc/youtube_cache.conf
- /etc/httpd/conf.d/youtube_cache.conf
- /etc/squid/squid.conf
Also, mention your operating system. And how did you install from rpm or source?
I have installed it via rpm and i could not fine any file name youtube_cache.conf in /etc/httpd/conf/ .
youtube_cache.conf
http://pastebin.com/m2b70f3a2
Squid.con
http://pastebin.com/m4a948566
In your youtube_cache.conf file, following things are wrong
1.proxy = http://10.100.x.x:80/
It should be proxy = http://10.100.50.1:8080/ Why are you putting 80 as port when your proxy is running on port 8080??
2.cache_host = 10.100.x.x
It should be cach_host = 10.100.50.1 I don't know why are you putting x there?
3.youtube_cache_dir = /var/spool/squid/vide0_cache/youtube
Did you read the description written in comments? It should be youtube_cache_dir = youtube .
Fix these options in /etc/youtube_cache.conf file.
Your squid.conf file is fine. No problems in there. If you read my reply carefully, the location for the other youtube_cache.conf is /etc/httpd/conf.d/ and not /etc/httpd/conf/ .
Thankzzz but now ive only one problem that is forbidden in browser when accessing http://10.100.50.1/video_chat. Ive removed all directory from /var/www/html/video_chat and recreated it .
Do this steps and you'll be able to access it.
[root@localhost ~]# chmod 755 /var/spool/squid [ENTER]
Let me know if there is some problem.
I think there was a issue with apache permission , well ive removed the old package and install the 1.5. 1.5 has different path for that fixes other bugs aswell but thanks for help Kulbir Saini . I've little thing to ask is that once ive .flv in /var/spool/youtube_cache/youtube how to move them under apache. if is file to create softlink ?? or I would better to creat a folder for that ?? manualy in apache /var/www/html/ .
Regards
Net_Spy
Is your youtube_cache working fine now? Actually there is no need to move files or create soft links. All the cached videos are already available at http://your_ip/video_cache/ . In case you want youtube files to be available in /var/www/html/, you can create a soft link like
ln -s /var/spool/video_cache/youtube /var/www/html/youtube/
But I'll recommend not to do so.
thanks I got few .flv file in youtube folder
I did tail -f /var/log/youtube_cache/youtube.log to check what is going on I found this
NEW_URL YOUTUBE Error in parsing the url http://www.youtube.com/get_video?video_id=jvwmA3N5HM4&t=OEgsToPDskJXl4CpE5uTETnJ6pTe_Yp8&el=detailpage&ps=
NEW_URL YOUTUBE Error in parsing the url http://v20.cache.googlevideo.com/videoplayback?id=d4771de2bca34be5&itag=18&begin=0&len=604800000&ip=202.30.111.33&signature=48973E5D8D3B23137B687AD146C09A5C3D947D83.B7F214D0A9192734CBA47FC429FB25E89957240D&sver=2&expire=1227292249&key=yt4&ipbits=0
like this for other urls of youtube. what if I increase the
max_parallel_downloads = 30 to 400 .
Regards
Net_Spy
Most probably your XMLRPC Server is not running or it has exited abnormally. Can you give me the output of command
[root@localhost ~]# netstat -atpn |grep python [ENTER]
If no app is listening on port 9100, then check /var/log/squid/cache.log for any errors related to python tracebacks. Get back with whatever result you get.
Don't increase max_parallel_downloads too much. Every parallel download will be one python process.
here is the out put
tcp 0 0 127.0.0.1:9100 0.0.0.0:* LISTEN 19865/(python)
tcp 0 0 127.0.0.1:48530 127.0.0.1:9100 ESTABLISHED 19945/(python)
tcp 0 0 127.0.0.1:48531 127.0.0.1:9100 ESTABLISHED 19942/(python)
tcp 0 0 127.0.0.1:48528 127.0.0.1:9100 ESTABLISHED 19936/(python)
tcp 0 0 127.0.0.1:48529 127.0.0.1:9100 ESTABLISHED 19939/(python)
tcp 0 0 127.0.0.1:48526 127.0.0.1:9100 ESTABLISHED 19883/(python)
tcp 0 0 127.0.0.1:48527 127.0.0.1:9100 ESTABLISHED 19865/(python)
tcp 0 242 127.0.0.1:46443 127.0.0.1:9100 ESTABLISHED 20161/(python)
tcp 0 242 127.0.0.1:46440 127.0.0.1:9100 ESTABLISHED 20159/(python)
tcp 0 242 127.0.0.1:46441 127.0.0.1:9100 ESTABLISHED 20160/(python)
tcp 0 242 127.0.0.1:46446 127.0.0.1:9100 ESTABLISHED 20164/(python)
tcp 0 242 127.0.0.1:46444 127.0.0.1:9100 ESTABLISHED 20162/(python)
tcp 0 242 127.0.0.1:46445 127.0.0.1:9100 ESTABLISHED 20163/(python)
tcp 0 242 127.0.0.1:46438 127.0.0.1:9100 ESTABLISHED 20157/(python)
tcp 0 242 127.0.0.1:46439 127.0.0.1:9100 ESTABLISHED 20158/(python)
tcp 0 242 127.0.0.1:46437 127.0.0.1:9100 ESTABLISHED 20156/(python)
I set the max_parallel download to 50 .
Regards
This is not really a normal behavior. Can you contact me via email using the contact tab above?
well another problem is that ive configured new package in moring at 11am and still I found old 8 files since moring and its now 10:09pm the files are same in youtube folder and rest of folders are empty.
Regards
Net_Spy
I am with this issue, resolved?
DOWNLOAD_ERR YOUTUBE An error occured while retrieving the video.
thanks
Hi!
Please provide further details from your /var/log/videocache/videocache.log, /var/log/squid/cache.log and /etc/videocache.conf. Any tracebacks or errors will help.
Thank you for using videocache :)
I'm using the directory /cache/videocache for the cache, in the log (error.log) of apache is giving the following message
"File does not exist: /cache/videocache/get_video"
my apache is configured, the browser (http://
sorry for my English.
Hi!
Did you run update-vc script after changing the base_dir in /etc/videocache.conf?
The update-vc script must be run every time you change something in /etc/videocache.conf.
Please get back if problem persists.
Thank you for using videocache :)
SOLVED! I'm caching!
was missing in the configuration of "proxy = " videocache.conf
thanks!