VideoCache

vidoe cache not running properly

by Anonymous on 15 Nov 2008
INFO 10.100.100.100 - NEW_URL YOUTUBE Error in parsing the url http://www.youtube.com/get_video?video_id=sq1dlf8nH-0&t=OEgsToPDskLpfqy3rw90Tdqu1V43Lmr6&el=detailpage&ps=&fmt=34
2008-11-15 10:55:47,834 INFO 10.100.110.120- REQUEST - http://v3.cache.googlevideo.com/videoplayback?id=b2ad5d95ff271fed&itag=34&begin=0&len=604800000&ip=x.x.x.x&signature=068C18B55071AED7821CC33617CE0015A55A77BA.3184C28A4C2A8B042C0B34CD47505D2187AA57B7&sver=2&expire=1226748416&key=yt4&ipbits=2

Well getting these types of messeages in youtube.log files ro miss_cache . if it is running then why i can not find any .flv files in /var/spool/squid/video_cache/youtube ? owner of this folder is squid and I also made a softlink of /var/spool/squid/vide_cache to /var/www/html/ but when i access it via browser i got error although ive set the permission to 777

Forbidden
You don't have permission to access /video_cache// on this server

Regards
Net_Spy

31 Answers

by Kulbir Saini on 16 Nov 2008

Did you configure your /etc/youtube_cache.conf properly? Please see this reply and try the tricks mentioned there. And if they don't work, get back with details on OS, version of youtube_cache etc.

PS : Why did you created a softlink? Was it mentioned in installation step or somewhere else?

by Anonymous on 16 Nov 2008

Thanks for reply,,

Well Im using fedoracore 6 and I install rpm package of youtube cache and install the apache on fedro installation I've disabled the Selinux im using cutomise firewall . I'm not using the default port of squid im using transparent porxy port 8080, in youtube_cache.conf cache_hosst = ip of machine that have squid and apache installed . ive define the port 80 in youtube_cache.conf .

Regards
Net_Spy

by Kulbir Saini on 16 Nov 2008

"i've define the port 80 in youtube_cache.conf "
which port are you talking about? rpc_port. If yes, you should not change rpc_port unless it is already in use by some other program. Can you please paste your youtube_cache.conf in some pastebin service like this.

by Anonymous on 17 Nov 2008

here is the conf file

http://pastebin.com/m2b70f3a2

Hope issue will be resolve soon.

Regards
Net_Spy

by Kulbir Saini on 17 Nov 2008

Hi!

Your proxy and cache_host options are not set correctly. What is x.x in 10.100.x.x ? Please replace x.x with actual integer values. For example,

proxy=http://10.100.2.3:3128/ and
cache_host=10.100.2.3 .

After doing this, restart squid service.

by Anonymous on 17 Nov 2008

well here is the peice of code ive added into my squid.conf file

http://pastebin.com/m2821ed6c

proxy=http://10.100.50.1:80/
cache_host=10.100.50.1 .

Regards

Net_Spy

by Kulbir Saini on 17 Nov 2008

Thats right. Now restart squid service and check if your youtube cache is working fine by having a look at youtube_cache.log. I think it should work now. Your squid.conf part is absolutely fine :)

by Anonymous on 17 Nov 2008

well here is the httpd.log

http://pastebin.com/m3d8ea423

I've removed all the folders from /var/www/html could you please tell me this procedure to create folder that may accessable via browser , cause i was getting forbiden error .

Regards
Net_Spy

by Kulbir Saini on 17 Nov 2008

I think you didn't follow the installation steps correctly. Please download the latest version and then try to follow the installation instructions step by step.

by Anonymous on 18 Nov 2008

Ive installed the 1.4 the latest one and follow the steps too, but still no use.

by head on 18 Nov 2008

i have trouble cachevideos
ready config youtube_cache.conf
no use iptables
disabled selinux
use centos 5.2 squid 2.6 (squid-arp)
have access http://10.0.0.6/video_cache/

Index of /video_cache

Icon  Name                    Last modified      Size  Description[DIR] Parent Directory   -   
[DIR] dailymotion/            15-Nov-2008 17:18    -   
[DIR] google/                 15-Nov-2008 17:18    -   
[DIR] metacafe/               15-Nov-2008 17:18    -   
[DIR] redtube/                15-Nov-2008 17:18    -   
[DIR] tmp/                    16-Nov-2008 13:00    -   
[DIR] vimeo/                  15-Nov-2008 17:18    -   
[DIR] wrzuta/                 15-Nov-2008 17:18    -   
[DIR] xtube/                  15-Nov-2008 17:18    -   
[DIR] youtube/                15-Nov-2008 18:58    -   

Apache/2.2.3 (CentOS) Server at apache Port 80

008-11-15 17:52:36,018 INFO - - SCHEDULEDER - Download Scheduler starting.
2008-11-15 17:52:36,061 INFO - - SCHEDULEDER - Download Scheduler starting.
2008-11-15 17:52:40,976 INFO 10.0.0.15 - REQUEST - http://dal-v145.dal.youtube.com/get_video?video_id=SCaWLaimDCQ&ip=201.230.102.164&signature=5E4C851E0C429CBAF66968B23FB982D590BFEB20.94C60C8358F637EAE862AE220D0F456597413986&sver=2&expire=1226810704&key=yt4&ipbits=2
2008-11-15 17:52:40,984 INFO 10.0.0.15 SCaWLaimDCQ URL_HIT YOUTUBE http://dal-v145.dal.youtube.com/get_video?video_id=SCaWLaimDCQ&ip=201.230.102.164&signature=5E4C851E0C429CBAF66968B23FB982D590BFEB20.94C60C8358F637EAE862AE220D0F456597413986&sver=2&expire=1226810704&key=yt4&ipbits=2
2008-11-15 17:52:40,985 INFO 10.0.0.15 SCaWLaimDCQ CACHE_MISS YOUTUBE Requested video was not found in cache.2008-11-15 17:52:36,018 INFO - - SCHEDULEDER - Download Scheduler starting.
2008-11-15 17:52:36,061 INFO - - SCHEDULEDER - Download Scheduler starting.
2008-11-15 17:52:40,976 INFO 10.0.0.15 - REQUEST - http://dal-v145.dal.youtube.com/get_video?video_id=SCaWLaimDCQ&ip=201.230.102.164&signature=5E4C851E0C429CBAF66968B23FB982D590BFEB20.94C60C8358F637EAE862AE220D0F456597413986&sver=2&expire=1226810704&key=yt4&ipbits=2
2008-11-15 17:52:40,984 INFO 10.0.0.15 SCaWLaimDCQ URL_HIT YOUTUBE http://dal-v145.dal.youtube.com/get_video?video_id=SCaWLaimDCQ&ip=201.230.102.164&signature=5E4C851E0C429CBAF66968B23FB982D590BFEB20.94C60C8358F637EAE862AE220D0F456597413986&sver=2&expire=1226810704&key=yt4&ipbits=2
2008-11-15 17:52:40,985 INFO 10.0.0.15 SCaWLaimDCQ CACHE_MISS YOUTUBE Requested video was not found in cache.
by Anonymous on 18 Nov 2008

well here is my youtubecache.log

2008-11-18 11:14:36,741 INFO 10.100.39.129 f7U6x5It5Fo NEW_URL YOUTUBE http://nyc-v312.nyc.youtube.com/get_video?video_id=f7U6x5It5Fo&ip=202.209.30.100.2&signature=9904A66792A194084C6510E43B4D70AD77DEA31D.9354D4CB676F0F918A79BFF80DED6437AD96661C&sver=2&expire=1227008816&key=yt4&ipbits=0
2008-11-18 11:14:39,676 INFO 10.100.39.129 f7U6x5It5Fo SCHEDULED YOUTUBE Video scheduled for download.
2008-11-18 11:14:39,703 INFO 10.100.39.129 f7U6x5It5Fo DOWNLOAD_ERR YOUTUBE An error occured while retrieving the video.
2008-11-18 11:14:50,150 INFO 10.100.9.88 - REQUEST - http://www.youtube.com/get_video?video_id=1DMUO419RZg&t=OEgsToPDskI_bdV21EWPgk0JkECkgXrf&el=detailpage&ps=
2008-11-18 11:14:50,153 INFO 10.100.9.88 1DMUO419RZg URL_HIT YOUTUBE http://www.youtube.com/get_video?video_id=1DMUO419RZg&t=OEgsToPDskI_bdV21EWPgk0JkECkgXrf&el=detailpage&ps=
2008-11-18 11:14:50,153 INFO 10.100.9.88 1DMUO419RZg CACHE_MISS YOUTUBE Requested video was not found in cache.
2008-11-18 11:14:50,155 INFO 10.100.9.88 1DMUO419RZg NEW_URL YOUTUBE http://www.youtube.com/get_video?video_id=1DMUO419RZg&t=OEgsToPDskI_bdV21EWPgk0JkECkgXrf&el=detailpage&ps=
2008-11-18 11:14:50,730 INFO 10.100.9.88 - REQUEST - http://v22.cache.googlevideo.com/get_video?origin=lax-v56.lax.youtube.com&video_id=1DMUO419RZg&ip=202.209.30.100.2&signature=048FA43B099FC5A00A31B3F87CB4A25FFB5AE0A0.5A002C9943385E9E08C263077D1AF8E4FC15A559&sver=2&expire=1227008830&key=yt4&ipbits=0
2008-11-18 11:14:51,688 INFO 10.100.9.88 1DMUO419RZg SCHEDULED YOUTUBE Video scheduled for download.
2008-11-18 11:14:51,715 INFO 10.100.9.88 1DMUO419RZg DOWNLOAD_ERR YOUTUBE An error occured while retrieving the video.
2008-11-18 11:15:02,988 INFO 10.100.9.88 - REQUEST - http://www.youtube.com/get_video?video_id=gCf6E1K4x8I&t=OEgsToPDskISrl-LSnbtIULDxwPs7BFM&el=detailpage&ps=
2008-11-18 11:15:02,992 INFO 10.100.9.88 gCf6E1K4x8I URL_HIT YOUTUBE http://www.youtube.com/get_video?video_id=gCf6E1K4x8I&t=OEgsToPDskISrl-LSnbtIULDxwPs7BFM&el=detailpage&ps=
2008-11-18 11:15:02,992 INFO 10.100.9.88 gCf6E1K4x8I CACHE_MISS YOUTUBE Requested video was not found in cache.
2008-11-18 11:15:02,994 INFO 10.100.9.88 gCf6E1K4x8I NEW_URL YOUTUBE http://www.youtube.com/get_video?video_id=gCf6E1K4x8I&t=OEgsToPDskISrl-LSnbtIULDxwPs7BFM&el=detailpage&ps=
2008-11-18 11:15:03,556 INFO 10.100.9.88 - REQUEST - http://dal-v124.dal.youtube.com/get_video?video_id=gCf6E1K4x8I&ip=202.209.30.100.2&signature=1C09FCF2B250C7BEB221DA6786CF0111340CBE08.ADFFEC59202CF9B1A97638FE4D2ACE5A27D72B7C&sver=2&expire=1227008843&key=yt4&ipbits=0
2008-11-18 11:15:03,713 INFO 10.100.9.88 gCf6E1K4x8I SCHEDULED YOUTUBE Video scheduled for download.
2008-11-18 11:15:03,742 INFO 10.100.9.88 gCf6E1K4x8I DOWNLOAD_ERR YOUTUBE An error occured while retrieving the video.
by Kulbir Saini on 18 Nov 2008

Can you paste the following files at pastebin.com .

  1. /etc/youtube_cache.conf
  2. /etc/httpd/conf.d/youtube_cache.conf
  3. /etc/squid/squid.conf

Also, mention your operating system. And how did you install from rpm or source?

by Anonymous on 18 Nov 2008

I have installed it via rpm and i could not fine any file name youtube_cache.conf in /etc/httpd/conf/ .

youtube_cache.conf
http://pastebin.com/m2b70f3a2

Squid.con
http://pastebin.com/m4a948566

by Kulbir Saini on 18 Nov 2008

In your youtube_cache.conf file, following things are wrong

1.proxy = http://10.100.x.x:80/

It should be proxy = http://10.100.50.1:8080/ Why are you putting 80 as port when your proxy is running on port 8080??

2.cache_host = 10.100.x.x

It should be cach_host = 10.100.50.1 I don't know why are you putting x there?

3.youtube_cache_dir = /var/spool/squid/vide0_cache/youtube

Did you read the description written in comments? It should be youtube_cache_dir = youtube .

Fix these options in /etc/youtube_cache.conf file.

Your squid.conf file is fine. No problems in there. If you read my reply carefully, the location for the other youtube_cache.conf is /etc/httpd/conf.d/ and not /etc/httpd/conf/ .

by Anonymous on 19 Nov 2008

Thankzzz but now ive only one problem that is forbidden in browser when accessing http://10.100.50.1/video_chat. Ive removed all directory from /var/www/html/video_chat and recreated it .

by Anonymous on 20 Nov 2008

opps it is video_cache not video_chat

by Kulbir Saini on 20 Nov 2008

Do this steps and you'll be able to access it.

[root@localhost ~]# chmod 755 /var/spool/squid [ENTER]

Let me know if there is some problem.

by Anonymous on 21 Nov 2008

I think there was a issue with apache permission , well ive removed the old package and install the 1.5. 1.5 has different path for that fixes other bugs aswell but thanks for help Kulbir Saini . I've little thing to ask is that once ive .flv in /var/spool/youtube_cache/youtube how to move them under apache. if is file to create softlink ?? or I would better to creat a folder for that ?? manualy in apache /var/www/html/ .

Regards
Net_Spy

by Kulbir Saini on 21 Nov 2008

Is your youtube_cache working fine now? Actually there is no need to move files or create soft links. All the cached videos are already available at http://your_ip/video_cache/ . In case you want youtube files to be available in /var/www/html/, you can create a soft link like

ln -s /var/spool/video_cache/youtube /var/www/html/youtube/

But I'll recommend not to do so.

by Anonymous on 21 Nov 2008

thanks I got few .flv file in youtube folder

I did tail -f /var/log/youtube_cache/youtube.log to check what is going on I found this

NEW_URL YOUTUBE Error in parsing the url http://www.youtube.com/get_video?video_id=jvwmA3N5HM4&t=OEgsToPDskJXl4CpE5uTETnJ6pTe_Yp8&el=detailpage&ps=
NEW_URL YOUTUBE Error in parsing the url http://v20.cache.googlevideo.com/videoplayback?id=d4771de2bca34be5&itag=18&begin=0&len=604800000&ip=202.30.111.33&signature=48973E5D8D3B23137B687AD146C09A5C3D947D83.B7F214D0A9192734CBA47FC429FB25E89957240D&sver=2&expire=1227292249&key=yt4&ipbits=0

like this for other urls of youtube. what if I increase the
max_parallel_downloads = 30 to 400 .

Regards
Net_Spy

by Kulbir Saini on 21 Nov 2008

Most probably your XMLRPC Server is not running or it has exited abnormally. Can you give me the output of command

[root@localhost ~]# netstat -atpn |grep python [ENTER]

If no app is listening on port 9100, then check /var/log/squid/cache.log for any errors related to python tracebacks. Get back with whatever result you get.

Don't increase max_parallel_downloads too much. Every parallel download will be one python process.

by Anonymous on 21 Nov 2008

here is the out put

tcp        0      0 127.0.0.1:9100              0.0.0.0:*                   LISTEN      19865/(python)
tcp        0      0 127.0.0.1:48530             127.0.0.1:9100              ESTABLISHED 19945/(python)
tcp        0      0 127.0.0.1:48531             127.0.0.1:9100              ESTABLISHED 19942/(python)
tcp        0      0 127.0.0.1:48528             127.0.0.1:9100              ESTABLISHED 19936/(python)
tcp        0      0 127.0.0.1:48529             127.0.0.1:9100              ESTABLISHED 19939/(python)
tcp        0      0 127.0.0.1:48526             127.0.0.1:9100              ESTABLISHED 19883/(python)
tcp        0      0 127.0.0.1:48527             127.0.0.1:9100              ESTABLISHED 19865/(python)
tcp        0    242 127.0.0.1:46443             127.0.0.1:9100              ESTABLISHED 20161/(python)
tcp        0    242 127.0.0.1:46440             127.0.0.1:9100              ESTABLISHED 20159/(python)
tcp        0    242 127.0.0.1:46441             127.0.0.1:9100              ESTABLISHED 20160/(python)
tcp        0    242 127.0.0.1:46446             127.0.0.1:9100              ESTABLISHED 20164/(python)
tcp        0    242 127.0.0.1:46444             127.0.0.1:9100              ESTABLISHED 20162/(python)
tcp        0    242 127.0.0.1:46445             127.0.0.1:9100              ESTABLISHED 20163/(python)
tcp        0    242 127.0.0.1:46438             127.0.0.1:9100              ESTABLISHED 20157/(python)
tcp        0    242 127.0.0.1:46439             127.0.0.1:9100              ESTABLISHED 20158/(python)
tcp        0    242 127.0.0.1:46437             127.0.0.1:9100              ESTABLISHED 20156/(python)

I set the max_parallel download to 50 .

Regards

by Kulbir Saini on 21 Nov 2008

This is not really a normal behavior. Can you contact me via email using the contact tab above?

by Anonymous on 21 Nov 2008

well another problem is that ive configured new package in moring at 11am and still I found old 8 files since moring and its now 10:09pm the files are same in youtube folder and rest of folders are empty.

Regards
Net_Spy

by Anonymous on 27 Dec 2008

I am with this issue, resolved?

DOWNLOAD_ERR YOUTUBE An error occured while retrieving the video.

thanks

by Kulbir Saini on 27 Dec 2008

Hi!

Please provide further details from your /var/log/videocache/videocache.log, /var/log/squid/cache.log and /etc/videocache.conf. Any tracebacks or errors will help.

Thank you for using videocache :)

by Anonymous on 27 Dec 2008

I'm using the directory /cache/videocache for the cache, in the log (error.log) of apache is giving the following message

"File does not exist: /cache/videocache/get_video"

my apache is configured, the browser (http:///videocache) I see the folders with the directories, and even are in the tmp files .flv, but size 0.

sorry for my English.

by Kulbir Saini on 28 Dec 2008

Hi!

Did you run update-vc script after changing the base_dir in /etc/videocache.conf?

The update-vc script must be run every time you change something in /etc/videocache.conf.

Please get back if problem persists.

Thank you for using videocache :)

by Anonymous on 2 Jan 2009

SOLVED! I'm caching!

was missing in the configuration of "proxy = " videocache.conf

thanks!

by Kulbir Saini on 2 Jan 2009

Hello,

Congratulations for the setup :)

Thank you for using videocache.

You need to sign in. Please sign in to add answer to this question.