VideoCache

regarding strange download problem videocache

by Anonymous on 14 Aug 2009

Hi,
I have installed videocache 1.9.1 on freebsd-7.2 with Squid Version 2.7.STABLE6

Earlier i was not getting any single request in /var/log/videocache/videocache.log when i start youtube.com video and after applying following i m able to get request but not getting any other in log.
I made two changes
1) added squid.conf file

acl videocache_allow_url url_regex -i \\.youtube\\.com\\/videoplayback

2)changes in videocache.py

2a)

if host.find('.youtube.com') > -1 and path.find('get_video') > -1 or  path.find('videoplayback') and path.find('get_video_info') < 0:

2b)

elif dict.has_key('id'):
       video_id = dict['id']

Now I am able to get request but caching and downloading is not happening.

My server is directly connected on public ip and able to reach internet without any issue. i dont know what could be the issue

plz find my configuration file.
serverip = 192.168.101.1

Squid.conf file:

192.168.101.1:3128
acl LAN_1 src squidip
http_access allow LAN_1

/etc/videocache.conf file:

cache_host = 192.168.101.1
proxy = http://192.168.101.1:3128/
rpc_host = 127.0.0.1
rpc_port = 9100

httpd.conf file

ServerName=192.168.101.1:80

My log file when i do youtube

[root@freebsd64 /var/spool/videocache/youtube]# tail -f /var/log/videocache/videocache.log 
2009-08-14 16:45:54,080 68802 - - RELOAD - videocache plugin was reloaded.
2009-08-14 16:45:54,083 68807 - - RELOAD - videocache plugin was reloaded.
2009-08-14 16:45:54,082 68805 - - RELOAD - videocache plugin was reloaded.
2009-08-14 16:45:54,083 68806 - - RELOAD - videocache plugin was reloaded.
2009-08-14 16:45:54,082 68804 - - RELOAD - videocache plugin was reloaded.
2009-08-14 16:46:07,556 68845 - - SCHEDULEDER - Download Scheduler starting.
2009-08-14 16:46:07,562 68845 - - XMLRPCSERVER - Starting XMLRPCServer on port 9100.
2009-08-14 16:46:09,594 68853 - - UPDATE_SIZE - Size of all caching directories updated successfully.
2009-08-14 16:46:42,305 68844 172.16.2.64 - REQUEST - http://v12.lscache5.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Cburst%2Cfactor&itag=34&ipbits=0&signature=56DABC44417743E5AE4DDC6DE3E358B0C9615368.5EF76CE61C5DBE516B0318250BC3C3219F92C561&sver=3&expire=1250272800&key=yt1&factor=1.25&burst=40&id=09a54519720bc50b
2009-08-14 16:47:49,056 68844 172.16.2.64 - REQUEST - http://v9.lscache5.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Cburst%2Cfactor&itag=34&ipbits=0&signature=4622DAF4AE69FDF53A4E684F306C71615668F290.AD38C04A7288D3042B46B0E981125FBF372AD366&sver=3&expire=1250272800&key=yt1&factor=1.25&burst=40&id=bb71a7a77b339635

Log file when i do http://video.google.com

[root@freebsd64 /var/spool/videocache/youtube]# tail -f /var/log/videocache/videocache.log 
2009-08-14 17:12:09,784 68844 172.16.2.64 - REQUEST - http://video.google.com/stats?ev=pvfull&tbt=1&len=63&docid=1811233136844420765&q=&start=0&num=12&so=0&srcdocid=&srcurl=http%3A%2F%2Fvideo.google.com%2Fvideoplay%3Fdocid%3D1811233136844420765&plindex=10&esrc=hs&view=hs
2009-08-14 17:12:09,784 68844 172.16.2.64 - URL_ERROR YOUTUBE video_id not found in http://video.google.com/stats?ev=pvfull&tbt=1&len=63&docid=1811233136844420765&q=&start=0&num=12&so=0&srcdocid=&srcurl=http%3A%2F%2Fvideo.google.com%2Fvideoplay%3Fdocid%3D1811233136844420765&plindex=10&esrc=hs&view=hs
2009-08-14 17:12:09,830 68844 172.16.2.64 - REQUEST - http://video.google.com/s/PefnNldGf5U/googleplayer.swf?videoUrl=http://v12.lscache5.googlevideo.com/videoplayback%3Fid%3Dd53e613f82f74c96%26itag%3D5%26begin%3D0%26ip%3D0.0.0.0%26ipbits%3D0%26expire%3D1250271619%26sparams%3Dip,ipbits,expire,id,itag%26signature%3D28468DC9352D054A9CB1B27EE2E0FFA777BFEC47.482AB258DC03C89343CF776E82CD10B063121C71%26key%3Dck1&thumbnailUrl=http://2.gvt0.com/ThumbnailServer2%3Fapp%3Dvss%26contentid%3Dd53e613f82f74c96%26offsetms%3D5000%26itag%3Dw160%26hl%3Den%26sigh%3DxKI2Sc9kJNm9zd-F_hEmrqyx6Q0&docid=1811233136844420765&hl=en&autoplay=1&enablejsapi=1&playerapiid=embed_player_3
2009-08-14 17:12:09,830 68844 172.16.2.64 - URL_ERROR YOUTUBE video_id not found in http://video.google.com/s/PefnNldGf5U/googleplayer.swf?videoUrl=http://v12.lscache5.googlevideo.com/videoplayback%3Fid%3Dd53e613f82f74c96%26itag%3D5%26begin%3D0%26ip%3D0.0.0.0%26ipbits%3D0%26expire%3D1250271619%26sparams%3Dip,ipbits,expire,id,itag%26signature%3D28468DC9352D054A9CB1B27EE2E0FFA777BFEC47.482AB258DC03C89343CF776E82CD10B063121C71%26key%3Dck1&thumbnailUrl=http://2.gvt0.com/ThumbnailServer2%3Fapp%3Dvss%26contentid%3Dd53e613f82f74c96%26offsetms%3D5000%26itag%3Dw160%26hl%3Den%26sigh%3DxKI2Sc9kJNm9zd-F_hEmrqyx6Q0&docid=1811233136844420765&hl=en&autoplay=1&enablejsapi=1&playerapiid=embed_player_3
2009-08-14 17:12:11,062 68844 172.16.2.64 - REQUEST - http://v12.lscache5.googlevideo.com/videoplayback?id=d53e613f82f74c96&itag=5&begin=0&ip=0.0.0.0&ipbits=0&expire=1250271619&sparams=ip,ipbits,expire,id,itag&signature=28468DC9352D054A9CB1B27EE2E0FFA777BFEC47.482AB258DC03C89343CF776E82CD10B063121C71&key=ck1
2009-08-14 17:12:11,062 68844 172.16.2.64 d53e613f82f74c96 URL_HIT YOUTUBE http://v12.lscache5.googlevideo.com/videoplayback?id=d53e613f82f74c96&itag=5&begin=0&ip=0.0.0.0&ipbits=0&expire=1250271619&sparams=ip,ipbits,expire,id,itag&signature=28468DC9352D054A9CB1B27EE2E0FFA777BFEC47.482AB258DC03C89343CF776E82CD10B063121C71&key=ck1
2009-08-14 17:12:11,063 68844 172.16.2.64 d53e613f82f74c96 CACHE_MISS YOUTUBE Requested video was not found in cache.
2009-08-14 17:12:11,067 68844 172.16.2.64 d53e613f82f74c96 NEW_URL YOUTUBE http://v12.lscache5.googlevideo.com/videoplayback?id=d53e613f82f74c96&itag=5&begin=0&ip=0.0.0.0&ipbits=0&expire=1250271619&sparams=ip,ipbits,expire,id,itag&signature=28468DC9352D054A9CB1B27EE2E0FFA777BFEC47.482AB258DC03C89343CF776E82CD10B063121C71&key=ck1
2009-08-14 17:12:11,525 68845 172.16.2.64 d53e613f82f74c96 SCHEDULED YOUTUBE Video scheduled for download.
2009-08-14 17:12:11,631 68845 - - SCHEDULED_ERR - Could not schedule video for download.

Even i am able to start all service squid,http,rpc

tcp4       0      0 192.168.101.1.3128   *.*                    LISTEN
tcp4       0      0 127.0.0.1.9100         *.*                    LISTEN
tcp46      0      0 *.80                   *.*                    LISTEN

I am behind on this since last two day but no clue on this.can anybody help on this.

Harindra.

5 Answers

by nilhari on 14 Aug 2009

Hi,
I have installed videocache 1.9.1 on freebsd-7.2 with Squid Version 2.7.STABLE6

Earlier i was not getting any single request in /var/log/videocache/videocache.log when i start youtube.com video and after applying following i m able to get request but not getting any other in log.
I made two changes
1) added squid.conf file

acl videocache_allow_url url_regex -i \\.youtube\\.com\\/videoplayback

2)changes in videocache.py

2a)

if host.find('.youtube.com') > -1 and path.find('get_video') > -1 or path.find('videoplayback') and path.find('get_video_info') < 0:

2b)

elif dict.has_key('id'):
       video_id = dict['id']

Now I am able to get request but caching and downloading is not happening.

My server is directly connected on public ip and able to reach internet without any issue. i dont know what could be the issue

plz find my configuration file.
serverip = 192.168.101.1

Squid.conf file:
192.168.101.1:3128

acl LAN_1 src squidip
http_access allow LAN_1

/etc/videocache.conf file:

cache_host = 192.168.101.1
proxy = http://192.168.101.1:3128/
rpc_host = 127.0.0.1
rpc_port = 9100

httpd.conf file

ServerName=192.168.101.1:80

My log file when i do youtube

[root@freebsd64 /var/spool/videocache/youtube]# tail -f /var/log/videocache/videocache.log 
2009-08-14 16:45:54,080 68802 - - RELOAD - videocache plugin was reloaded.
2009-08-14 16:45:54,083 68807 - - RELOAD - videocache plugin was reloaded.
2009-08-14 16:45:54,082 68805 - - RELOAD - videocache plugin was reloaded.
2009-08-14 16:45:54,083 68806 - - RELOAD - videocache plugin was reloaded.
2009-08-14 16:45:54,082 68804 - - RELOAD - videocache plugin was reloaded.
2009-08-14 16:46:07,556 68845 - - SCHEDULEDER - Download Scheduler starting.
2009-08-14 16:46:07,562 68845 - - XMLRPCSERVER - Starting XMLRPCServer on port 9100.
2009-08-14 16:46:09,594 68853 - - UPDATE_SIZE - Size of all caching directories updated successfully.
2009-08-14 16:46:42,305 68844 172.16.2.64 - REQUEST - http://v12.lscache5.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Cburst%2Cfactor&itag=34&ipbits=0&signature=56DABC44417743E5AE4DDC6DE3E358B0C9615368.5EF76CE61C5DBE516B0318250BC3C3219F92C561&sver=3&expire=1250272800&key=yt1&factor=1.25&burst=40&id=09a54519720bc50b
2009-08-14 16:47:49,056 68844 172.16.2.64 - REQUEST - http://v9.lscache5.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Cburst%2Cfactor&itag=34&ipbits=0&signature=4622DAF4AE69FDF53A4E684F306C71615668F290.AD38C04A7288D3042B46B0E981125FBF372AD366&sver=3&expire=1250272800&key=yt1&factor=1.25&burst=40&id=bb71a7a77b339635

Log file when i do http://video.google.com

[root@freebsd64 /var/spool/videocache/youtube]# tail -f /var/log/videocache/videocache.log 
2009-08-14 17:12:09,784 68844 172.16.2.64 - REQUEST - http://video.google.com/stats?ev=pvfull&tbt=1&len=63&docid=1811233136844420765&q=&start=0&num=12&so=0&srcdocid=&srcurl=http%3A%2F%2Fvideo.google.com%2Fvideoplay%3Fdocid%3D1811233136844420765&plindex=10&esrc=hs&view=hs
2009-08-14 17:12:09,784 68844 172.16.2.64 - URL_ERROR YOUTUBE video_id not found in http://video.google.com/stats?ev=pvfull&tbt=1&len=63&docid=1811233136844420765&q=&start=0&num=12&so=0&srcdocid=&srcurl=http%3A%2F%2Fvideo.google.com%2Fvideoplay%3Fdocid%3D1811233136844420765&plindex=10&esrc=hs&view=hs
2009-08-14 17:12:09,830 68844 172.16.2.64 - REQUEST - http://video.google.com/s/PefnNldGf5U/googleplayer.swf?videoUrl=http://v12.lscache5.googlevideo.com/videoplayback%3Fid%3Dd53e613f82f74c96%26itag%3D5%26begin%3D0%26ip%3D0.0.0.0%26ipbits%3D0%26expire%3D1250271619%26sparams%3Dip,ipbits,expire,id,itag%26signature%3D28468DC9352D054A9CB1B27EE2E0FFA777BFEC47.482AB258DC03C89343CF776E82CD10B063121C71%26key%3Dck1&thumbnailUrl=http://2.gvt0.com/ThumbnailServer2%3Fapp%3Dvss%26contentid%3Dd53e613f82f74c96%26offsetms%3D5000%26itag%3Dw160%26hl%3Den%26sigh%3DxKI2Sc9kJNm9zd-F_hEmrqyx6Q0&docid=1811233136844420765&hl=en&autoplay=1&enablejsapi=1&playerapiid=embed_player_3
2009-08-14 17:12:09,830 68844 172.16.2.64 - URL_ERROR YOUTUBE video_id not found in http://video.google.com/s/PefnNldGf5U/googleplayer.swf?videoUrl=http://v12.lscache5.googlevideo.com/videoplayback%3Fid%3Dd53e613f82f74c96%26itag%3D5%26begin%3D0%26ip%3D0.0.0.0%26ipbits%3D0%26expire%3D1250271619%26sparams%3Dip,ipbits,expire,id,itag%26signature%3D28468DC9352D054A9CB1B27EE2E0FFA777BFEC47.482AB258DC03C89343CF776E82CD10B063121C71%26key%3Dck1&thumbnailUrl=http://2.gvt0.com/ThumbnailServer2%3Fapp%3Dvss%26contentid%3Dd53e613f82f74c96%26offsetms%3D5000%26itag%3Dw160%26hl%3Den%26sigh%3DxKI2Sc9kJNm9zd-F_hEmrqyx6Q0&docid=1811233136844420765&hl=en&autoplay=1&enablejsapi=1&playerapiid=embed_player_3
2009-08-14 17:12:11,062 68844 172.16.2.64 - REQUEST - http://v12.lscache5.googlevideo.com/videoplayback?id=d53e613f82f74c96&itag=5&begin=0&ip=0.0.0.0&ipbits=0&expire=1250271619&sparams=ip,ipbits,expire,id,itag&signature=28468DC9352D054A9CB1B27EE2E0FFA777BFEC47.482AB258DC03C89343CF776E82CD10B063121C71&key=ck1
2009-08-14 17:12:11,062 68844 172.16.2.64 d53e613f82f74c96 URL_HIT YOUTUBE http://v12.lscache5.googlevideo.com/videoplayback?id=d53e613f82f74c96&itag=5&begin=0&ip=0.0.0.0&ipbits=0&expire=1250271619&sparams=ip,ipbits,expire,id,itag&signature=28468DC9352D054A9CB1B27EE2E0FFA777BFEC47.482AB258DC03C89343CF776E82CD10B063121C71&key=ck1
2009-08-14 17:12:11,063 68844 172.16.2.64 d53e613f82f74c96 CACHE_MISS YOUTUBE Requested video was not found in cache.
2009-08-14 17:12:11,067 68844 172.16.2.64 d53e613f82f74c96 NEW_URL YOUTUBE http://v12.lscache5.googlevideo.com/videoplayback?id=d53e613f82f74c96&itag=5&begin=0&ip=0.0.0.0&ipbits=0&expire=1250271619&sparams=ip,ipbits,expire,id,itag&signature=28468DC9352D054A9CB1B27EE2E0FFA777BFEC47.482AB258DC03C89343CF776E82CD10B063121C71&key=ck1
2009-08-14 17:12:11,525 68845 172.16.2.64 d53e613f82f74c96 SCHEDULED YOUTUBE Video scheduled for download.
2009-08-14 17:12:11,631 68845 - - SCHEDULED_ERR - Could not schedule video for download.

Even i am able to start all service squid,http,rpc

tcp4       0      0 192.168.101.1.3128   *.*                    LISTEN
tcp4       0      0 127.0.0.1.9100         *.*                    LISTEN
tcp46      0      0 *.80                   *.*                    LISTEN

I am behind on this since last two day but no clue on this.can anybody help on this.

Harindra.

by mscandian on 25 Aug 2009

Hi Harindra,

I'm using FreeBSD too. I'm using:

FreeBSD 7.2-RELEASE
squid-2.7.6_1
python25-2.5.4_2
py25-iniparse-0.3.1
py25-urlgrabber-3.1.0_1

And I made the changes that you saw.

After this, I get only REQUEST and no download/cache video.

2009-08-25 15:15:22,753 4548 192.168.16.2 - REQUEST - http://v14.lscache2.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Cburst%2Cfactor&itag=34&ipbits=0&signature=6695ED2BDCDBE5C7E315E60930B29569F0496197.0BFD84FBE1857BD48469AF9A7C0BBFB31983981E&sver=3&expire=1251248400&key=yt1&factor=1.25&burst=40&id=aa587b27ce56f142
2009-08-25 15:15:24,086 4548 192.168.16.2 - REQUEST - http://v6.cache5.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Cburst%2Cfactor&itag=34&ipbits=0&signature=6695ED2BDCDBE5C7E315E60930B29569F0496197.0BFD84FBE1857BD48469AF9A7C0BBFB31983981E&sver=3&expire=1251248400&key=yt1&factor=1.25&burst=40&id=aa587b27ce56f142&redirect_counter=1

Do you fix this problem in your server?

Regards,
MS

by Harindra on 26 Aug 2009

We done the following and its working fine.try this one

 # Check if videocache plugin is on.
        if enable_video_cache:
            # Youtube.com caching is handled here.
            if enable_youtube_cache:
              # if host.find('.youtube.com') > -1 and path.find('get_video') > -1 and path.find('get_video_info') < 0:
                if host.find('.youtube.com') > -1 and path.find('get_video') > -1 or path.find('videoplayback') and path.find('get_video_info') < 0:
                    type = 'YOUTUBE'
                    arglist = params.split('&')
                    dict = {}
                    for arg in arglist:
                        try:
                            dict[arg.split('=')[0]] = arg.split('=')[1]
                        except:
                            continue
                    if dict.has_key('video_id'):
                        video_id = dict['video_id']
                        new_url = submit_video(pid, client, type, url, video_id)
                      #  ADDED 2 Lines HARINDRA
                    elif dict.has_key('id'):
                        video_id = dict['id']
                        new_url = submit_video(pid, client, type, url, video_id)
                    else:
                        log(format%(pid, client, '-', 'URL_ERROR', type, 'video_id not found in ' + new_url))

            # Youtube videos served via cache.googlevideo.com are handled here.
by mscandian on 26 Aug 2009

Hi Harindra,

Ok, I change my videocache.py using this settings. Now, I'm getting this in videocache.log:

2009-08-26 10:24:35,263 7801 192.168.16.2 aa587b27ce56f142 SCHEDULED YOUTUBE Video scheduled for download.
2009-08-26 10:24:35,376 7818 192.168.16.2 aa587b27ce56f142 DOWNLOAD_ERR YOUTUBE An error occured while retrieving the video.

I have two interfaces:

bge0 = 201.45.X.Y
bge1 = 192.168.16.1

In my videocache.conf I have this:

enable_video_cache = 1
cache_host = 127.0.0.1
proxy = http://127.0.0.1:3128/
rpc_host = 127.0.0.1
rpc_port = 9100
pf2# sockstat -4l
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS
squid    python     7801  4  tcp4   127.0.0.1:9100        *:*
squid    squid      7764  8  udp4   *:55644               *:*
squid    squid      7764  19 tcp4   *:3128                *:*
squid    squid      7764  21 udp4   *:3130                *:*

My /var/spool/videocache has squid:squid permissions.

What's wrong with my configuration?

Regards,
MSC

by G on 13 Sep 2009

Hello.
I'm downloading from youtube, using "Keepvid.com" and after a certain time or percentage,the download stops and when i resume it it's popping a msg "authentication required" site:v6.cache5.c.youtube.com, username and password.this time it stopped downloading at the 99% so please any help.I'm downloading Daughter From Danang.
Regards
Thanks.

You need to sign in. Please sign in to add answer to this question.