SCHEDULED_ERR strange problems on debian 5.3, log provided
by Anonymous on 23 Sep 2009Hello all
VideoCache seems to be a good piece of software, however i've run into some trouble trying to use it. I have installed a brand new copy of debian 5 for this. Here is a complete log showing my problem:
2009-09-24 11:34:39,243 1720 - - XMLRPCSERVER - Starting XMLRPCServer on port 9100.
2009-09-24 11:34:39,246 1720 - - SCHEDULEDER - Download Scheduler starting.
2009-09-24 11:35:08,153 1717 192.168.1.105 - REQUEST - http://video.google.com/videoplay?docid=-572077907195969915&hl=en&emb=1
2009-09-24 11:35:08,156 1717 192.168.1.105 -572077907195969915 URL_HIT YOUTUBE http://video.google.com/videoplay?docid=-572077907195969915&hl=en&emb=1
2009-09-24 11:35:08,161 1717 192.168.1.105 -572077907195969915 CACHE_MISS YOUTUBE Requested video was not found in cache.
2009-09-24 11:35:08,190 1717 192.168.1.105 -572077907195969915 NEW_URL YOUTUBE http://video.google.com/videoplay?docid=-572077907195969915&hl=en&emb=1
2009-09-24 11:35:10,424 1717 192.168.1.105 - REQUEST - http://video.google.com/videoplay?docid=-572077907195969915&hl=en&output=json&title_len=128&desc_len=1024
2009-09-24 11:35:10,427 1717 192.168.1.105 -572077907195969915 URL_HIT YOUTUBE http://video.google.com/videoplay?docid=-572077907195969915&hl=en&output=json&title_len=128&desc_len=1024
2009-09-24 11:35:10,431 1717 192.168.1.105 -572077907195969915 CACHE_MISS YOUTUBE Requested video was not found in cache.
2009-09-24 11:35:10,454 1717 192.168.1.105 -572077907195969915 NEW_URL YOUTUBE http://video.google.com/videoplay?docid=-572077907195969915&hl=en&output=json&title_len=128&desc_len=1024
2009-09-24 11:35:10,523 1720 - - SCHEDULED_ERR - Could not schedule video for download.
2009-09-24 11:35:10,528 1720 192.168.1.105 -572077907195969915 SCHEDULED YOUTUBE Video scheduled for download.
2009-09-24 11:35:11,441 1717 192.168.1.105 - REQUEST - http://video.google.com/s/rsBLQEkAzFs/googleplayer.swf?videoUrl=http://v9.lscache2.googlevideo.com/videoplayback%3Fid%3D20fd1f03f14f455d%26itag%3D5%26begin%3D0%26ip%3D0.0.0.0%26ipbits%3D0%26expire%3D1253741711%26sparams%3Dip,ipbits,expire,id,itag%26signature%3D5DC76AF7E157075FA939026DEEF24C2E08289ABA.7B0C4AD44AC6C4083174C4FD4E47F17B4255298D%26key%3Dck1&thumbnailUrl=http://1.gvt0.com/ThumbnailServer2%3Fapp%3Dvss%26contentid%3D20fd1f03f14f455d%26offsetms%3D25000%26itag%3Dw160%26hl%3Den%26sigh%3DV1JoX4lrUIghKK1SulMPs1Qu48s&docid=-572077907195969915&hl=en&speedcontrol=0&autoplay=1&enablejsapi=1&playerapiid=embed_player_1
2009-09-24 11:35:13,698 1717 192.168.1.105 - REQUEST - http://v9.lscache2.googlevideo.com/videoplayback?id=20fd1f03f14f455d&itag=5&begin=0&ip=0.0.0.0&ipbits=0&expire=1253741711&sparams=ip,ipbits,expire,id,itag&signature=5DC76AF7E157075FA939026DEEF24C2E08289ABA.7B0C4AD44AC6C4083174C4FD4E47F17B4255298D&key=ck1
2009-09-24 11:35:13,700 1717 192.168.1.105 20fd1f03f14f455d URL_HIT YOUTUBE http://v9.lscache2.googlevideo.com/videoplayback?id=20fd1f03f14f455d&itag=5&begin=0&ip=0.0.0.0&ipbits=0&expire=1253741711&sparams=ip,ipbits,expire,id,itag&signature=5DC76AF7E157075FA939026DEEF24C2E08289ABA.7B0C4AD44AC6C4083174C4FD4E47F17B4255298D&key=ck1
2009-09-24 11:35:13,703 1717 192.168.1.105 20fd1f03f14f455d CACHE_MISS YOUTUBE Requested video was not found in cache.
2009-09-24 11:35:13,727 1717 192.168.1.105 20fd1f03f14f455d NEW_URL YOUTUBE http://v9.lscache2.googlevideo.com/videoplayback?id=20fd1f03f14f455d&itag=5&begin=0&ip=0.0.0.0&ipbits=0&expire=1253741711&sparams=ip,ipbits,expire,id,itag&signature=5DC76AF7E157075FA939026DEEF24C2E08289ABA.7B0C4AD44AC6C4083174C4FD4E47F17B4255298D&key=ck1
2009-09-24 11:35:15,768 1720 - - SCHEDULED_ERR - Could not schedule video for download.
2009-09-24 11:35:15,775 1720 192.168.1.105 20fd1f03f14f455d SCHEDULED YOUTUBE Video scheduled for download.
As you can see, squid seems to be configured properly to use videocache. Videocache is giving a scheduled_err, then a scheduled message. My cache folder is empty. I am able to access the videocache folder via http://server-ip/videocache, however these are also empty.
I would love to get this software working soon, as bandwidth costs are relatively high here in australia than many places in europe, asia and america.
Thanks all
Fabs
16 Answers
I have found that by using a different cache dir it works. I realised that WHS network shares report free space incorrectly, and i assume that videocache is reading this and thinking that there is no space left on the device (a mounted smb share). My problem would be solved if I could change the code to not check for free space on the device, however I have no idea where to start.
Some background on WHS free space reporting: WHS uses pooled storage between drives, but reports the size of the entire pool over SMB as the size of the system drive only. As the data in the pool is larger than the size of the system drive alone, it reports that (eg) 2000GB/120GB is used. This must be what is causing problems with videocache.
Fabs,
Sorry for replying this late, I was busy with other projects. Apply the following patch to your /usr/share/videocache/videocache.py file to stop disk space checking.
diff --git a/videocache/videocache.py b/videocache/videocache.py
index 2806773..1c1f94e 100755
--- a/videocache/videocache.py
+++ b/videocache/videocache.py
@@ -410,24 +410,18 @@ def download_from_source(args):
continue
# Check the disk space left in the partition with cache directory.
- disk_stat = os.statvfs(cache_dir)
- disk_available = disk_stat[statvfs.F_BSIZE] * disk_stat[statvfs.F_BAVAIL] / (1024*1024.0)
- video_id_pool = ServerProxy('http://' + rpc_host + ':' + str(rpc_port))
- # If cache_size is not 0 and the cache directory size is more than cache_size, we are done with this cache directory.
- #if cache_size != 0 and video_id_pool.get_cache_dir_size(base_tup[0]) >= cache_size:
- if cache_size != 0:
- log(format%(pid, client, video_id, 'CACHE_FULL', type, 'Cache directory \\'' + base_tup[0] + '\\' has exceeded the maximum size allowed.'))
- # Check next cache directory
- continue
+ #disk_stat = os.statvfs(cache_dir)
+ #disk_available = disk_stat[statvfs.F_BSIZE] * disk_stat[statvfs.F_BAVAIL] / (1024*1024.0)
+ #video_id_pool = ServerProxy('http://' + rpc_host + ':' + str(rpc_port))
# If disk availability reached disk_avail_threshold, then we can't use this cache anymore.
- elif disk_available < disk_avail_threshold:
- log(format%(pid, client, video_id, 'CACHE_FULL', type, 'Cache directory \\'' + base_tup[0] + '\\' has reached the disk availability threshold.'))
- # Check next cache directory
- continue
- else:
- index = base_dir.index(base_tup)
- # Search complete. Just write wherever possible.
- break
+ #if disk_available < disk_avail_threshold:
+ # log(format%(pid, client, video_id, 'CACHE_FULL', type, 'Cache directory \\'' + base_tup[0] + '\\' has reached the disk availability threshold.'))
+ # # Check next cache directory
+ # continue
+ #else:
+ # index = base_dir.index(base_tup)
+ # # Search complete. Just write wherever possible.
+ # break
if index is not None:
(path, max_size, min_size, cache_size, cache_dir, tmp_cache) = video_params_all(base_dir[index], video_id, type, url, index)
I hope this helps.
Thank You!
Thanks for the fast reply with the patch. It now works pretty well.
I have a new problem which someone here might be able to help with, but it might be more of a general squid config issue... I have set up squid as a transparent proxy, but when used in this configuration videocache is not invoked at all. If anyone can help me to solve this final issue with my configuration, I will be very thankful.
I have to add that videocache is a great plugin, and caches a very large proportion of youtube videos nicely.
Thank you.
I've setup the videocache for my transparent proxy as well with freebsd and it was working fine for me. I'm not sure for debian but it definitely work in freebsd.
I'm using VC1.9.2, do I need to apply this patch if I'm encountered the said problem at the moment?
Thanks for reassuring me that it can work. I'll post on some squid mailing lists/forums soon to see if i can find the problem.
It all works perfectly now. My problem was that even though the DHCP server was sending clients two gateways with the linux squid cache as the first one, the windows clients decided to use the second one for some strange reason.
Thanks for help all.
CK Tan,
No. Don't apply the patch if you Videocache is running fine.
Thank You!
Fabs,
Thanks for posting the solution here. I hope it'll be of great help for others. And congratulations for the setup :) Enjoy caching :D
Thank You!
Hi,
I am novice in this subject.
I am not sure if this is the right place to post the request here.
Can you please let me know what kind of a hardware and operating system will be needed to cache the videos.
I am a cable tv operator and internet service provider, we are planning to cache in the videos and are wondering what kind of a hardware can handle would be better to handle this without choking itself.
Please do let me know
Arun
hi dear
this time i am facing a problem
DOWNLOAD_ERR YOUTUBE An error occured while retrieving the video
on centos 5.3 ?
Arun,
Videocache doesn't use much resources other than bandwidth and hard disk. You can get a machine with 1 or 2GB RAM if you have something like 500 customers. You can deploy hard disk depending on your bandwidth. A 500G or 1TB hard disk will be a good start. You can add more disks as you go on.
Thank You!
Hi Ez,
1) Check the permissions of directories defined by cache_dir in /etc/videocache.conf .
2) Does you squid server allow itself to connect to internet via itself?
Thank You!
I have the same problem as above, I've tried to apply beberpa suggest changes like those in the admin, but obtaining the same result "ERROR"
ubuntu 10.10
Help me admin
root@proxy:# tail -f /var/log/videocache/videocache.log
2011-03-10 13:52:05,572 716 192.168.168.1 56Cr5SiuvXk CACHE_MISS YOUTUBE Requested video was not found in cache.
2011-03-10 13:52:05,575 716 192.168.168.1 56Cr5SiuvXk NEW_URL YOUTUBE http://www.youtube.com/get_video?video_id=56Cr5SiuvXk&el=detailpage&t=vjVQa1PpcFPI42dVgRVF0TxvbjRATLaSILoumivJy3s=&tmi=1&asv=3&fmt=34&noflv=1
2011-03-10 13:52:05,752 734 192.168.168.1 56Cr5SiuvXk SCHEDULED YOUTUBE Video scheduled for download.
2011-03-10 13:52:05,756 734 - - SCHEDULED_ERR - Could not schedule video for download.
2011-03-10 13:55:14,580 716 192.168.168.1 - REQUEST - http://www.youtube.com/get_video?video_id=nzynx-dZQ2Y&t=vjVQa1PpcFPOYeHIiCLN8JoMkkLQETFcB-xcKwpmX5E=&tmi=1&fmt=34&el=detailpage&noflv=1&asv=3
2011-03-10 13:55:14,580 716 192.168.168.1 nzynx-dZQ2Y URL_HIT YOUTUBE http://www.youtube.com/get_video?video_id=nzynx-dZQ2Y&t=vjVQa1PpcFPOYeHIiCLN8JoMkkLQETFcB-xcKwpmX5E=&tmi=1&fmt=34&el=detailpage&noflv=1&asv=3
2011-03-10 13:55:14,581 716 192.168.168.1 nzynx-dZQ2Y CACHE_MISS YOUTUBE Requested video was not found in cache.
2011-03-10 13:55:14,584 716 192.168.168.1 nzynx-dZQ2Y NEW_URL YOUTUBE http://www.youtube.com/get_video?video_id=nzynx-dZQ2Y&t=vjVQa1PpcFPOYeHIiCLN8JoMkkLQETFcB-xcKwpmX5E=&tmi=1&fmt=34&el=detailpage&noflv=1&asv=3
2011-03-10 13:55:15,759 734 192.168.168.1 nzynx-dZQ2Y SCHEDULED YOUTUBE Video scheduled for download.
2011-03-10 13:55:15,765 734 - - SCHEDULED_ERR - Could not schedule video for download.