You are hereForums / Bugs / [Resolved] Cache directory size limiting is checked only before video is queued

Version 1.9.9 is out. We have added support for Youtube mobile videos, Facebook, AOL and MySpace videos as well. Fixed caching issues for some already supported websites. Options to control cache directory selection algorithm, bandwidth usage by videocache, etc. More robust videocache scheduler with a lot more options and features. Have a look at changelog for more details. If you have purchased Videocache in last one year, Contact Us to claim your free upgrade. Please mention the email address using which you purchased Videocache. New users proceed to Pricing Page right now, secure your copy and enjoy caching :-) Also, check our new and simplified upgrade policy :-)

[Resolved] Cache directory size limiting is checked only before video is queued


6 replies [Last post]
Anonymous
[Resolved] Cache directory size limiting is checked only before video is queued
Printer-friendly versionSend to friend

Hi :)

Size limits are only being checked in cache_video, and not also in download_from_source - this means that if one limits the number of max_parallel_downloads, a video could be queued while the directory still has space, but actually downloaded when the directory is over the max size.

-- imriz

admin's picture
Online
Joined: Nov 2 2008

Hi Imriz!

Fixed upstream. Thanks for reporting. Thats was a nice observation.

PS : Why don't you register. I hope Anonymous is not a name you like :P

Offline
Joined: Dec 14 2008

Uhmm, with the new version the CPU usage went sky high. stracing the processes showed excessive stat() usage. Disabling the size limit checks solved it.

I think that the size checking should be done only before actually downloading the files (download_from_source), or globally - once every couple of minutes.

Doing stat() on all the files in the cache directory for every non-cached object killed an 8-cores machine at 100-150 requests per seconds.

admin's picture
Online
Joined: Nov 2 2008

Imriz,

Another use reported similar problem with cache size calculating with 1.3TB cache. I'll have to work on. May be we can do it periodically and cache the results for some time.

Please post your suggestions for the same.

Thank You!

Offline
Joined: Dec 14 2008

I think that doing it periodically and cache it, is the right way, But the docs should explain that the cache size might overflow a bit.

Offline
Joined: Dec 14 2008

I'm a UNIX system administrator -laziness is my middle name ;)

Thanks Kulbir, I'll try the GIT version and report back

By the way, we're pushing 250 mbit per second through videocache+squid.

admin's picture
Online
Joined: Nov 2 2008

LOL!! I am also a system administrator at my university but not the UNIX one :) We use RHEL/CentOS. But agreed to "Sysadmins are lazy ;)". I would love to see your videocache stats in statistics forum :D

Post new comment

The content of this field is kept private and will not be shown publicly. If you have a Gravatar account, used to display your avatar.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <img> <code>
  • Lines and paragraphs break automatically.
  • HTML tags will be transformed to conform to HTML standards.

More information about formatting options

CAPTCHA
Are you a human visitor ?
Image CAPTCHA
Enter the characters (without spaces) shown in the image.

My Book on Squid

Purchase/Renew

One License - USD $399
One License - USD $349*
Validity - One Year
Need more? Contact Us
*Includes free upgrades and support for one year.



For details, check pricing.

Recent comments

Who's online

There are currently 2 users and 138 guests online.