VideoCache

[Resolved] Cache directory size limiting is checked only before video is queued

by Anonymous on 14 Dec 2008

Hi :)

Size limits are only being checked in cache_video, and not also in download_from_source - this means that if one limits the number of max_parallel_downloads, a video could be queued while the directory still has space, but actually downloaded when the directory is over the max size.

-- imriz

6 Answers

by Kulbir Saini on 14 Dec 2008

Hi Imriz!

Fixed upstream. Thanks for reporting. Thats was a nice observation.

PS : Why don't you register. I hope Anonymous is not a name you like :P

by imriz on 14 Dec 2008

I'm a UNIX system administrator -laziness is my middle name ;)

Thanks Kulbir, I'll try the GIT version and report back

By the way, we're pushing 250 mbit per second through videocache+squid.

by Kulbir Saini on 14 Dec 2008

LOL!! I am also a system administrator at my university but not the UNIX one :) We use RHEL/CentOS. But agreed to "Sysadmins are lazy ;)". I would love to see your videocache stats in statistics forum :D

by imriz on 19 Feb 2009

Uhmm, with the new version the CPU usage went sky high. stracing the processes showed excessive stat() usage. Disabling the size limit checks solved it.

I think that the size checking should be done only before actually downloading the files (download_from_source), or globally - once every couple of minutes.

Doing stat() on all the files in the cache directory for every non-cached object killed an 8-cores machine at 100-150 requests per seconds.

by Kulbir Saini on 19 Feb 2009

Imriz,

Another use reported similar problem with cache size calculating with 1.3TB cache. I'll have to work on. May be we can do it periodically and cache the results for some time.

Please post your suggestions for the same.

Thank You!

by imriz on 19 Feb 2009

I think that doing it periodically and cache it, is the right way, But the docs should explain that the cache size might overflow a bit.

You need to sign in. Please sign in to add answer to this question.