Youtube Cache and Mikrotik
by Kulbir Saini on 24 Nov 2008If anybody has successfully youtube cache with mikrotik, then please reply to this message. It would be great if you can also mention the details.
44 Answers
Well I will just configure it with mikrotic senerio ,Ive mikrotic and one squid server for proxy/chache all browsing request sent to my squid .Now im going to config it on that squid server :) .
Regards
Thank you for considering youtube cache. If you get youtube cache working with MikroTik, please reply back. It would great.
Thanks in advance :)
here i´m using mikrotik as gateway, redirecting http traffic to my SQUID server.
the mikrotik web-proxy isn´t open for customizing and cachevideos only can installed on a separated box running squid...
:)
Hi Alexandre,
As you are redirecting your web traffic to your squid server, so you can setup videocache on your squid server itself.
Thank You :)
yes, i´m using videocache .. and it working very fine !!
thank you :)
Hi Alexander
i am trying to let Mikrotik just take youtube and the 9 sites that videocache cache to take it from the 2nd box only. can this happen with Mikrotik?
you said that videocache is working very fine with you.
its stopping every 12 hours with me. any help to resolve my problem? what os are you using?
10x in advanced
hello,
i´m using this project
this project based on squid HEAD 2.6/2.7 with many improvments, here i´m running very proxies using cacheboy and performance are very better than squid...
Hi There
i was running SQUID with ZPH to MARK cache object and don't set any limit on these object. videocache object not seen as a cache object
Hi Alexandre Correa
you said that videocache is working very fine with you.
about mine it is stopping nearly every 12 hours i have to reboot it to let it work again.
is this ever happens with you?
can you tell me about the steps you've made?
thanks and waiting for your reply.
ZPH should work fine with 2.7 and the proper configuration. I have tested it and we have a redirect with mikrotik and 5 proxies working with this. One big drive for the Videocache mount obviously. Reiserfs on our proxies etc.
As far as running it on an actual Mikrotik device, there will be a limit on the amount of users due to the Geode processors and the BUS on the boards. Easier to have separate proxies and ZPH working.
I'm using Mikrotik as a gateway , squid 2.7 as a cache proxy with zph , finally i add the amazing videocache to the squid server , and it works like a charm , EXCEPT one thing , the video file doesnt marked by zph rule !!! why it doesnt treated like any other hit object , its so important to give video files in max speed , thats wahy i'm caching them ..
any help ??
Anon,
Thanks for your interest in videocache. The files are served from videocache if they are available otherwise they are fetched from upstream. But they are never marked as hit object because squid doesn't allow url_rewrite_program or the plugin to interfere with it. You can create custom rule for the IP you set as cache_host in your /etc/videocache.conf file.
Thank you for using videocache :)
thanks alot for your great help ..
but what rules you mean ? what and where should i write ?
i'm not that expert in linux ..
it will be very complete solution if you just tell me how to do this ..
thanks in advance ..
Hi all
all you all said was fine but still no-body said how you can make mikrotik cache video
if any body know how to please reply
thanks in advance
Hi everyone!
I use Mikrotik + Videocache 1.9 and it works perfect, without having to put the cache proxy ip to the clients browser, if anyone is trying to do a combination with squid and mikrotik this may be useful.
First set up your videocache box then test it with entering the cache_host ip to the client web browser and if it works fine then all you have to do is make a rule in Mikrotik firewall :
[bedri@mikrotik]> ip firewall nat add chain=dstnat src-address="x.x.x.x/x" protocol=tcp dst-port=80 action=dst-nat to-addresses="your squid ip address" to-ports=3128
this rule will redirect the http traffic to the squid box
Enjoy caching videos :D
Invisible :)
Thanks for the tips. I hope they'll be useful to videocache users :D
Thank You!
Hi Admin,
I use Mikrotik as a gateway and proxy (http and ftp) on DMZ. From proxy can forward client A to use WAN on Link A and client b to use WAN Link B (in this case just for http connection).
For redirect http connection to my proxy on DMZ i'm using DNAT and Mark routing. For FTP i'm using mark routing
and .............
EVERTHING IS OK !!!
Thanks for your video cache
Hello Evrey Body
i am use squid 3.0 and videocache i will config all file conf but i have problem in add this comand in the squid.conf ( /usr/share/videocache/videocache.py ) squid cache is dead after add this comand
This is my all confgrtion
-------------------------
Squid.conf
######################################################
# WELCOME TO SQUID 2
# ------------------
#
http_port 3128 transparent
icp_port 3130
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \\?
no_cache deny QUERY
auth_param basic children 5
auth_param basic credentialsttl 2 hours
ipcache_size 10240
ipcache_low 95
ipcache_high 98
fqdncache_size 16384
cache_dir aufs /var/spool/squid 800000 16 256
coredump_dir /var/spool/squid
visible_hostname Mesho-Alhazeen-Server
cache_mgr mesho_alhazeen@yahoo.com
####################################################################
acl nimda-eml urlpath_regex -i \\.eml$
acl nimda-nws urlpath_regex -i \\.nws$
acl wml-files urlpath_regex -i \\.wml$
acl exe-files urlpath_regex -i \\.exe$
acl avi-files urlpath_regex -i \\.avi$
acl mp3-files urlpath_regex -i \\.mp3$
acl vbx-files urlpath_regex -i \\.vb?$
acl mpg-files urlpath_regex -i \\.mpg$
acl zip-files urlpath_regex -i \\.zip$
acl scr-files urlpath_regex -i \\.scr$
acl rm-files urlpath_regex -i \\.rm$
acl wav-files urlpath_regex -i \\.wav$
acl net-files urlpath_regex -i \\.net$
acl com-files urlpath_regex -i \\.com$
acl asp-files urlpath_regex -i \\.asp$
acl php-files urlpath_regex -i \\.php$
acl co-files urlpath_regex -i \\.co$
acl org-files urlpath_regex -i \\.org$
acl rar-files urlpath_regex -i \\.rar$
acl html-files urlpath_regex -i \\.html$
acl htm-files urlpath_regex -i \\.htm$
acl jpg-files urlpath_regex -i \\.jpg$
acl gif-files urlpath_regex -i \\.gif$
acl php-files urlpath_regex -i \\.php$
acl swf-files urlpath_regex -i \\.swf$
acl jv-files urlpath_regex -i \\.jv$
acl db-files urlpath_regex -i \\.db$
acl cab-files urlpath_regex -i \\.cab$
acl aspx-files urlpath_regex -i \\.aspx$
####################################################################
acl all src 84.11.138.0/28
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl SSL_ports port 443 563
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 563 # https, snews
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
####################################################################
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access allow all
icp_access allow all
ie_refresh on
###################################################################
tcp_outgoing_tos 0x20
zph_mode tos
zph_local 0x30
zph_parent 0
####################################################################
#snmp
acl snmppublic snmp_community public
snmp_access allow snmppublic all
snmp_incoming_address 0.0.0.0
snmp_outgoing_address 0.0.0.0
####################################################################
cache_swap_low 98
cache_swap_high 99
cache_log /var/log/squid/cache.log
emulate_httpd_log off
pid_filename /var/run/squid.pid
mime_table /usr/share/squid/mime.conf
log_fqdn off
quick_abort_min 0
quick_abort_max 0
quick_abort_pct 98
negative_ttl 2 minutes
half_closed_clients off
read_timeout 15 minutes
client_lifetime 2 hours
pconn_timeout 60 seconds
positive_dns_ttl 60 seconds
negative_dns_ttl 30 seconds
cache_effective_user proxy
cache_effective_group proxy
vary_ignore_expire on
pipeline_prefetch on
reload_into_ims on
snmp_port 3401
client_db on
store_objects_per_bucket 10
store_avg_object_size 13 KB
icon_directory /usr/share/squid/icons
error_directory /usr/share/squid/errors/English
buffered_logs off
logfile_rotate 7
forwarded_for on
icp_hit_stale on
log_icp_queries off
query_icmp on
####################################################################
cache_mem 1500 MB
maximum_object_size 500 MB
minimum_object_size 0 KB
maximum_object_size_in_memory 512 KB
dns_nameservers 83.170.1.240 83.170.1.241 4.2.2.4 4.2.2.1 192.9.9.3
request_header_max_size 10 KB
request_timeout 30 seconds
negative_ttl 5 minutes
connect_timeout 2 minutes
read_timeout 15 minutes
shutdown_lifetime 10 seconds
####################################################################
url_rewrite_program /usr/bin/python /usr/share/videocache/videocache.py
url_rewrite_children 10
acl videocache_allow_url url_regex -i \\.youtube\\.com\\/get_video
acl videocache_allow_url url_regex -i \\.cache[a-z0-9]?[a-z0-9]?[a-z0-9]?\\.googlevideo\\.com\\/videoplayback
acl videocache_allow_url url_regex -i \\.cache[a-z0-9]?[a-z0-9]?[a-z0-9]?\\.googlevideo\\.com\\/get_video
acl videocache_allow_url url_regex -i proxy\\-[0-9][0-9]\\.dailymotion\\.com\\/
acl videocache_allow_url url_regex -i [a-z0-9][0-9a-z][0-9a-z]?[0-9a-z]?[0-9a-z]?\\.xtube\\.com\\/(.*)flv
acl videocache_allow_url url_regex -i bitcast\\.vimeo\\.com\\/vimeo\\/videos\\/
acl videocache_allow_url url_regex -i va\\.wrzuta\\.pl\\/wa[0-9][0-9][0-9][0-9]?
acl videocache_allow_url url_regex -i \\.files\\.youporn\\.com\\/(.*)\\/flv\\/
acl videocache_allow_url url_regex -i \\.msn\\.com\\.edgesuite\\.net\\/(.*)\\.flv
acl videocache_allow_dom dstdomain v.mccont.com vp.video.google.com dl.redtube.com
acl videocache_deny_url url_regex -i http:\\/\\/[a-z][a-z]\\.youtube\\.com http:\\/\\/www\\.youtube\\.com
url_rewrite_access deny videocache_deny_url
url_rewrite_access allow videocache_allow_url
url_rewrite_access allow videocache_allow_dom
redirector_bypass on
###################################################################
#refresh_pattern for cache sites
#refresh patterns to enable caching of MS windows update
refresh_pattern http://.*\\.windowsupdate\\.microsoft\\.com/ 0 80% 20160 reload-into-ims override-lastmod
refresh_pattern http://office\\.microsoft\\.com/ 0 80% 20160 reload-into-ims override-lastmod
refresh_pattern http://windowsupdate\\.microsoft\\.com/ 0 80% 20160 reload-into-ims override-lastmod
refresh_pattern http://w?xpsp[0-9]\\.microsoft\\.com/ 0 80% 20160 reload-into-ims override-lastmod
refresh_pattern http://w2ksp[0-9]\\.microsoft\\.com/ 0 80% 20160 reload-into-ims override-lastmod
refresh_pattern http://download\\.microsoft\\.com/ 0 80% 20160 reload-into-ims override-lastmod
#and some other windows updaters
refresh_pattern http://download\\.macromedia\\.com/ 0 80% 20160 reload-into-ims override-lastmod
refresh_pattern ftp://ftp\\.nai\\.com/ 0 80% 20160 reload-into-ims override-lastmod
refresh_pattern http://ftp\\.software\\.ibm\\.com/ 0 80% 20160 reload-into-ims override-lastmod
refresh_pattern cgi-bin 1 20% 2
refresh_pattern \\.asp$ 1 20% 2
refresh_pattern \\.acgi$ 1 20% 2
refresh_pattern \\.cgi$ 1 20% 2
refresh_pattern \\.pl$ 1 20% 2
refresh_pattern \\.shtml$ 1 20% 2
refresh_pattern \\.php3$ 1 20% 2
refresh_pattern \\? 1 20% 2
refresh_pattern \\.gif$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.jpg$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.bom\\.gov\\.au 30 20% 120 reload-into-ims
refresh_pattern \\.html$ 480 50% 22160 reload-into-ims override-lastmod
refresh_pattern \\.htm$ 480 50% 22160 reload-into-ims override-lastmod
refresh_pattern \\.class$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.zip$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.jpeg$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.mid$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.shtml$ 480 50% 22160 reload-into-ims override-lastmod
refresh_pattern \\.exe$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.thm$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.wav$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.txt$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.cab$ 10080 90% 560000 reload-into-ims override-lastmod
refresh_pattern \\.au$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.mov$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.xbm$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.ram$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.avi$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.chtml$ 480 50% 22160 reload-into-ims override-lastmod
refresh_pattern \\.thb$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.dcr$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.bmp$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.phtml$ 480 50% 22160 reload-into-ims override-lastmod
refresh_pattern \\.mpg$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.pdf$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.art$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.swf$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.mp3$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.ra$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.spl$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.viv$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.doc$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.gz$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.Z$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.tgz$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.tar$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.vrm$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.vrml$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.aif$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.aifc$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.aiff$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.arj$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.c$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.cpt$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.dir$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.dxr$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.hqx$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.jpe$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.lha$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.lzh$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.midi$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.movie$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.mp2$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.mpe$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.mpeg$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.mpga$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.flv$ 10080 90% 56000 reload-into-ims override-lastmod
refresh_pattern \\.pl$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.ppt$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.ps$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.qt$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.qtm$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.ras$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.sea$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.sit$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.tif$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.tiff$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.snd$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern \\.wrl$ 10080 90% 43200 reload-into-ims override-lastmod
refresh_pattern ^gopher:// 1440 25% 10080
refresh_pattern ^ftp:// 1440 20% 1440 reload-into-ims override-lastmod
refresh_pattern . 320 40% 56000 reload-into-ims override-lastmod
cache_access_log /var/log/squid/access.log
cache_store_log none
cache_log none
cache allow all
############################################End squid.conf
and the confgrtion in videocache
#videocache.py
cache_host = 84.11.138.6 my proxy ip
proxy = http://84.11.138.6:3128/
i am using mikrotik and squid+videocache and it's work, thank for kulbir.
Internet >> mikrotik ( BW Management+load balance ) >> Squid + SquidGuard + VideoCache >> LAN ( 100 user )
now i try use mikrotik 3.2
Hey guys
please i am trying to configure a mikrotik server from a to z ( web cash) video Cash) and the clients will wokr on fake ip and not on DHCP and i am using the version 2.9.6 can some body please help
Hi Peter. If you want me to help you, please contact me on skype: ryszko207.
I can help you if you want.
Regards
Ryz
Dear Sir,
I want to configure Video Cahe Squid server but I am not able to configure this. Please help me regarding this.
Regards
Mahfuzur Rahman
iam using mikrotik v3.2 + Tproxy bridge mode with videocache + 100 pc clients is working very well
Hi guys.
I'am using videocache in a testing scenario, and I have a problem caching redtube video. It seems that every time a hit a video, the URL of the video changes and despite the video was previously cached, I get a MISS and the video is downloaded again, it seems like if the video file name changes on the last characters
Here there's an example,
[ ] 4b7498c9 11-Feb-2010 20:53 5.4M
[ ] 4b74993b 11-Feb-2010 20:54 5.4M
Does anyone know a solution for this?
Thank you.
hello mesho
please try to fix permition for /usr/share/videocache
you can try chmod 777 /usr/share/videocache.
the key for your problem is permition folder
best regards
hi guys:
any one help me how to configure videocache in squid. plz E-mail me
Dear admin,
I have over 1500 pc clients in dhcp mode with mikrotik version 5.11. I configured squid proxy and also videocache but its not working properly. maximum time clients page are going to slow. sometimes its not opening. When i again and again clear the cache than its run quiet well but after 15 or 20 minutes later again same problem will you tell me how can i recover from this problem???
regards,
Md. Farhad Khan.
Hi Farhad,
Most probably heavy I/O activity is the reason of such a behaviour. Try to disable Squid's disk cache and check the behaviour to confirm the same.
Thank You!
Hi Shakeel,
Please contact us using contact link at the top.
Thank You!
Hi guys.
I'am using Mikritik 4.11 And now i update My Mikrotik 5.11 I have a problem to configure My WEB PROXY.
Please help me.....
regards,
Md. Abu Sayed
Hey, I'm using Mikrotik I'm using squid for caching my webpages and using videocache for caching the youtube videos, I have a problem, the videos is caching but when I try to open the same video it appears a error. Can you help me please?.
Hi, 5.11 version have several problems with network cards auto-negotiation, make a upgrade to 5.14 asap.
Jorge
I am using mikrotik and squid 2.7 with zph it works perfectly. normal hit's can be marked using zph and since the cached video is served using Apache web server it can be bypassed by marking the packet from the web servers http port which is 80 using mangle and moving it higher in queue list ( simple queue ) or from queue tree.
Hi, can you help me with quid + videocache configuration?, I've done all and the video cache after this command: url_rewrite_program /usr/bin/python /usr/share/videocache/videocache.py
I'm using Debian 6 + quid 2.7 and videocache 1.9.2.
Hi Stalin,
Could you please use the contact link at the top and contact us with the exact error you are getting?
Thank You!
Dear All,
I need youtube caching in squid server. Please help me to do this.
i know about caching when snat is running on network squid runnning