VideoCache
Videocache is no longer in development.

two instances

by soho on 30 Dec 2008

Currently I'm use the Ubuntu-Server 8.0.4 with the squid compiled manually and configured to run on a core of each of my server processor Core 2 Duo.

Squid

configure options: '--prefix=/usr' '--enable-async-io' '--enable-icmp' '--enable-useragent-log' '--enable-snmp' '--enable-cache-digests' '--enable-follow-x-forwarded-for' '--enable-storeio=ufs,aufs,coss,diskd,null' '--enable-removal-policies=heap,lru' '--with-maxfd=32768' '--enable-pool' '--disable-ident-lookups' '--enable-truncate' '--exec-prefix=/usr' '--bindir=/usr/sbin' '--libexecdir=/usr/lib/squid' '--enable-linux-netfilter' '--with-dl' '--with-large-files' '--with-aio' '--enable-epoll' '--with-pthreads' '--sysconfdir=/etc/squid' '--enable-wccp' '--enable-wccpv2' '--with-build-environment=POSIX_V6_LP64_OFF64' '--enable-default-err-language=Portuguese' '--enable-referer-log' '--disable-dependecy-tracking' '--enable-carp' '--enable-arp-acl' '--enable-delay-pools' '--enable-cachemgr-hostname=localhost' '--enable-http-violations' '--enable-ntlm-fail-open' '--enable-stacktraces' '--enable-x-accelerator-vary' '--enable-large-cache-files' 'CFLAGS=-DNUMTHREADS=120 -march=nocona -O3 -pipe -fomit-frame-pointer -funroll-loops -ffast-math -fno-exceptions'

I have 2 internal networks I redirect a network to a cache and another network to another cache. The caches are configured in parent sibling. My cache directory of HDDs are separated.

squid1.conf

http_port 33127 transparent
snmp_port 3400
cache_mem 768 MB
cache_swap_high 90
cache_swap_low 80
maximum_object_size 90 MB
maximum_object_size_in_memory 128 KB
ipcache_size 51200
ipcache_high 95
ipcache_low 90
fqdncache_size 51200

cache_replacement_policy heap LFUDA
memory_replacement_policy heap GDSF
cache_dir aufs /cache01 120000 32 256

# hide squid version
httpd_suppress_version_string on

access_log /var/log/squid/access1.log
cache_log none
cache_store_log none

coredump_dir /var/spool/squid

mime_table /etc/squid/mime.conf
pid_filename /var/run/squid1.pid
debug_options ALL, 5
log_fqdn off

half_closed_clients off

## ACLs
acl all src all
acl public_aloo snmp_community public
# Public network
acl internet src 200.x.x.x/24 200.y.y.y/24
# Private network
acl intranet src 192.168.0.0/16 10.0.0.0/8 172.16.0.0/12

# drop miss access
acl irmaos src 127.0.0.0/255.0.0.0
miss_access deny irmaos

#http_access allow all
miss_access allow all
snmp_access allow public_aloo all

## No Cache
acl NOCACHE url_regex "/etc/squid/no-cache"
acl NOCACHE url_regex gateway/gateway.dll?
no_cache deny NOCACHE

acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443 563 82
acl Safe_ports port 1025-65535  # unregistered ports
acl purge method PURGE
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge

#http_access deny !Safe_ports
#http_access deny CONNECT !SSL_ports

http_access allow localhost
http_access allow internet
http_access allow intranet
http_access deny all

cache_mgr webmaster@example.com
cache_effective_user squid
cache_effective_group squid
visible_hostname proxy.example.com

unique_hostname proxyex-1
via off
forwarded_for off
logfile_rotate 24
pipeline_prefetch on
shutdown_lifetime 1 second
read_ahead_gap 2 KB
quick_abort_min 0 KB
client_db off
buffered_logs on
positive_dns_ttl 24 hours
negative_dns_ttl 10 seconds
request_timeout 40 seconds
connect_timeout 40 seconds
pconn_timeout 40 seconds
dns_nameservers 200.x.x.x 200.y.y.y 
emulate_httpd_log off
log_ip_on_direct on

# hierarchy
icp_port 3127
icp_access allow all

# parent sibling
cache_peer 127.0.0.1 sibling 33128 3128 proxy-only round-robin name=proxyex-2

squid2.conf

http_port 33128 transparent
snmp_port 3401

cache_mem 768 MB
cache_swap_high 90
cache_swap_low 80
maximum_object_size 90 MB
maximum_object_size_in_memory 128 KB
ipcache_size 51200
ipcache_high 95
ipcache_low 90
fqdncache_size 51200

cache_replacement_policy heap LFUDA
memory_replacement_policy heap GDSF
cache_dir aufs /cache02 120000 32 256

# hide squid version
httpd_suppress_version_string on

access_log /var/log/squid/access2.log
cache_log none
cache_store_log none

coredump_dir /var/spool/squid

mime_table /etc/squid/mime.conf
pid_filename /var/run/squid2.pid
debug_options ALL, 5
log_fqdn off

half_closed_clients off

## ACLs
acl all src all
acl public_aloo snmp_community public
# Public network
acl internet src 200.x.x.x/24 200.y.y.y/24
# Private network
acl intranet src 192.168.0.0/16 10.0.0.0/8 172.16.0.0/12

# drop miss access
acl irmaos src 127.0.0.0/255.0.0.0
miss_access deny irmaos

#http_access allow all
miss_access allow all
snmp_access allow public_aloo all

## No Cache
acl NOCACHE url_regex "/etc/squid/no-cache" \\?
acl NOCACHE url_regex gateway/gateway.dll? \\?
no_cache deny NOCACHE

acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443 563 82
acl Safe_ports port 1025-65535 # unregistered ports
acl purge method PURGE
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge

#http_access deny !Safe_ports
#http_access deny CONNECT !SSL_ports

http_access allow localhost
http_access allow internet
http_access allow intranet
http_access deny all

cache_mgr webmaster@example.com
cache_effective_user squid
cache_effective_group squid
visible_hostname proxy.example.com

unique_hostname  proxyex-2
via off
forwarded_for off
logfile_rotate 24
pipeline_prefetch on
shutdown_lifetime 1 second
read_ahead_gap 2 KB
quick_abort_min 0 KB
client_db off
buffered_logs on
positive_dns_ttl 24 hours
negative_dns_ttl 10 seconds
request_timeout 40 seconds
connect_timeout 40 seconds
pconn_timeout 40 seconds
dns_nameservers 200.x.x.x 200.y.y.y 
emulate_httpd_log off
log_ip_on_direct on

# hierarchy
icp_port 3128
icp_access allow all

# parent sibling
cache_peer 127.0.0.1 sibling 33127 3127 proxy-only round-robin name=proxyex-1

If I set the squid.conf of both servers, the videocache work normally? In both instances? Or set up a instance only for videocache (if so, leaving only the settings in one of squid.conf url_rewrite)?

5 Answers

by Kulbir Saini on 30 Dec 2008

Hi Sergio,

Configuring videocache on parent squid will be enough. You don't need to configure it on both the squids.

I hope you'll be benefited by videocache.

Thank you for considering videocache :)

by soho on 30 Dec 2008

Hi Kulbir,

thank you for answering. I set here and in case of any doubt, I will post in the forum again.

My biggest doubt is about the videocache meet the two networks simultaneously, because each network has its own squid.conf

by Kulbir Saini on 30 Dec 2008

Hello Sergio,

As long as the the exit point to Internet is a single machine from the networks, configuring videocache on the outermost squid would be sufficient.

Let me know if I can help you in any other way :)

by Rafael Guedes on 9 Jan 2009

I'm having the same problem. Setup in only one did not work with me.

Sergio you already tried to redirect a domain to match ACLs?

For example:

Set the videocache in squid 1

And the squid 2:

acl redirects url_regex-i \\. youtube \\. com \\ / get_video
cache_peer_access cache1.example.com allow redirects

I think it makes no difference, but .... Thanks for your answers.

by Kulbir Saini on 19 Jan 2009

Hi Rafael,

You can try contacting Sergio using his contact address.

Thank You!