XMLRPC Server stability
by bellera on 19 Feb 2009Kulbir,
Even be using version 1.8 I still have problems with XMLRPC Server stability.
Following your advice (in a post on this forum) I changed url_rewrite_children from 20 to 10. But then cache.log shows me warnings like this:
2009/02/19 10:03:40| WARNING: All url_rewriter processes are busy.
2009/02/19 10:03:40| WARNING: up to 10 pending requests queued
It should be able to work more url_rewrite_children because it is compromised the speed of the proxy.
As you know, I use zapchain to chain squidGuard and videocache. Moreover, since a week ago I have in production videofilter.py. Sometimes videofilter.py has some error (I'm polishing the code). Then squid makes a restart that can be seen perfectly in cache.log. Apart from stopping the navigation (logical), the restart causes the fall of XMLRPC Server, which can be seen in videocache.log. Example:
2009-02-18 00:05:05,358 41774 - - XMLRPCSERVER - Starting XMLRPCServer on port 9100.
2009-02-18 09:55:00,703 44114 - - DEQUEUE_ERR - Error querying XMLRPC Server.
2009-02-18 09:55:10,768 44071 - - DEQUEUE_ERR - Error querying XMLRPC Server.
2009-02-18 09:55:10,770 44112 - - DEQUEUE_ERR - Error querying XMLRPC Server.
2009-02-18 09:55:10,771 44073 - - DEQUEUE_ERR - Error querying XMLRPC Server.
2009-02-18 09:55:10,770 44116 - - DEQUEUE_ERR - Error querying XMLRPC Server.
2009-02-18 09:55:10,773 44118 - - DEQUEUE_ERR - Error querying XMLRPC Server.
2009-02-18 09:55:16,819 44159 - - STRAT_XMLRPC_SERVER_ERR - Cannot start XMLRPC Server - Exiting
2009-02-18 09:55:17,074 44156 - - STRAT_XMLRPC_SERVER_ERR - Cannot start XMLRPC Server - Exiting
2009-02-18 09:55:18,239 44154 - - STRAT_XMLRPC_SERVER_ERR - Cannot start XMLRPC Server - Exiting
XMLRPC Server is depending of the behavior of the redirector(s) for squid. It would be ideal run XMLRPC Server as a service of the machine, without having to rely on what squid is doing. Only need to videocache check if XMLRPC Server is on or not.
Regards,
Josep Pujadas
4 Answers
Josep,
Decoupling XMLRPC server from videocache is one nice suggestion but currently I don't see a method to do that because then we'll have to find ways to start XMLRPC server on its own whenever squid is started/restarted/reloaded. If you have suggestions in that regard, kindly post them here.
Thank you for the suggestion.
I second the suggestion to split the XMLRPC. It will make it easier to scale videocache that way. Monitoring and restarting can be done with the same tools one monitors any other daemon.
I have a similar problem with XMLRPC Server. After I used Webmin to increase the size of my cache, Apply Configuration, I now get lots of XMLRPC error in the log and Videocache seems to have stopped functioning:
2009-10-26 21:53:18,479 2332 - - RELOAD - videocache plugin was reloaded.
2009-10-26 21:53:18,481 2337 - - RELOAD - videocache plugin was reloaded.
2009-10-26 21:53:18,481 2335 - - RELOAD - videocache plugin was reloaded.
2009-10-26 21:53:18,481 2334 - - RELOAD - videocache plugin was reloaded.
2009-10-26 21:53:18,484 2338 - - RELOAD - videocache plugin was reloaded.
2009-10-26 21:53:18,485 2336 - - RELOAD - videocache plugin was reloaded.
2009-10-26 21:53:18,483 2339 - - RELOAD - videocache plugin was reloaded.
2009-10-26 21:57:58,996 9098 - - DEQUEUE_ERR - Error querying XMLRPC Server.
2009-10-26 21:57:58,996 9098 172.17.101.169 5826GxBlpaD.flv DOWNLOAD XTUBE 4894196 Video was downloaded and cached.
2009-10-26 21:58:03,962 8771 - - DEQUEUE_ERR - Error querying XMLRPC Server.
2009-10-26 21:58:04,043 8771 172.17.214.187 37a3a064afee31e7 DOWNLOAD YOUTUBE 7541946 Video was downloaded and cached.
2009-10-26 21:59:04,772 6870 - - DEQUEUE_ERR - Error querying XMLRPC Server.
2009-10-26 21:59:04,772 6870 172.17.214.197 16eda895ebe93688 DOWNLOAD YOUTUBE 20265488 Video was downloaded and cached.
2009-10-26 21:59:14,342 8354 - - DEQUEUE_ERR - Error querying XMLRPC Server.
2009-10-26 21:59:14,342 8354 172.17.101.169 6230GxHsCw9.flv DOWNLOAD XTUBE 17008320 Video was downloaded and cached.
2009-10-26 21:59:35,665 7996 - - DEQUEUE_ERR - Error querying XMLRPC Server.
Any advice on how to get rid of the error without rebooting?