Thin, Ruby on Rails & Nginx fair proxy: Performance testing 36

Posted by science on April 07, 2008

Meaurement scale

By science & wayneeseguin

Thin a new-ish application server, primarily designed for serving the same community as Zed Shaw’s (and now community managed) masterwork Mongrel. Its job is to dispatch web requests, primarily Rails and other Ruby frameworks. There’s plenty already written about Thin, to get you up and running.

I’ve been eying Thin and thanks to Wayne, I got motivated to test it out. He and I spent the better part of a day doing configuration analysis and performance testing on Thin in the context of EngineYard’s hosting environment. We had access to a brand new, unloaded “slice” (aka web server) on their server farm. The stack we used looks like:

Load balancers => Nginx => unix sockets => Thin => Rails

We ran a quite a few performance tests against Thin using the above setup and overall liked what we saw.

Summary:

  • It’s stable – no crashing even with high concurrency
  • It’s fast – performance was impressive (we will get some comparisons to Mongrel for you soon)
  • It doesn’t use much memory – We think this is at least partly because it lets you use unix sockets instead of ip ports.

We went a little deeper too. In particular, because Thin is using epoll/event_machine to pull from the unix sockets, we were worried that while network performance would be improved, user-interface latency would rise because fast requests would get stuck behind slow requests inside the unix socket queuing mechanism. [2]

Nginx offers a way out of this problem in the form of its “fair proxy” module – it ensures that each request is sent to a socket only if that socket is not currently handling any work. So sockets are not allowed to queue – nginx does that. Wayne and I both have experience with nginx + fair with mongrels. So how would nginx + fair do with Thin? About as expected which is pretty darn well.

Wayne and I hypothesized that the fair proxy module would result in higher latency but lower “worst case” performance. In fact “max time to return” was faster in all cases for the fair proxy. But “90% of requests” were returned slower for fair. (Both were within one standard deviation but appear significant to my eye b/c we got the same pattern for each set of tests). Average performance was statistically identical in all cases as were the total number of requests made in 300 seconds.


30 threads, running for 300 seconds (30 second warm-up) against 3 Thins

Test setup # of Requests Avg time 90% returned Max time to return
Fair Proxy 1738 5.08s 10.1s 14.3s
Socket queue 1730 5.10s 9.26s 16.7s

But this doesn’t tell the whole story. Web servers are often a little spooky – with some requests stalling for weird reasons (bad programming being a big one). To simulate the putative bad programmer, I installed a sleep function into our test Rails application (which is a full-sized commercial web app). It was designed to sleep for 4 seconds for 5% of the requests. This test was designed to determine if the fair proxy was doing its job. And it was.


Here’s what we found. For cases where 5% of your dynamic calls are “slow” using the “fair proxy” could boost your page throughput by 25%. What’s crucial about this finding is that if you are looking at your average “time to return” or even your “max time to return” you won’t see much difference between “fair proxy” and queuing in the unix sockets. But your users will see a difference in that your server will return more pages per minute because fewer pages will get caught in “slow lanes.”


10 Threads running for 300 seconds against 3 Thins, with 5% of requests “stuck” for 4s

Test setup # of Requests Avg time 90% returned Max time to return
Fair Proxy 1341 2.24s 6.09s 11.53s
Socket queue 1071 2.79s 6.93s 11.84s

We were quite happily surprised by these results and hope that others find them helpful in planning their server architecture in Ruby/Rails or whatever your platform of choice might be. Read on for more performance testing fun, in a comparison between Mongrel and Thin’s performance. Final Note: One thing I did notice was that under heavy load nginx fair proxy caused some weird “clustering” of requests – the graphs (unpublished) show it pretty clearly while the statistics (published above) do not. So while fair proxy looks pretty good overall, if you are heavily loaded, I might recommend from the study that you do not use fair proxy, at least the version I tested (current stable release). Post a comment if you have any questions about this.

End Notes:

  1. You can see our raw performance data here.
  2. To understand why this might occur, think of a rental car return area at an airport. They have long parking lanes, where you queue up behind cars in front of you waiting to return. Now if there’s a guy at the start of the lanes and he directs into a lane based on a round-robin strategy, if you get stuck behind a slow customer, two or three cars will pass you by in other lanes. That will make you annoyed. On the web you can’t see the other customers passing you, but queuing into sockets using round-robin can lead someone who has a lightweight request to wait behind a somewhat slower request. Unless your peak latency is under 1s, then this will cause noticeable problems for your customers.
  3. Science also has a writeup on optimizing Nginx and Rails page caching which may be of interest to readers.
Trackbacks

Use this link to trackback from your own site.

Comments

Leave a response

  1. Mark Foster Fri, 13 Jun 2008 12:32:18 UTC

    Interesting results. I wonder if you could share your nginx settings? We’ve also been using nginx+fair in front of mongrel and unsatisfied with the ‘blockages’ that are occurring. Is it better to use shorter fail_timeouts with fair?

  2. science Fri, 13 Jun 2008 12:42:15 UTC

    Mark: I’m running a pretty stock nginx config from EngineYard with just a few tweaks for page caching. You can get the nginx.conf from EY by booting their “express” version to see how their setup differs from yours:

    http://express.engineyard.com/

    I gave up on fair b/c my server responses are all reasonable even and the weird blockages you refer to were bugging me. I’m not sure what is causing them so I don’t know if fail_timeouts would make a different or not. My only suggestion is to build an eval rig and put a test load on it in the various configs.

  3. Matt Thu, 23 Jul 2009 15:36:52 UTC

    I’m wondering what version of nginx you used with the most recent version of upstream-fair. Like, Mark, upstream-fair has been causing me some issues.

    I have 4 backend thin servers connected by tmp sockets, but sometimes get small bursts of traffic (like 10 requests in a second), and I’m finding that nginx is blocking requests #5-10 because at that instance all 4 backend servers are busy. Is there anyway for nginx to que up those requests rather than reject them flat out?

    thanks.

    –matt
    VoIP Predictive Dialer

  4. WALLACE Fri, 20 Jul 2012 00:59:44 UTC


    Buy Viagra

    Check Quality Generic Pills Today!…

  5. sam Tue, 26 Aug 2014 07:18:55 UTC

    marble@crumlish.tulips” rel=”nofollow”>.…

    thank you….

  6. Allan Wed, 26 Nov 2014 16:06:33 UTC

    flesh@belligerent.landscape” rel=”nofollow”>.…

    good!!…

  7. Austin Fri, 28 Nov 2014 20:29:11 UTC

    classmates@singers.airless” rel=”nofollow”>.…

    ñïñ çà èíôó!…

  8. Gregory Fri, 28 Nov 2014 20:59:04 UTC

    sandy@repayment.photographing” rel=”nofollow”>.…

    ñýíêñ çà èíôó!!…

  9. alfred Mon, 01 Dec 2014 01:34:24 UTC

    facaded@emption.eye” rel=”nofollow”>.…

    thank you!!…

  10. fernando Mon, 01 Dec 2014 10:49:56 UTC

    firms@altered.desensitized” rel=”nofollow”>.…

    ñïàñèáî çà èíôó!!…

  11. wade Mon, 01 Dec 2014 13:22:29 UTC

    enumeration@relinquish.minimized” rel=”nofollow”>.…

    ñïñ çà èíôó!…

  12. Gordon Mon, 15 Dec 2014 08:59:00 UTC

    wynston@patriot.buckling” rel=”nofollow”>.…

    áëàãîäàðñòâóþ!!…

  13. carlos Thu, 18 Dec 2014 15:45:54 UTC

    clusters@vex.strangeness” rel=”nofollow”>.…

    ñýíêñ çà èíôó!!…

  14. franklin Fri, 19 Dec 2014 04:15:24 UTC

    strays@absinthe.rhymes” rel=”nofollow”>.…

    good info!…

  15. Kent Wed, 14 Jan 2015 03:33:00 UTC

    nolens@mon.graced” rel=”nofollow”>.…

    ñýíêñ çà èíôó….

  16. Jerry Wed, 14 Jan 2015 04:03:32 UTC

    sydney@carmine.anglican” rel=”nofollow”>.…

    ñïñ çà èíôó!!…

  17. aaron Wed, 14 Jan 2015 04:34:55 UTC

    enjoying@klees.sublimed” rel=”nofollow”>.…

    hello!!…

  18. Jon Wed, 14 Jan 2015 05:04:56 UTC

    ter@repayment.mechanism” rel=”nofollow”>.…

    ñïñ….

  19. calvin Wed, 14 Jan 2015 05:35:35 UTC

    missiles@chickens.reinvestigation” rel=”nofollow”>.…

    ñïñ….

  20. jessie Fri, 16 Jan 2015 00:16:22 UTC

    sarcasms@convocation.inroads” rel=”nofollow”>.…

    ñïàñèáî!…

  21. mitchell Fri, 16 Jan 2015 00:48:06 UTC

    etiquette@cloudcroft.owly” rel=”nofollow”>.…

    ñïàñèáî çà èíôó!!…

  22. Fred Fri, 16 Jan 2015 14:31:57 UTC

    banquet@overlooks.boardinghouses” rel=”nofollow”>.…

    ñýíêñ çà èíôó!!…

  23. Jesus Tue, 20 Jan 2015 05:08:25 UTC

    woolworkers@pyrometers.couve” rel=”nofollow”>.…

    tnx for info!!…

  24. Raul Tue, 20 Jan 2015 05:39:39 UTC

    nodules@newborn.epigenetic” rel=”nofollow”>.…

    áëàãîäàðþ….

  25. Aaron Tue, 20 Jan 2015 06:12:09 UTC

    irruptions@triplet.custom” rel=”nofollow”>.…

    ñïñ çà èíôó!!…

  26. jorge Tue, 20 Jan 2015 06:44:41 UTC

    committment@application.flem” rel=”nofollow”>.…

    thanks!…

  27. alvin Wed, 28 Jan 2015 12:28:16 UTC

    bornholm@courtyard.german” rel=”nofollow”>.…

    thank you!…

  28. Walter Thu, 29 Jan 2015 05:59:25 UTC

    fagets@gawky.imposition” rel=”nofollow”>.…

    ñïñ çà èíôó!…

  29. Ernest Thu, 05 Feb 2015 15:56:35 UTC

    suspicious@oman.obliterated” rel=”nofollow”>.…

    ñïñ!!…

  30. Hubert Thu, 05 Feb 2015 16:28:33 UTC

    projectiles@ryusenji.straits” rel=”nofollow”>.…

    ñýíêñ çà èíôó….

  31. Johnnie Fri, 06 Feb 2015 05:18:13 UTC

    impinging@debut.gimbel” rel=”nofollow”>.…

    good!!…

  32. luis Tue, 10 Feb 2015 01:27:01 UTC

    vessel@terrace.schwarzen” rel=”nofollow”>.…

    ñïñ!!…

  33. Ruben Tue, 10 Feb 2015 02:05:29 UTC

    enroll@emanation.overconfident” rel=”nofollow”>.…

    ñïñ çà èíôó….

  34. Marc Tue, 10 Feb 2015 02:41:53 UTC

    agatha@doors.tortured” rel=”nofollow”>.…

    tnx….

  35. Robert Tue, 10 Feb 2015 03:17:44 UTC

    flaunting@heterogamous.recluse” rel=”nofollow”>.…

    tnx for info!!…

  36. Free classified ads worldwide Sat, 18 Mar 2017 02:29:32 UTC

    http://www.eajila.com/...

    Free classified advertising worldwide have a distinctive and best connection with advertising around enjoy investing have a great time in advertising free advertising posting site…

Comments