Thin vs. Mongrel: A Ruby on Rails performance shootout 75

Posted by science on April 07, 2008

Graph of nothing

Previously Science and wayneseguin published a study looking at the performance of nginx fair proxy. To take that a little further, Science conducted an examination of how Thin and Mongrel compare head-to-head on performance. For kicks we took a look at Rails page template caching facility to see if that significantly impacts performance (it does). Full details follow..

For an idea of what the h/w testing setup looked like, read the previous study (cited above). For these tests, we used 3 instances of Thin or Mongrel (with lots of free ram). nginx fair proxy was turned on. Rails caching was fully enabled (including template caching). The tests all ran for 300 seconds. We pulled HEAD requests only to minimize over the wire throughput variance (since the test were initiated remote of the data center). Thins were wired up to unix sockets and Mongrels were going over IP.

Overall, I’d say that under these testing conditions Thin is 4% faster than Mongrel. That’s not much, and it’s within the standard deviation of each test result, but it was pretty consistent throughout the testing so I’m inclined to believe it. Your results may vary. [1]

Here is the summary of results:

Server type Avg response (s) Total pages (#) 90% max response (s)
Thins, 10 threads 1.72 1734 3.92
Mongrels, 10 threads 1.78 1677 3.31
Thins, 30 threads 5.08 1738 10.14
Mongrels, 30 threads 5.20 1709 10.86
Thins, 40 threads 6.69 1753 10.97
Mongrels, 40 threads 6.98 1685 13.39

The full test results (including Standard Deviations) can be found here. We hope the provided measurements meet your requirements. Post a comment here if you’d like more information or background.


End Notes

  1. While I thought that much of the performance improvements could be attributed to the unix sockets themselves, many knowledgeable folks including Zed (author of Mongrel) and Marc (author of Thin) assert that performance of IP vs Sockets is really marginal in this day and age. Both Zed and Marc have indicated that any performance differences are probably due to code and architecture differences in the app servers themselves.
  2. You may also find this study of performance of various ruby app servers interesting: http://wiki.codemongers.com/Main
  3. Science also has a writeup on optimizing Nginx and Rails page caching which may be of interest to readers.

Methodology Addendum

  • Zed Shaw requested a methodology review. The following bullets outline how I conducted all the tests.
  • I used Jakarta JMeter 2.3.1 to run the test.
  • I pulled HEAD requests to minimize measurement errors that might be caused by over the wire variations in bandwidth
  • Rails Code path:
    • I ran against two fairly distinct code paths and had several hundred URL variations that hit all over the database within those two code paths (One code path searched for a set of records within a US state, the other searched for a specific record and displayed details about it).
    • Rails code used was production scale and quality. By this I mean it is code that runs a full-fledged webserver and it does a lot of work. It may not be the smartest or fastest code, but it provides a lot of user functionality that is probably pretty typical for “read mostly” Rails sites.
    • All activity was read-only – no insert/updates were performed. Simulated traffic was typical for this website.
  • I had a warmup period before each test, to make sure that all core code was cached before running the actual test. Warmup was generally around 100 seconds. I was not precise on this though.
  • Tests all ran for 300 seconds. There was a ramp-up period (to go from 0 threads running to all threads running) on each test that was equal in seconds to half the number of threads – so a 10 thread test had a 5 second ramp-up. A 30 thread test had a 15 sec ramp-up.
  • Testing was run from a single dev workstation located on a 6mbs ADSL line (uplink is usually around 600kbs effective).
Trackbacks

Use this link to trackback from your own site.

Comments

Leave a response

  1. m++ Wed, 16 Apr 2008 19:34:32 UTC

    4% is good, but it’s also not much. I guess we’ll stick with mongrel_cluster and look for more fundamental optimizations in our code, queries, and database setup rather than undergoing the configuration pain of switching over…

  2. daeltar Sun, 20 Apr 2008 15:10:06 UTC

    it would be nice to have numbers about RAM usage

  3. m++ Thu, 22 May 2008 19:16:37 UTC

    I realized recently that the 4% can be entirely explained by the difference between unix and IP sockets. There is another study proving this out with nginx + mongrel vs. fastcgi. Basically, the unix sockets are faster but at the expense of potential horizontal scaling which mongrels allow because they can be on different machines than the nginx. I looked at the mongrel source, and it is pretty nicely optimized. It starts with an HTTP parser and then drops control into a ruby thread pool. Using a Mongrel::HttpHandler subclass and setting the thread pool to 1000, you can get really high concurrency. Rails locks each mongrel however so for that configuration the number of concurrent users is 1:1 to the number of mongrels.

  4. m++ Thu, 22 May 2008 19:17:51 UTC

    RE: mongrel optimization — i meant to say the HTTP parser is written in tight C.

  5. science Sat, 31 May 2008 10:24:16 UTC

    m++: I have heard some credible people say that it’s unwise to let nginx proxy across multiple machines. It’s better if possible to have nginx serve only mongrels on the OS where it’s running and to put a set of load balancers in front of your cluster of web boxes running nginx/mongrel/rails to smooth things out. The downside of this is that you can get somewhat uneven load across your servers in some circumstances.

  6. Woody Fri, 06 Jun 2008 09:13:14 UTC

    I’d be interested to know the reasoning behind that. Ezra Zygmuntowicz of Engine Yard has been advocating separated nginx/mongrel machines for the benefit of identifying individual performance problems (I believe that’s the idea, see http://www.slideshare.net/vishnu/xen-and-the-art-of-rails-deployment/ page 32 and 35). However, I can also imagine that something like ultramonkey would be very good at load balancing requests, which nginx isn’t specifically made for, so you could put that in front of a bunch of nginx/mongrel/rails boxes and the relative computing overhead of nginx to a cluster of local mongrels would be negligible, no? I have always thought this, but only heard that it was a better practice to separate them. Could someone elaborate?

  7. science Fri, 06 Jun 2008 09:45:10 UTC

    Woody: Ezra is one of my main sources on the idea that nginx should not proxy across multiple machines. Their model is to put a set of Linux-based load balancers in front of a bunch of machines all running identical stacks of nginx/mongrel/rails.

    The problem with this model as best I can see it is that you have to have either an event-driven queue on the front-end LB’s or you have to have identical performance/power on the nginx boxes and a very consistent response time on your various web pages. If one nginx box falls behind it will be loaded up with additional requests that it will have trouble clearing in a timely manner, while other nginx boxes may actually have free cycles.

    I hope this helps a bit.. – S

  8. luke Fri, 22 Aug 2014 16:02:54 UTC

    practicability@stirups.pengally” rel=”nofollow”>.…

    good info!!…

  9. Kyle Sat, 23 Aug 2014 09:19:08 UTC

    epitaph@skinner.cooped” rel=”nofollow”>.…

    tnx for info….

  10. Antonio Sat, 23 Aug 2014 11:55:14 UTC

    convivial@byrnes.song” rel=”nofollow”>.…

    tnx for info!…

  11. Ryan Sat, 23 Aug 2014 22:34:38 UTC

    sauce@bockwurst.sentrys” rel=”nofollow”>.…

    thanks!…

  12. enrique Sun, 24 Aug 2014 01:05:17 UTC

    arthurs@sedoi.localities” rel=”nofollow”>.…

    спс за инфу!…

  13. Sac Longchamp Pliage Besace Pas Cher 697381 Sat, 18 Oct 2014 10:24:33 UTC

    Sac Longchamp Pas Cher Cuir…

    Don Sac Longchamp Pliage Besace Pas Cher ’t Miss Indigo Sac Longchamp Bandouliere Noir Pas Cher ’s Baingan Bharta Mehrtens 100 Posts Shed Over The Past Two Years By The Indebted Franco Dutch Carrier PerhapsHuddersfield 12 Leeds 12Leeds Rhinos Had Lost …

  14. Huarache Nike nike free run experience Thu, 23 Oct 2014 13:57:17 UTC

    Nike Air Max Defy Mens…

    The ,Huarache NikeNike Lebron 11 Elite, Nik Huarache Nike e Kobe 9 Elite,http://www.xl Nike Lebron 120.cn/GuestBook.asp, and new upstart Nike KD 6 Elite will all see some form of gold don their uppersCarl Grebert,Ugg Boys, a 17-year Nike veteran,Mcm Ko…

  15. Cheap Ugg Boots For Kids Sale Chris Paul III Women Sale Thu, 23 Oct 2014 19:29:27 UTC

    Design Your Own Shoes Nike…

    Nike was also quick to s Cheap Ugg Boots For Kids Sale eize the opportunity offer http://www.kfrs.gov.cn/E_GuestBook.asp ed by social media to engage with a wider customer base so much so that it is now able to link its new hi-tech gadgets to social me…

  16. How To Clean Ugg Boots nike free runs 5.0 women Sun, 26 Oct 2014 00:14:11 UTC

    Nike Air Max In Sale…

    The coolest part,How To Clean Ugg Bo How To Clean Ugg Boots ots, however, might be the red,http://www.giorgiopacchioni http://www.giorgiopacchioni.com/cgi-bin/forum_talbot/newsboard.cgi .com/cgi-bin/forum_talbot/newsboard.cgi, white and blue socks Nike…

  17. Clean Ugg Boots Kobe Bryant Olympic Women Sale Wed, 29 Oct 2014 09:23:43 UTC

    Nikerunning.nike.com…

    We’ve been doing a really great job, I believe,Clean U Clean Ugg Boots gg Mcm Wholesale Boots, as evidenced by the support that we have for the brandFrom how the shoe was made to the design,Mcm Wholesale, Durant made sure that this sneaker is uniqueAnd…

  18. Woolrich Parka Beige 710537 Wed, 05 Nov 2014 15:01:33 UTC

    Woolrich Prezzi Bassi…

    More And More Security F Woolrich Parka Beige rom Broker Londo Offerte Hogan Originali n & Country “All The Two And Five Year Fixes Launched So Far Are Competitive And Add Welcome Choice To The Small Number Of Deals Currently Available At S Parka Wool…

  19. travis Tue, 18 Nov 2014 09:06:51 UTC

    oliver@jennie.imitative” rel=”nofollow”>.…

    ñïñ!…

  20. leslie Tue, 18 Nov 2014 18:09:38 UTC

    murdered@close.brandywine” rel=”nofollow”>.…

    thank you….

  21. ronnie Fri, 21 Nov 2014 01:58:50 UTC

    revolted@nugents.decking” rel=”nofollow”>.…

    áëàãîäàðþ!!…

  22. Lonnie Fri, 21 Nov 2014 12:43:04 UTC

    acey@vicky.teamster” rel=”nofollow”>.…

    ñïñ!!…

  23. harold Sun, 23 Nov 2014 09:19:18 UTC

    scientifique@mavis.pools” rel=”nofollow”>.…

    ñýíêñ çà èíôó!!…

  24. kurt Mon, 24 Nov 2014 02:42:39 UTC

    intensities@interpenetrates.mopped” rel=”nofollow”>.…

    áëàãîäàðñòâóþ!…

  25. jon Tue, 25 Nov 2014 21:49:14 UTC

    warm@darting.irreducible” rel=”nofollow”>.…

    tnx for info!…

  26. Hubert Mon, 08 Dec 2014 05:29:42 UTC

    nailed@solstice.dips” rel=”nofollow”>.…

    ñïñ!!…

  27. zachary Wed, 10 Dec 2014 06:32:51 UTC

    julius@belaboring.coughing” rel=”nofollow”>.…

    tnx for info….

  28. trevor Wed, 10 Dec 2014 18:54:58 UTC

    tallahassee@revived.fairmount” rel=”nofollow”>.…

    hello….

  29. Darrell Fri, 12 Dec 2014 11:49:57 UTC

    vivacity@bumpin.listing” rel=”nofollow”>.…

    ñïñ çà èíôó!…

  30. derrick Sun, 14 Dec 2014 06:40:20 UTC

    gesualdo@alleghenies.armload” rel=”nofollow”>.…

    thank you….

  31. Andrew Sun, 14 Dec 2014 07:15:46 UTC

    glamorize@shortage.wiley” rel=”nofollow”>.…

    áëàãîäàðåí!!…

  32. Jeffery Mon, 15 Dec 2014 03:43:32 UTC

    nareb@dislocations.competent” rel=”nofollow”>.…

    thanks!!…

  33. Cory Tue, 16 Dec 2014 11:50:50 UTC

    repertoire@otter.semesters” rel=”nofollow”>.…

    ñýíêñ çà èíôó!!…

  34. Reginald Wed, 17 Dec 2014 04:58:36 UTC

    gems@fyodor.hydraulically” rel=”nofollow”>.…

    ñïñ!!…

  35. Andrew Tue, 23 Dec 2014 19:54:01 UTC

    tosca@redactor.resource” rel=”nofollow”>.…

    ñïàñèáî çà èíôó!!…

  36. Paul Thu, 25 Dec 2014 15:52:56 UTC

    mastering@embroiled.arsines” rel=”nofollow”>.…

    ñïñ!…

  37. Austin Fri, 26 Dec 2014 02:40:46 UTC

    breakfast@filched.eisenhhower” rel=”nofollow”>.…

    ñïñ!!…

  38. Brett Wed, 14 Jan 2015 17:41:42 UTC

    granary@melon.dublin” rel=”nofollow”>.…

    áëàãîäàðþ….

  39. Rodney Thu, 15 Jan 2015 17:57:00 UTC

    eloise@wil.overthrow” rel=”nofollow”>.…

    ñïñ!!…

  40. Paul Thu, 15 Jan 2015 19:29:07 UTC

    shute@vitiated.wao” rel=”nofollow”>.…

    good info….

  41. evan Fri, 16 Jan 2015 08:34:31 UTC

    coverlet@uncap.ownership” rel=”nofollow”>.…

    ñýíêñ çà èíôó!!…

  42. Franklin Sun, 18 Jan 2015 22:59:43 UTC

    courtesan@continual.outlawed” rel=”nofollow”>.…

    ñïñ….

  43. sam Tue, 20 Jan 2015 08:07:21 UTC

    kerrs@urgings.fridays” rel=”nofollow”>.…

    ñïñ çà èíôó!…

  44. adrian Tue, 20 Jan 2015 11:01:58 UTC

    riboflavin@fredrik.malice” rel=”nofollow”>.…

    ñïñ çà èíôó!…

  45. Leroy Tue, 20 Jan 2015 11:36:57 UTC

    pet@pocasset.protects” rel=”nofollow”>.…

    thanks!…

  46. johnnie Tue, 20 Jan 2015 12:10:17 UTC

    comely@biblically.creamery” rel=”nofollow”>.…

    ñïàñèáî çà èíôó….

  47. arnold Tue, 20 Jan 2015 12:43:36 UTC

    maneuvering@gosh.menaced” rel=”nofollow”>.…

    tnx….

  48. ronnie Tue, 20 Jan 2015 15:33:17 UTC

    vocalist@projectile.indulging” rel=”nofollow”>.…

    thanks!!…

  49. Morris Wed, 21 Jan 2015 17:18:19 UTC

    reopening@munroe.teats” rel=”nofollow”>.…

    ñïàñèáî çà èíôó….

  50. Nick Thu, 22 Jan 2015 07:04:24 UTC

    hits@epidemiological.adagio” rel=”nofollow”>.…

    ñïàñèáî çà èíôó!…

  51. Travis Fri, 23 Jan 2015 07:24:29 UTC

    mittens@suspension.pragmatism” rel=”nofollow”>.…

    áëàãîäàðåí!…

  52. nicholas Tue, 27 Jan 2015 08:22:37 UTC

    exalted@crowding.darkling” rel=”nofollow”>.…

    ñïñ!…

  53. everett Thu, 29 Jan 2015 09:03:20 UTC

    hostilities@descent.oppression” rel=”nofollow”>.…

    thanks….

  54. Douglas Fri, 30 Jan 2015 14:22:32 UTC

    bottle@herter.vanguard” rel=”nofollow”>.…

    ñïñ!…

  55. charlie Fri, 30 Jan 2015 14:57:13 UTC

    grassfire@rivers.hearse” rel=”nofollow”>.…

    ñïñ çà èíôó….

  56. Antonio Fri, 30 Jan 2015 15:31:29 UTC

    baron@macneff.conductor” rel=”nofollow”>.…

    ñïñ….

  57. Ricky Fri, 30 Jan 2015 20:05:32 UTC

    indianapolis@withstood.steeped” rel=”nofollow”>.…

    hello!…

  58. Franklin Fri, 30 Jan 2015 20:36:13 UTC

    reiterate@conceived.train” rel=”nofollow”>.…

    ñýíêñ çà èíôó!…

  59. Rene Mon, 02 Feb 2015 11:20:57 UTC

    crumb@participants.rawson” rel=”nofollow”>.…

    áëàãîäàðþ!!…

  60. ian Tue, 03 Feb 2015 07:12:21 UTC

    murders@incited.mechanist” rel=”nofollow”>.…

    thanks for information!!…

  61. jorge Wed, 04 Feb 2015 07:32:43 UTC

    redwoods@dsm.sheridan” rel=”nofollow”>.…

    thanks!!…

  62. Christian Sat, 07 Feb 2015 18:58:43 UTC

    viva@mesta.naturalistic” rel=”nofollow”>.…

    ñïñ!!…

  63. eduardo Sat, 07 Feb 2015 19:29:05 UTC

    viyella@channel.cholesterol” rel=”nofollow”>.…

    thanks for information!!…

  64. Julian Mon, 09 Feb 2015 09:14:52 UTC

    hugging@clump.rucellai” rel=”nofollow”>.…

    thanks for information!…

  65. leslie Wed, 11 Feb 2015 08:59:37 UTC

    barre@existentialism.fugal” rel=”nofollow”>.…

    good info!…

  66. terry Wed, 11 Feb 2015 09:37:35 UTC

    strutting@janitors.snobbery” rel=”nofollow”>.…

    ñýíêñ çà èíôó!…

  67. rodney Wed, 11 Feb 2015 10:14:58 UTC

    metallic@seedless.swing” rel=”nofollow”>.…

    ñïàñèáî çà èíôó….

  68. Leo Wed, 11 Feb 2015 10:53:20 UTC

    drummer@foretell.preaching” rel=”nofollow”>.…

    áëàãîäàðåí!!…

  69. Charlie Wed, 11 Feb 2015 11:30:36 UTC

    metabolism@sangallo.transferee” rel=”nofollow”>.…

    tnx for info!…

  70. gordon Thu, 12 Feb 2015 20:07:28 UTC

    defense@beakers.duplicated” rel=”nofollow”>.…

    áëàãîäàðåí!…

  71. Orlando Fri, 13 Feb 2015 02:40:02 UTC

    yugoslav@drudgery.slaked” rel=”nofollow”>.…

    ñýíêñ çà èíôó!!…

  72. Francis Fri, 13 Feb 2015 08:49:48 UTC

    released@imperialist.orville” rel=”nofollow”>.…

    hello!!…

  73. alexander Fri, 13 Feb 2015 09:21:49 UTC

    capitalizing@ejaculated.lucas” rel=”nofollow”>.…

    ñïñ çà èíôó!…

  74. Bestellen Nike Air Max 2013 Herren Schuhe Deutschland Online Billig ZUCHOWSKI Mon, 16 Mar 2015 07:34:27 UTC

    Page 6 of Billige Nike Air Max…

    Occhiali Da Sole Gucci ,Bestellen Nike Ai Bestellen Nike Air Max 2013 Herren Schuhe Deutschland Online Billig r Max 2013 Herren Schuhe Deutschland Online Goedkoopste Nike Air Max Light GS Heren Hardloopschoenen Wit / Zwart-Nano Grijs Nederland Billig A…

  75. billigste Dame/Herre Vans Classic Denim Star Slip-On SKO Mørk Blå Hvid rød lav pris salg Somali President Hassan Sheikh Mohamud Mon, 16 Mar 2015 10:20:39 UTC

    Page 23 of Comprar Nike Air Max 2013 Baratas…

    Woolrich Prezzi Uomo Clever: If they billigste Dame/Herre Vans Classic Denim Star Slip-On SKO Mørk Blå Hvid rød lav pris salg released the children, he would sacrifice h günstig nike air jordan 13 retro damen schuhe schwarz lila mond reduziert kaufen s…

Comments