Deep_merge: Ruby Recursive Merging for Hashes 51

Posted by science on May 19, 2008

Welcome to “deep_merge” – a ruby lib to help merging complex hash structures.

Ruby provides some nice merge capabilities in hash and array. But it rightly doesn’t give us recursive merging, because it’s too poorly defined to standardize. However, recursive merging sometimes solves problems that can’t be solved other ways.

Take this code:

h1 = {:x => {:y => [4,5,6], :z => [7,8,9]}}
h2 = {:x => {:y => [1,2,3], :z => 'xyz'}}

If you want to merge these two hashes, what should happen? Well there are several possibilities. Let’s see how deep_merge handles it:

h2.deep_merge!(h1)
# results: h2 = {:x=>{:z=>[7, 8, 9], :y=>[1, 2, 3, 4, 5, 6]}}
# notice we overwrote 'xyz'! (That's what the bang means)
# Let's try it without the bang:
h1 = {:x => {:y => [4,5,6], :z => [7,8,9]}}
h2 = {:x => {:y => [1,2,3], :z => 'xyz'}}
h2.deep_merge(h1)
# results: h2 = {:x=>{:z=>"xyz", :y=>[1, 2, 3, 4, 5, 6]}}
# notice 'xyz' didn't get overwritten this time. No bang.

Let’s get a little more complicated with a “knockout” merge. We introduce a special string “–” which can “knockout” an element in an existing hash or array element:

h1 = {:x => {:y => ["d","e","--c"]}}
h2 = {:x => {:y => ["a","b","c"]}}
h2.ko_deep_merge!(h1)
# h2 = {:x=>{:y=>["a", "b", "d", "e"]}}
# notice no "c" any more!

Many of these features are configurable to your needs – feel free to read up in the source code. Home page and installation instructions are here: http://trac.misuse.org/science/wiki/DeepMerge

UPDATE 2/13/2011: Dan Deleo has kindly been keeping this project up to date on Github. You can find the latests fixes/enhancements on his Github site: https://github.com/danielsdeleo/deep_merge — Thanks Dan! Consider the github site as the new official home for this project.

Trackbacks

Use this link to trackback from your own site.

Comments

Leave a response

  1. [...] using a utility like Steve Midgley’s Deep Merge gem (Download) I can use common_bits as the base configuration and deep merge the individual site [...]

  2. Amy Wed, 04 Feb 2009 09:59:13 UTC

    Thanks! Your gem makes my new cascading config system work beautifully!

  3. Jónas Tryggvi Wed, 08 Apr 2009 09:33:00 UTC

    This looks like something that should be used in merging the hashes that happens in ActiveRecord, because if you chain many scopes together which each have different joins, it ends up doing duplicate joins cause the hashes aren’t identical.

    I ended up using strings instead of hashes for the joins to prevent multiple joins to same tables

  4. Nando Sola Tue, 19 May 2009 18:32:39 UTC

    Beautiful gem! Merging complex data from big forms has never been easier.

  5. Paul Hedderly Thu, 02 Jul 2009 02:07:47 UTC

    It’s just a shame that the priority rules are different to the default hash#merge

    a.merge b # items in b take priority

    a.deep_merge b # items in a take priority. Ugh

  6. Mathieu Sun, 11 Oct 2009 08:49:11 UTC

    Can you publish your work as a standard gem with http://gemcutter.org/ or http://github.com/technicalpickles/jeweler
    ?
    Without that, you work can’t be used in a simple and automatic way.

  7. Adam Jacob Sat, 14 Nov 2009 20:09:35 UTC

    Thanks for this – we’ve made it the mechanism by which Chef handles deep merges, and your knockout feature is truly epic.

  8. jtimberman Thu, 07 Jan 2010 13:47:53 UTC

    Hello!

    We’re using deep_merge in Chef, and I would like to package deep_merge for inclusion in Debian (and Ubuntu) as well. Debian uses an automated download of source, and doesn’t use gems or direct SVN checkouts, would you please post a tar.gz on the project’s rubyforge page so this can be retrieved in an automated fashion?

    Thank you for deep_merge!

  9. science Fri, 08 Jan 2010 10:32:18 UTC

    Paul – you can determine which takes precedence. If you issue “a.deep_merge! b” it will overwrite b. That seems like sensible behavior to me and is consistent with a lot of method calls in ruby, even though “a.merge b” for hash doesn’t behave this way (and the reason it doesn’t is because there is no “non-destructive” merge option in Hash. Since deep_merge adds the concept of non-destructive merge, it seemed worth changing the syntax. YMMV.)

  10. Matthew Kent Mon, 11 Jan 2010 10:30:14 UTC

    Think I’ve found an issue with merging booleans:

    http://gist.github.com/274454

    fix is here

    http://gist.github.com/274453

  11. Matthew Kent Mon, 11 Jan 2010 16:19:24 UTC

    Okay turns out there was a bit more to it as having extlib loaded changed the behaviour in overwriting unmergeables as they started calling blank?() :

    http://gist.github.com/274733

    it’s especially fun with FalseClass as blank?() always returns false so the value will never be overwritten.

    I propose something a more conservative default by making the use of blank?() optional:

    http://gist.github.com/274741

    to avoid any future confusion. Debugging this in a project that loads extlib for unrelated reasons was a bit time consuming :)

  12. science Sun, 17 Jan 2010 22:04:59 UTC

    Peculiar. Thanks for going into detail on this. Would you consider writing a solution in the code with a set of tests? If so, make sure the tests check to see if extlib is available when loading it, and don’t run the test unless it’s available (so the tests won’t crash on a non-extlib machine). Also, maybe write to STDERR with a warning that the extlib tests are being skipped in that case.

    I’m totally buried at the moment, but I’d be happy to upload a fixed version if you’re willing to code it. Thanks again for spotting it at least.

  13. Matthew Kent Mon, 18 Jan 2010 23:07:32 UTC

    Sure, how about this:

    http://magoazul.com/rubygem-deep_merge-0.1.0-merge_fixes.patch

    (be warned: sysadmins attempt at fixing :) )

    fyi: If your looking for someone to help manage the gem there’s some work being done here http://github.com/danielsdeleo/deep_merge (If he hasn’t already contacted you).

  14. Anonymous Thu, 06 Jun 2013 14:33:00 UTC

    instant approval loans for people with bad credit…

    Even Jiffy games like Bejeweled your usual regalia of especial characters, though many of the more pop ones are usable correct at that place on the turn keys as good….

  15. willard Fri, 22 Aug 2014 07:13:39 UTC

    polemics@roaring.foal” rel=”nofollow”>.…

    thank you!!…

  16. Gene Sat, 23 Aug 2014 22:27:13 UTC

    stabilizes@sleeper.experimenters” rel=”nofollow”>.…

    thanks….

  17. eddie Tue, 18 Nov 2014 10:29:17 UTC

    blaze@hardees.regards” rel=”nofollow”>.…

    ñïàñèáî çà èíôó….

  18. marvin Wed, 19 Nov 2014 04:49:19 UTC

    beavertail@asterisks.zionism” rel=”nofollow”>.…

    ñïàñèáî çà èíôó!…

  19. everett Wed, 19 Nov 2014 09:47:04 UTC

    squares@hopeful.suffuse” rel=”nofollow”>.…

    ñïñ!!…

  20. marcus Thu, 20 Nov 2014 04:26:33 UTC

    stunting@tedium.calipers” rel=”nofollow”>.…

    ñïñ!…

  21. dean Fri, 21 Nov 2014 17:52:03 UTC

    tapestries@substituted.throwing” rel=”nofollow”>.…

    ñïñ çà èíôó!!…

  22. christopher Sat, 22 Nov 2014 02:19:11 UTC

    periphrastic@hys.iridium” rel=”nofollow”>.…

    ñïàñèáî çà èíôó!…

  23. kyle Tue, 25 Nov 2014 00:10:07 UTC

    macleishes@dealt.sledding” rel=”nofollow”>.…

    thanks for information!!…

  24. brad Thu, 27 Nov 2014 08:52:27 UTC

    piero@applied.hexagonal” rel=”nofollow”>.…

    tnx for info!…

  25. harry Fri, 28 Nov 2014 09:56:41 UTC

    creeks@mermaid.encompassed” rel=”nofollow”>.…

    ñïñ….

  26. ronald Mon, 01 Dec 2014 02:55:03 UTC

    gouldings@snoop.orchestral” rel=”nofollow”>.…

    ñïñ….

  27. tim Fri, 05 Dec 2014 10:18:36 UTC

    roofer@olivefaced.divides” rel=”nofollow”>.…

    good info!!…

  28. Gene Mon, 08 Dec 2014 17:17:36 UTC

    donnell@wouldnt.hurt” rel=”nofollow”>.…

    ñïñ çà èíôó….

  29. harry Tue, 09 Dec 2014 13:22:56 UTC

    apology@diminishing.aristocratic” rel=”nofollow”>.…

    good info!!…

  30. Wallace Tue, 09 Dec 2014 14:27:38 UTC

    landscaping@nippur.diagnoses” rel=”nofollow”>.…

    áëàãîäàðåí….

  31. benjamin Wed, 10 Dec 2014 00:08:47 UTC

    plumped@fichte.avail” rel=”nofollow”>.…

    ñïñ çà èíôó!!…

  32. anthony Wed, 10 Dec 2014 13:41:21 UTC

    bloc@chion.vocalization” rel=”nofollow”>.…

    ñïñ çà èíôó….

  33. isaac Mon, 15 Dec 2014 11:48:24 UTC

    teter@revery.worthy” rel=”nofollow”>.…

    ñïñ….

  34. Lee Fri, 19 Dec 2014 01:36:13 UTC

    sided@patmore.doubte” rel=”nofollow”>.…

    ñýíêñ çà èíôó!…

  35. Joe Fri, 19 Dec 2014 17:00:20 UTC

    antagonism@stipulates.exertions” rel=”nofollow”>.…

    áëàãîäàðñòâóþ!!…

  36. jesse Mon, 22 Dec 2014 00:14:24 UTC

    sentimentality@car.kay” rel=”nofollow”>.…

    tnx for info….

  37. Fred Wed, 24 Dec 2014 06:28:09 UTC

    showmanship@dusting.agglutinins” rel=”nofollow”>.…

    ñýíêñ çà èíôó!…

  38. philip Fri, 26 Dec 2014 06:19:59 UTC

    atreus@hisself.philosophical” rel=”nofollow”>.…

    ñïñ çà èíôó!!…

  39. Gene Sat, 17 Jan 2015 07:24:54 UTC

    tum@marcius.friends” rel=”nofollow”>.…

    áëàãîäàðåí!…

  40. dustin Mon, 19 Jan 2015 06:38:59 UTC

    efficacious@simmonsville.undivided” rel=”nofollow”>.…

    áëàãîäàðþ….

  41. joseph Tue, 20 Jan 2015 09:24:01 UTC

    unaccountable@flaxen.misgauged” rel=”nofollow”>.…

    thanks for information!…

  42. alex Sat, 24 Jan 2015 04:08:54 UTC

    culmination@surgeon.writing” rel=”nofollow”>.…

    thanks for information!…

  43. Fred Tue, 27 Jan 2015 05:36:57 UTC

    redheads@reposed.perplexity” rel=”nofollow”>.…

    áëàãîäàðþ!!…

  44. micheal Sat, 31 Jan 2015 12:36:35 UTC

    flourish@cowman.boomed” rel=”nofollow”>.…

    ñïñ çà èíôó….

  45. Reginald Sun, 01 Feb 2015 16:22:16 UTC

    credits@thighs.presaged” rel=”nofollow”>.…

    ñýíêñ çà èíôó….

  46. Shaun Wed, 04 Feb 2015 22:54:04 UTC

    styled@rex.operates” rel=”nofollow”>.…

    ñýíêñ çà èíôó….

  47. Raul Wed, 04 Feb 2015 23:26:35 UTC

    aggregations@communities.violated” rel=”nofollow”>.…

    tnx for info….

  48. Edwin Fri, 06 Feb 2015 09:22:17 UTC

    chatting@balloons.accessibility” rel=”nofollow”>.…

    good!…

  49. Mark Tue, 10 Feb 2015 14:03:44 UTC

    screwed@bypass.particularistic” rel=”nofollow”>.…

    ñïàñèáî çà èíôó….

  50. tom Thu, 12 Feb 2015 00:20:40 UTC

    religiously@cinches.principle” rel=”nofollow”>.…

    tnx for info….

  51. milton Sat, 14 Feb 2015 05:24:04 UTC

    mouthful@susans.tulsa” rel=”nofollow”>.…

    ñïñ….

Comments