The secret to making this a success is simple: the right icing recipe. It's like cement. It works perfectly. And then all your candy house dreams can come true :)
Royal Frosting for Christmas Houses (works for gingerbread or graham crackers).
Beat together for 1 minute 1-1/2 large egg whites, 1/8 tsp cream of tarter, dash of salt. (I use hand mixer). Then slowly add 2 cups powdered sugar until well blended. Beat on high speed until very stiff and satiny about 3-5 minutes. Keep bowl covered with wet cloth while using to keep from drying out.
Makes enough for about 3 basic small cracker houses or 1 big gingerbread house. Can easily double recipe with 3 egg whites.