Cut the gingerbread into 1-inch cubes. Arrange half of the cake cubes in the bottom of a trifle dish or in individual glasses or bowls (I use eight 12-ounce glasses). Spread half of the pudding over top, along with half of the whipped cream and half of the chopped toffee. Arrange the remaining cake over top, followed by the rest of the pudding and whipped cream. Garnish with the remaining toffee.