Preheat the oven to 350˚F (175˚C). Spritz an 8-inch square baking pan with cooking spray or rub it with oil. Line the dish crosswise with a piece of parchment paper, leaving an overhang (this will make it easy to remove the cake after it’s baked).
Make the streusel topping: In the bowl of a stand mixer with the paddle attachment (or a large bowl using an electric hand mixer), combine the sugar, flour, and spice blend. Blend on low speed to combine. Add the softened butter and blend until the mixture is damp and clumpy. Scrape the streusel into a bowl and set aside.
Make the gluten-free coffee cake batter: Clean and dry the mixing bowl and paddle. In the now clean bowl, combine the flour, sugar, baking powder, and salt. Blend on low to combine. With the machine running, pour in the melted butter and blend until evenly incorporated (it will be lumpy and still floury). Add the eggs and vanilla, and blend until combined. With the machine running, pour in the milk. Blend just until the batter is smooth (it will be quite thick, but if it's not creamy then add another small splash of milk).
Assemble and bake: Scrape half of the cake batter into the prepared pan and spread it in an even layer. Sprinkle with half of the streusel mixture, breaking it up with your fingers. Scatter chopped dried fruits and/or nuts over top if using. Dollop the remaining cake batter over the filling, then spread it in an even layer. Sprinkle the remaining streusel evenly over the top.
Bake the cake until it’s golden brown and a toothpick inserted in the center comes out clean, about 30 minutes.
Let the cake cool in the pan on a rack for 10-15 minutes. Run a thin knife along the edges, then use the parchment handles to remove the cake from the pan. Return the cake to the rack.
Glaze the cake (optional): In a small bowl, stir together the powdered sugar, melted butter, and vanilla extract. Add the hot water, 1 tablespoon at a time, until the glaze is satiny and pourable. Immediately pour the warm glaze over the cake (you might not need all the glaze).
Let the cake sit until the glaze is hardened. Cut it into slices and enjoy.