Select the button base layer and do: Layer->Merge Visible. This will combine all the layers except the background layer. Turn off the background layer.