As I did the final step of attaching fabric B cap over the fabric A cap, I found that the cap had too much fabric as I was attaching it to the brim and I had to gather it a bit to fit it...I am wondering what I did wrong and how I can fix it on a future hat...