It won't be terribly noticeable in either case. It MIGHT go a little bit deeper after some time, but it will not get any louder - that's a parameter that is independent of suspension stiffness. If the enclosure is built to the proper specifications, then you have what you are going to get.
You may need to go to different woofers, or a different enclosure. The polyfill actually hampers output, by the way, and if you built your enclosure too small, and added that to compensate, then you DEFINITELY need to build a proper enclosure.
My personal opinion is that overall, "break-in" is a myth, but there are two schools of thought... clicky... I believe that the differences you might hear are all in your head, I've never been able to actually PROVE there are ANY electromechanical differences in response after a given period of time.