What you do within your 4 walls doesn’t deserve anyone’s opinion. Of course cross dressing is a fancy to some men. They do get a fair amount of pleasure when they don woman’s underclothes or normal clothes, so nothing is abnormal here. You’re 35, so are you married, does your wife know your fantasy, if no, try to tell her your fantasy as this might light up your sex life to a great extent.
As for buying a correct fit, I suggest you take your size by manually taking your measurements by hand and then making a purchase for you to feel good in your new bra.
There will be absolutely no health issues by merely cross dressing, so enjoy and have a good time in your fantasy. Feeling to don female clothes exposes the hidden feminine side and this is absolutely no issue or problem.