Others may chime in. But at 41 years old. Total Testosterone of 280 to 430 is pretty dismal.
Did they test SHBG. With total T ans SHBG you can calculate free T.
Free t is the active form. So even if you have plenty of total T, if the majority of it is bound up by SHBG, you may have low active free Testosterone available leaving you symptomatic. Men typically report having symptoms when free Testosterone is about 10.
T levels decline with age, so things are only likely to get worse. And you have already said that your symptoms have harmed relationships and have affected your life negatively. So Iím not quite sure what your hesitation is about starting testosterone?