Just make both the decimal dot and the E-then-exponent part optional:
[1-9][0-9]*\.?[0-9]*([Ee][+-]?[0-9]+)?
I don’t see why you don’t want a leading [+-]?
to capture a possible sign too, but, whatever!-)
Edit: there might in fact be no digits left of the decimal point (in which case I imagine there must be the decimal point and 1+ digits after it!), so a vertical-bar (alternative) is clearly needed:
(([1-9][0-9]*\.?[0-9]*)|(\.[0-9]+))([Ee][+-]?[0-9]+)?