The nearest thing to death by old age for a bacterium would be by internal accidents, something happening to a particular molecule or something of that type.

Statistically this is unlikely for any one bacterium, though it certainly must happen from time to time. Remember that though you were born and will die, your genetic material extends indefinitely back (with certain changes of course!) for billions of years (or you would not be here to ask that question!) Most bacteria, most of the time, have at least as long a history behind them as we do, possibly even a history that in one way or another joins ours. Something of a difference is that when a bacterium splits, it is not like a birth and a parent; each of the bacteria could equally validly be regarded as being  the same as the one that had split. If bacteria had memories, any Escherichia coli in your gut would have a memory going back perhaps three billion years...

In such a sense, mortality and old age are tricky philosophical  concepts.

It might not be something to think about too deeply!

Go well,

Jon