How do we know time travelers aren't constantly changing the past?

How do we know time travelers aren't constantly changing the past?: "We take it for granted that the past is fixed. History always happened the way we remember it happening. But how do we know for sure that that's the case? If time travel is possible, time travelers could be zipping back and changing things around all the time. How would we even be able to tell if that was the case?"