On November 12th, some customers had trouble accessing ChargeOver.
This was a networking issue at the primary data center ChargeOver is hosted within.
On 12 November at 10:26 CT, ChargeOver began receiving automated reports of connectivity issues.
The issue was tracked down to a networking issue at the data center ChargeOver’s primary location is within (Flexential in Chaska, MN).
The problem was immediately escalated to Flexential engineers who identified an internet circuit at Flexential’s Chicago POP as the source of the issue.
Flexential opened a ticket with the circuit provider who confirmed there was an issue with packets over a certain size traversing the device due to an MTU misconfiguration. The circuit provider confirmed that they fixed the issue, however Flexential kept production traffic off the link while more investigation is performed.
Total impact to service was around 41 minutes.
ChargeOver is working with our data center provider Flexential to improve monitoring and connectivity.