5:00pm Update: Internap has reported back to us with the following information:
- StockCharts.com was not the target of the Denial-of-Service attack. We were "collateral damage."
- The problem happens when the DOS attack swamps Internap's connection to the Cogent backbone (one of several backbones they use).
- People who are accessing our site via that backbone lost their connections pretty much immediately.
- Unfortunately, that slow backbone eventually causes our servers to "fill up" with charts waiting to go out. Once that happens our site stops working for everyone.
- This latest attack lasted much longer (about 1 hour) than the previous ones.
- Internap installed new software and new procedures today to help cope with future attacks.
- Internap is prepared to move us off of the Cogent backbone if this issue happens again.
- Internap is investing in a long-term solution that will completely eliminate this issue in the future. That solution will be in place "in the summer."
We are satisfied that these steps will help mitigate the effects of future attacks. If it does happen again, we will ask Internap to move us off of the Cogent backbone.
12:50am Update: Internap is reporting that everything is back to normal and working again. We don't agree. Because this is the third time this has happened in the past year, we have notified their Senior VP of Technology about this issue. We'll be following up with them aggressively to try and determine why this continues to happen.
11:45am Update: Things are finally starting to work again. We are continuing to monitor the situation and are still waiting for a response from Internap on why this continues to happen. We will post an update again after we hear back from them.
11:20am: Around 10:45am Eastern, we started seeing problems with the delivery of our charts to our users.
At 11:15am, we received the following message from our ISP:
At approximately 07:54 PST our network monitoring system notified us of an increase in traffic over our upstream provider, Cogent, in our SEF PNAP location. The increase in traffic is due to a DoS. Customer traffic inbound over Cogent would experiencing packet loss and latency during this time. At this time we are investigating the issue and will update customers once the traffic has.
We are currently still seeing issues with our service. We have called our ISP about this issue and they are working on resolving it. We apologize for the problems this issue has caused our users. Once service is restored, we will be contacting Internap and asking for a better solution to these kind of situations.