We are currently conducting an out-of-band security patch on mail1.ownwebnow.com network. We regret to make this modification during business hours but the scope of security issue was wide enough to warrant immediate action.

Service has been restored to 100% at 4:48 PM EST.

We are conducting some routine maintenance on our Windows portion of the Virtual Hosting and Web Hosting network. We are applying patches, installing new hardware, general system maintenance task. All systems should be affected by a brief outage, and will be back within an hour at most.

Update: Maintenance cycle completed.

We are currently tracking accessibility problems on mail1.ownwebnow.com. Please stand by while we research the issue, the server appears up but several customers are reporting access issues, we are trying to resolve them right now.

We will update this site as soon as we have more information. The cluster is currently undergoing a reboot.

Update: All issues have been resolved, a scheduler service hung on the load balancer.

We are currently working with RoadRunner (formerly Time Warner, AOL) service provider in United States, they are experiencing issues with their SMTP servers and randomly rejecting SMTP traffic. Currently mail is flowing through but some is bouncing back from them due to a reason they are still trying to narrow down. We will update when we have further information or a resolution.

This issue affects our entire global network, and some external sites we have tested.

Update: 6:34 PM EST: Even though we have not been officially updated, the problems with RoadRunner appear to have been resolved. 

From approximately 10 AM - 2 PM EST, our shared mail hosting platform (mail1.ownwebnow.com) suffered a large scale distributed denial of service attack (DDoS). Everything is under full control now and we have been able to filter out the offending systems. Unfortunately, there is little that can be done in terms of scale and protection against a DDoS as we already have both Cisco and Tipping Point in place.

DDoS attacks tend to be flared up by the regular user activity, as the system slows down end users keep clicking Send/Receive and effectively flood the connection until it times out. Systems are back to normal and messages are starting to arrive in regular sequence.

Our virtual hosting network (mail1.ownwebnow.com) is experiencing performance issues at the moment. We are troubleshooting the problem but a performance profile has been unusually low since about 4 PM EST (GMT -5).

We will update this item as soon as the issue has been identified. In the meantime, we have added some extra horsepower to the cluster to keep it moving.

The issue is classified as critical due to the potential mail loss that can occur when the system loads across the cluster spike beyond a certain level and shut down the inbound SMTP channel. We do not expect this to happen but are working under the assumption that the further performance degredadation will shut the network down.

We are working on isolating and eliminating the mailbomb that is currently targeting mail1.ownwebnow.com customers. Mail deliveries are currently delayed but we expect to have this issue filtered out shortly.

We are currently power cycling the entire web hosting cluster powered by Windows 2003. A critical security issue has been identified in one of the ASP.NET component libraries our customers rely on heavily and a reboot was required.

The system is currently being cycled and should be back in a moment.

We are currently waiting the delisting interval for mail1.ownwebnow.com (shared virtual hosting email) off SpamCop XBL (CBL). The delist was put in place around 10 AM EST and will be removed within the hour, so by 11 AM EST there should be no issues.

This issue only affected certain virtual hosting customers, as always, we advise all our customers to use local SMTP servers provided by their ISPs.

We are currently tracking the issue on the virtual hosting email server related to an AT&T blacklist. Sample error:

—– Transcript of session follows —– … while talking to aln-mailrelay.att.net.:
>>> MAIL From: SIZE=22519
<<< 550-65.99.192.3 blocked by ldap:ou=rblmx,dc=worldnet,dc=att,dc=net
<<< 550 Blocked for abuse. See
http://www.att.net/general-info/rblinquiry.html
554 5.0.0 Service unavailable
… Deferred: Connection timed out with xyz

We have reported the issue to AT&T and are awaiting their response. According to their postmaster the response will be received within two days:

Thank you for submitting your inquiry. A response will be sent within two business days to the e-mail address you specified.

We’re sorry about the inconvenience and will update the ticket when the issue is resolved. Currently this is having a minimal impact to a single address on our virtual mail hosting network (if you absolutely must get the email through just try sending it again and it might hit a different gateway).

Update from AT&T at 4:30 PM EST:

Thank you for contacting the Postmaster/Abuse team at ATT.NET and BELLSOUTH.NET.

The mail-server IP address(es) associated with your request are NOT CURRENTLY BLOCKED from sending mail to any of our servers.  You may have received this message for any of the following reasons: the IP address you submitted has already been removed due to another request; the IP address you submitted is incorrect and the actual blocked IP address was not removed; the IP address you submitted has never been blocked at ATT.NET and BELLSOUTH.NET.  ATT.NET and BELLSOUTH.NET, its affiliates, and network services customers do NOT intentionally block legitimate mail in the course of our anti-spam initiatives.  We regret any inconvenience this may have caused.

At approximately 6 AM EST. we started investigating an issue on a shared mail server network, with one of the storage arrays in critical condition. New hard drives have been placed in the system and the array has been rebuilt, service has been restored as of 7:10 AM EST.