Own Web Now Corp Network Operations » Exchange Hosting

Dewey Upgrade

travis — Tue, 02 Mar 2010 22:08:24 +0000

Tonight at 9:00 PM Eastern we will be starting extended maintenance on the Exchange 2007 server DEWEY. During maintenance, access to the DEWEY server will be limited and service is expected to be restored by 11:00PM Eastern. No mail will be lost during the maintenance window.

Update 8:00 PM Eastern: We will be moving the maintenance to 8:30 PM to alleviate the latency issue with DEWEY sooner.

Update 10:05 PM Eastern: We will be extending maintenance on DEWEY as we migrate users to the newer database. Mail destined for users on DEWEY is being queued by ExchangeDefender and any users needing immediate access to their live mail should login to LiveArchive at https://livearchive.exchangedefender.com using their ExchangeDefender credentials.

Update 7:13 AM Eastern: Maintenance and optimization has been completed on DEWEY.

Emergency Reboot DEWEY

travis — Tue, 02 Mar 2010 18:35:06 +0000

Due to use complaints about access issues, we’ve had to conduct an emergency reboot on DEWEY to increase the resources available to the server. Service is expected to be restored in under 15 minutes.

Update 1:52 PM Eastern: The server came up from the first reboot, but the changes did not apply properly. The server is going down for another reboot to attempt to apply the changes.

Split GALs Australia

travis — Mon, 22 Feb 2010 15:25:12 +0000

Tonight (2/22/10) around 6PM Eastern we will begin to implement split GALs on the Australia Exchange 2007 server. Between 6PM and 8PM Eastern there is expected to be brief interruptions in service as the new security policies are implemented.

Dewey BES

travis — Mon, 15 Feb 2010 15:40:01 +0000

The Dewey BES service has expierenced a few random issues since Microsoft patching on the 10th. In an attempt to revert the changes, we’re in the process of removing the updates before restoring from a backup. Service on DEWEY BES will not be interrupted throughout the extended maintenance of the BES server.

Update 10:50 AM Eastern: The BES service for Dewey has been restored 100%. We found an issue with two security updates, including KB977914

Dewey access issues

travis — Fri, 05 Feb 2010 14:09:50 +0000

We’ve had reports from a few clients that they are having issues accessing our Exchange 2007 server DEWEY. We are currently investigating into this issue.

Update 11:20 AM: Service on DEWEY has been running 100% over an hour. It seems close to 10 PM Eastern on 2/4/2010, IIS had issues compiling the metadatabase..which ultimately did not throw any errors in windows logs or in any of our monitoring software. We ran a quick repair on the metadatabase soon after being notified of service outage and restarted IIS which resolved the issue.

DEWEY Reboot

travis — Mon, 18 Jan 2010 21:02:21 +0000

At 4:10 PM Eastern we will be rebooting DEWEY for feature enhancements and more analytical monitoring. The reboot is expected to last no longer than 10 minutes and service should be 100% functional by 4:20 PM Eastern.

HUEY Reboot

travis — Thu, 14 Jan 2010 20:14:15 +0000

We are about to reboot huey.exchangedefender.com due to user accessibility complaints. Service is expected to be fully impacted on HUEY for the 15 minutes while the reboot commences.

Update 3:21 PM Eastern: The server is back online from the reboot and service has been restored.

Huey Certificate

travis — Wed, 13 Jan 2010 19:28:37 +0000

We are in the process of replacing the certificate for HUEY in our Exchange 2007 cluster. We highly apologize for the inconvenience however service should be restored before 3pm Eastern.

Updated 2:42 PM Eastern: The certificate has been successfully replaced on HUEY. Service on HUEY is now 100% operational.

Huey Reboot

travis — Fri, 11 Dec 2009 14:59:54 +0000

We’ve had reports from a few partners that RPC over HTTPs is not connecting on HUEY. We are rebooting the server to rectify the issue.

Update 10:12 AM Eastern: Service has been restored and we have confirmed RPC over HTTP is functioning properly.

Server reboot HUEY

travis — Fri, 20 Nov 2009 17:07:04 +0000

We’ve had a couple reports of users unable to connect to HUEY through RPC. We’ve restarted the IIS service but some users are still reporting issues. The HUEY server will be going offline in a couple minutes for a service reboot. The server is estimated to be down for 15 minutes during the reboot.

Update 12:04 PM Eastern: The server is now going down for the reboot.

Update 12:16 PM Eastern: The server has returned from the reboot and service has been restored.

Daisy SSL Renewal

travis — Wed, 11 Nov 2009 03:32:57 +0000

We are about to replace the SSL certificate on our 2003 server Daisy. During the replacement RPC over HTTPs may be unavailable but will be restored shortly.

Update 10:39 PM Eastern: The SSL has been replaced on Daisy and service has been restored 100%.

Stress test HUEY

travis — Sun, 18 Oct 2009 23:10:54 +0000

At 7:10 PM Eastern we will begin stress test maintenance on HUEY that is scheduled to last until 10PM Eastern. We are performing click tests and harddrive upgrades. There will be periods of inaccessibility through OWA and Outlook.

Update 2:07 AM: Maintenance has completed and the database has been moved to the new drive. Service has been restored 100%.

Update 937 PM: The hard drives are in place however the copy is taking longer than expected. New mail is being queued so there will be no lost emails during this maintenance.

Australia Exchange

travis — Thu, 15 Oct 2009 03:02:37 +0000

The Australian exchange server is currently offline for network upgrades from 00:00 – 04:00 GMT +10

Due to increased network activity over the past several month in the data center, Servers Australia technicians will be replacing several Fast Ethernet Interfaces with Gigabit Ethernet and Multimode Fiber interfaces in the Sydney Core Routers.

The maintaince upgrades of our entire data center network within Equinix Mascot and the SAU Data Center at Tuggerah is to provide a more reliable and responsive experience.

Service Maintenance on HUEY

travis — Fri, 09 Oct 2009 00:20:17 +0000

We have received a few complaints about outlook not updating on HUEY. We will be restarting the services momentarily and if a reboot is required, this post will be updated.

Exchange server offline

travis — Thu, 24 Sep 2009 19:45:45 +0000

HUEY and DEWEY are currently offline. While working with Microsoft to troubleshoot an authentication issue, the servers went lights out. We’ve requested a reboot at the DC an service is expected to be restored within 15 minutes.

Update 4:57 PM: Service has been restored.

Update 4:36 PM: The server took a bit longer than expected to recover from the reboot, however the information store timed out when mounting. We are rebooting the server again to address the issue.

Exchange Server Reboots

travis — Mon, 21 Sep 2009 15:54:53 +0000

We will be rebooting two of our US Exchange servers at 8:00 PM Eastern. HUEY and DEWEY will be unavailable for 15 minutes during the reboot.

Update 8:20 PM: Service on Dewey has been restored.

Update 8:13 PM: Huey’s services have been restored. Dewey hasn’t rebooted yet.

Update 8:05 PM: The reboot process has begun. Service will be restored momentarily.

Exchange Server Australia Down

travis — Thu, 17 Sep 2009 20:53:30 +0000

Our Australian Exchange 2007 server is currently down hard. The datacenter is experiencing a networking issue and we are working to resolve the issue.

Update 6:15 PM Eastern: The server is back online. The issue was caused by the network ISP change at the data center and the downtime was unexpected. We will be changing the IP address of Australia to match the new network configuration. This change will not require any modification by our partners.

Exchange Server Reboot

travis — Tue, 15 Sep 2009 14:38:00 +0000

We will be rebooting our US Exchange Server Scrooge today to relieve resource usage on the server. The reboot time has not been established yet however this post will be update when a time has been set.

Update 12:25 AM: Scrooge has been rebooted and is back online.

Update 5:22 PM: We have delayed the scheduled reboot until Midnight tonight due to concerns of unavailability by our West coast customers.

Update 2:15 PM: We have delayed the scheduled reboot until 9pm tonight in order to run extra maintenance on the server.

Update 10:50 AM: We have scheduled the reboot time to occur at 5pm Eastern today.

Australia server outage

travis — Tue, 08 Sep 2009 16:52:31 +0000

The Australian exchange server is currently unavailable. There is an issue with the datacenter and it has gone beyond the normal 1 hour SLA. We are doing everything in our power to restore service immediately.

Update 1:34 PM Eastern: The Australian server is back online. The issue was caused by a failure in the core network of the datacenter. Service is now 100% operational.

Australia Server IP Change

travis — Thu, 03 Sep 2009 20:55:55 +0000

On Friday, September 4th 2009 at 10:00 AM Eastern we will be changing the IP address for our Australia Exchange server due to changes in the network layout in the data center. Before the change we will lower the TTL time on the DNS record (exchange.australia.exchangedefender.com) to minimize the chance of downtime/unavailability.

Users on the Australia server should not have to make any changes after the IP address change for service to continue working.

Update 12:10 PM Eastern: The IP change has been delayed until later today. We will update this post before the change occurs.