We’re currently investigating an issue on the G1 gateway that is causing mail to not be delivered properly. It appears at this time that mail is queuing on the gateway and not being delivered to destination servers. We will be performing a database maintenance on the gateway to attempt to alleviate this issue.

We will update this post as soon as we have additional information.

Update: Database maintenance has completed and mail is flowing smoothly again. Please contact support@purity.net if you have any additional issues.

We are currently performing emergency database maintenance on the MF1 gateway to address an issue that was causing errors in mail delivery.

ETA: Unknown – database maintenance times can vary, though it should be complete within the next couple of hours

Mail Delivery Status: Mail will be queued on the sender’s server and will be delivered once mail services restart after the database maintenance

We will update this post as soon as the issue has been corrected.

Update 9:22PM Central: After further investigation it was determined that mail services on all three of our gateways was impacted for approximately 1 hour and 20 minutes this evening due to issues with the DNS servers that our gateways use to resolve IPs.  This issue caused our gateways to be unable to resolve IP addresses properly causing mail to not be able to be delivered properly.

We have since changed to different DNS servers for resolution, resolving the issue.  MF1 continues to work on its database maintenance but mail is being delivered as it works.  Customers on MF1 may notice a slight delay in delivery times as the database maintenance completes.

During the outage emails destined for customer servers will have queued on the sending servers and should be delivered as those servers retry at a later time.

Our apologies for this situation! Due to the length of the outage we will be crediting all customers 5% of their monthly service charges which will be applied against your next bill. Please contact us if you have any questions.

Mailfoundry engineers have temporarily taken the G1 gateway offline as they investigate an issue causing some mail to get stuck in queue.  We expect services to be restored shortly but currently do not have an ETA.  At this time mail will queue on the sending server and will be delivered after G1 comes back online and the sender retries.

Update: G1 is back up and performing normally.

We are currently performing emergency database maintenance on the G1 gateway to repair a corrupted user database.  We estimate a total time for repair of 30-60 minutes.  During a portion of this time SMTP services on G1 will be disabled.  Sending servers will queue the mail until services are restored and will deliver at that time.

Mail is temporarily being deferred on G1 (g1.ijnet.net) due to the user list index being corrupted.  We are currently rebuilding the index and expect service to be restored shortly.

Mail being temporarily deferred means that the gateway is responding to SMTP requests, but rather than accepting messages is responding with a 451 error to the sending server, asking the server to try again shortly.  Once the user list is rebuilt and mail is being accepted again, senders will be free to retry to deliver mail that was deferred.  No mail will be lost from this issue.

UPDATE: Service has been restored as of 8:30am central US time.

This morning at approximately 7am central time, our MF2 gateway began to experience issues. Although the gateway was up and responding to network requests, it was not responding to SMTP requests. After investigating the issue it appears that the gateway was hit with an extremely heavy dictionary attack, essentially to the point of a denial of service, sent to a domain that was misconfigured and was not rejecting unknown users.

This caused our gateway to accept all mail received for that domain and queue it up in an attempt to deliver it to the destination server. Due to the large volume of mail, it caused the gateway’s processor load to spike, causing the gateway to shut down SMTP services to protect itself.

Once we determined what the issue was, it took us some time to go through the outgoing queue and remove the offending messages from the queue manually to reduce the amount of work the gateway had to do so that processor levels could be reduced enough to enable SMTP again.

SMTP services were re-enabled at approximately 8:30am central time and all mail should be flowing normally. No mail will have been lost during this outage, though some mail may be delayed past the point where services were restored due to the retry settings on the sender’s mail server.

We apologize for this outage and will be discussing internally ways to prevent something like this from happening again. The most likely outcome of this will be that we will fully activate the automated catch-all checking and disabling, which is designed to disable domains that accept mail for all users like this. We’ve put off doing so several times in the past due to customers expressing concern about it, but I believe it’s time to go ahead and implement this for the sake of all of our users.

All customers with service on MF2 will be receiving a 5% service credit due to this outage.