Salesforce has resolved a problem with one of its instances in North America that halted access for several hours. However, several core functions remain unavailable as the system takes time to come back online.
The firm said that the problem began at 12:41 UTC on 10 May and was resolved at 9:30 UTC on 11 May.
However, while the service is back Salesforce warned that customers in the region will continue to experience problems.
"The NA14 instance continues to operate in a degraded state. Customers can access the Salesforce service, but we have temporarily suspended some functionality such as weekly exports and sandbox copy functionality. In addition, search indexing may be delayed," the firm said.
The initial outage prompted apologises from CEO Mark Benioff on Twitter after the firm revealed on its status website that a problem with the NA14 region left customers unable to access services.
Salesforce explained that a database failure with the NA14 instance created a “file integrity issue”.
“The Salesforce Technology Team has been working around the clock with our vendors to pursue multiple paths to repair the file integrity issue, which we believed to be the fastest and most reliable path to recovery,” the firm said.
Despite this, Salesforce does not believe it can fix the fault and has reverted to a backup that was saved before the problems occurred.
“To bring NA14 back to full health, we have shifted our focus to recovering from a prior backup, which was not affected by the file integrity issue,” the company said.
The situation has left customers very unhappy, and many have appealed to CEO Marc Benioff directly on Twitter.
Benioff responded to several personally asking them to email him so that he can contact them, probably just to offer more apologies rather than fixing the problem over the phone.
@jbrianlynch email me and I'll give you all the details CEO@salesforce.com— Marc Benioff (@Benioff) May 10, 2016
The incident underlines the problems with cloud-based services that can be knocked offline and leave customers with little recourse beyond sitting and waiting.
However, as former Peterborough City Council IT chief Richard Godfrey told V3 during the V3 Cloud Summit, Salesforce's huge resources mean that the firm can often resolve a problem far more quickly than could a small in-house IT team.
Just spent a year working on them? Too bad, Intel's lost interest
Sony factory in Wales now making 100,000 Raspberry Pis every week
38-year-old Alexander Vinnik faces up to 55 years in jail
Threadripper also available from today if you want a lot more power - but you'll have to wait for the motherboards to appear