Tuesday, July 6, 2010

Cloud computing upgrade info

Update: July 26th, 2010: VmWare is having difficulties converting 75% of the servers to the cloud due to errors (perhaps in their conversion software).  So our vendor (Peak 10) is investigating a 3rd party product to do the conversions.  More information will be posted here once we know more.



Update: July 7th, 2010: My apologies for the problems with uploads/downloads on vWorker this morning. Very late last night (our time) several machines were converted to the cloud, but a few of them did not "take". So the technician at peak 10 (who is doing the conversions) switched us back to the physical machines. Unfortunately, he made a mistake and did not correctly switch on one of them which provided a root/directory required to allow downloads to work. This broke downloads/uploads overnight. When vWorker staff came in early this morning, they saw the problem and instructed peak 10 to turn back on the physical server. I just spoke with the technician and he and his team have been instructed as the importance of double checking all work, as well as working more slowly (doing less servers at a time) to prevent issues like this from occuring again. Also, rather than doing conversions of important servers in the middle of the night (when mistakes can occur), the cutovers will be done earlier in the night. I apologize as this was an inconvenience. It is a non-trivial task to convert over so many servers and we may continue to have a few more unexpected outages during this conversion process (for example, some of the Microsoft virtual machines are not converting well...possibly due to a driver issue/bug). However Peak 10 is now taking all steps possible to minimize those outages.

---------------------

My apologies, as there was a site outage on July 5th from about 8AM EST to about 1:30PM EST. Protectpoint, our firewall provider, had a technical issue which caused the site to become inaccessable. They fixed this issue around 1:30PM EST and are watching the machines more closely in case a repeat occurs.

Also, we have begun the first migration of servers to a cloud computing/virtual environment that will result in better performance and reliability. Upgrades are being performed from 6pm EST - 9pm EST throughout this week and will result in 5-30 minute interruptions each day as switchovers of systems occur. Again, my apologies for the inconvenience as we make these improvements.

Ian Ippolito
vWorker

11 comments:

SUNRISE said...

We need a faster site. We appreciate that you and your workers are working on it. Thank you for giving us a chance to be a vWorker. We hope the system will be upgraded soon.

Faizan Aslam said...

Hi Ian,

That time was very horrifying. Because it never happened before. I kept clicking refresh button for hours. Then I changed my ISP but no use.

Thanks God its back, I wish it will not happen again.

Best Regards,

Faizan.

Faizan Aslam said...

Hi Ian,

That time was very horrifying. Because it never happened before. I kept clicking refresh button for hours. Then I changed my ISP but no use.

Thanks God its back, I wish it will not happen again.

Best Regards,

Faizan.

Srini said...

Hi Ian

Greetings. Thanks for the update. Do you also have a site updates strategy for the outage period work not being reflected when the site is live again.

I mean like 3rd party sites working with you as vWorker.com partners working with you might have some technical issues with you. So you plan to look into these things as well.

Thanks
-Srini
www.SEOFastGuide.com

musicnowandever said...

Hi. There are still some problems when trying to post messages and attachments in vWorker.com . Also, i cannot download files that were posted by me earlier to clients. Thank you. Looking forward to hearing from you.

myBiodentity said...

Ditto to musicnowandever's comments.

The site is crashing and files cannot be found. I am getting a message referring to "if file was uploaded more than 3 years ago...." when it was within the last 30 minutes.

Frustrating.

musicnowandever said...

Yes, it's like myBiodentity said, i am trying to download a file posted by me a week or two ago and it says that the file was uploaded more than 3 years ago, which is not correct. Hope everything will be solved. Thank you.

Ian Ippolito said...

Musicnowandever,mybiodentity, and all others experiencing problems from between midnight last night and this morning with attachments: my sincere apologies for this issue. It was a mistake made by a tech who did not turn on a physical server properly after a conversion of the machine to the cloud failed unexpectedly. It's been fixed and I've posted full information on what happened and the new precautions being taken on this blog posting.
Ian

wishcraft said...

Oh yeah I am making a cloud for XOOPS called Xortify.com - It has an API for JSON, CURL and SOAP with mirrored functions to showcast some of my GPL from www.chronolabs.coop

Clouds are kewl, I like making my tweet about bans!!

willalber said...

Any chance that the "Search" functionality will actually work when the site is moved into the cloud? It's beyond lame at the moment - if it works, it takes 30+ seconds to do so - and another 30+ seconds to move to the next page of results.



Otherwise - am loving the new site layout - especially now I've found the menu layout option (although - why no option to have the 'bar' on the LHS rather than RHS?).

Ian Ippolito said...

WillAlber,

>>Any chance that the "Search" functionality will actually work when the site is moved into the cloud? It's beyond lame at the moment - if it works, it takes 30+ seconds to do so - and another 30+ seconds to move to the next page of results.

My apologies for the frustration. If you are doing advanced searches then yes you are prone to timeouts, now. Going to the cloud will give us a lot more CPU horsepower to power it. However, long term we actually have a project to rework it completely with a hybrid Google/native database scheme that will be much faster and give you more flexibility and capabilities.


>>Otherwise - am loving the new site layout - especially now I've found the menu layout option (although - why no option to have the 'bar' on the LHS rather than RHS?).

Thanks, and I'm glad to hear that. And I'll add your request for a LHS version to the feedback page on your behalf.

Ian

Post a Comment