Takeaway: Something
always goes wrong in a “go live” scenario. Prepare your development
team, your work environment, and yourself to handle these high-pressure
situations.
Create a game plan
You need to have a well-defined plan in advance. Here is a checklist of things you need to know before you can go live:- What are the exact steps to perform any needed data migrations or server changes?
- How do we perform the deployment? (Make a checklist and give every team member a printed copy of it. Cross things off as you do them, noting who did them and what date and time they were done.)
- How do you back up the application and data pre-deployment?
- If something goes catastrophically wrong, what is the procedure to restore the original version or restart the deployment?
- How do you reach the IT or support/help departments of all involved parties?
- When do you reach your go/no-go point (i.e., the point when you either commit to finishing the deployment or start to roll back the changes), and what are your criteria?
- What are your post-deployment triage priorities? (This one is especially important, because there will likely be a lot of “squeaky wheels” begging for grease, and you need to know which ones to prioritize first.)
- How do you take things offline and bring them back online if needed?
- How do you contact your customers to let them know things are okay or not okay?
Prepare physically and emotionally
Before heading into the last stretch before a go live, everyone needs a lot of sleep and rest because when things go wrong, you’ll work long hours to sort them out. A team of folks already at their breaking point will not work well together, will have a hard time executing the basics, and will fall down flat when challenges arise.For employees with spouses, kids, etc., you need to get them on board too. There is no way I could have accomplished what I did on any of my go live adventures if my wife was calling me constantly begging me to come home or laying on a thick guilt trip about not seeing the kids.
Prepare your home office
During a go-live, there is a good chance you will have to work for long hours at a stretch or at odd hours of the night. Many times, the work cannot wait for you to get into the office.Unlike me, most developers work only a few hours a week from home, if at all, so they are not well-equipped to do it. Make sure your home office setup is ready to do real work, which means a proper keyboard, mouse, and monitors, as well as a squared away Internet connection. I also make sure that I have enough music and a good speaker or headphone setup so I can drown out the household noises.
Be flexible with your team
Your team will likely work long, hard hours during the go live. It doesn’t help anyone if you are a hard case about things like getting to the office at 9:00 AM, shaving, dressing up, etc. unless those things are absolutely necessary (such as on customer meeting days). For the go live I just went through, I often worked until 4:00 AM, 5:00 AM, and even 6:00 AM a couple of times. If I had forced myself to go into the office at 9:00 AM, I would have been too tired to be effective for the entire day. Instead, I let myself get the extra sleep, and then only went into the office if it made sense.Also, if folks are stuck at the office late, buy them dinner.
Hope for the best, prepare for the worst
NASA builds triple redundancy into projects because of the fact that things will go wrong, no matter how much you plan. Unless you are working for a big enterprise, you probably can’t do that. You have a Development environment, a Testing/QA/Staging environment or two, and a Production environment. The best thing we did for our latest go live was import the data from Production into our Testing environment about a week in advance, which let us catch a lot of issues early and test our data migration with the most realistic data set we could.Remember you’re all on the same side
As the pressure mounts during the go live, things often get tense. It is easy for folks to start snapping at and working against each other instead of with each other. Finger-pointing will not solve the issues.You absolutely must not allow the pressure, lack of sleep, or other issues interfere with how you work with your teammates. When you are through it all, you’ll be glad you kept your cool.
