The SDLC, Part 3 – Common pitfalls when applying configuration updates

You’ve followed the recipe to the letter…only to discover you’re out of propane for the barbecue.

You brush and floss just like you’re supposed to….but the dentist tells you you have a cavity anyway.

You’ve conducted your workflow development with as much rigor and care as humanly possible…but your configuration update fails to apply.

Sometimes things just don’t go your way. This is true with all things and, when it happens, it often leaves you scratching your head. When it comes to failed Configuration Updates, it’s sometimes difficult to figure out what went wrong, but there are some common pitfalls that affect everyone eventually. I’ll discuss a few of the more common ones with the hope that you are one of the fortunate ones who can avoid pain by learning from the experiences of others.

Pitfall #1: Developing directly on Production

The whole premise of the SDLC is that development only takes place in the development environment, and no where else. While this sounds simple, it’s a policy frequently broken. Workflow configuration is a rich network of related objects. Every time you define a relationship through a property that is either an entity reference or a set, you extend the “object network”. In fact, your entire configuration is really an extension of the core object network provided by the Extranet product.

The Extranet platform is designed from the ground up to manage this network, but its world is scoped to a single instance of a store. The runtime is not, and in the case of the SDLC, should not be aware of other environments. This means that it can make no assumptions about objects that exist in the store to which the configuration update is applied. It must assume that the object network in development reflects the object network in staging and production. Because of this, it’s trivially easy to violate this assumption by configuring workflow directly on production. If that’s done, all assumptions that the state of production reflects the state of development at the time the current round of development began are incorrect and the Configuration Update is likely to fail.

Errors in the Patch Log that indicate you may be a victim of this pitfall will often refer to an inability to find entities or establish references.

One common cause for such an error is when you add a user to the development store but there is no corresponding user with the same user ID on production. Some objects include a reference to their owner. In the case of Saved Searches, for example, the owner will be the developer that created the saved search. In order to successfully install the new saved search on the target store, that same user must also exist there.

Troubleshooting this type of problem is tedious and sometimes tricky because it’s often necessary to unravel a portion of the object network. It’s a good idea to do whatever you can to avoid the problem in the first place when you can.

Bottom Line: Only implement your workflow on the development store and make sure that all developers have user accounts on development and production (TIP: You don’t need to make them Site Managers on production).

Pitfall #2: Not applying the update as a Site Manager

If your update fails to apply and you see a message that has this in the log entry:

Only the room owner(s) can edit this information.

you are probably not specifying the credentials for a site manager account when Applying the Update.

This can happen when a user is not provided in the Apply Update command via the Administration Manager or if the provided user is not a Site Manager. The installation of new or updated Page Templates causes edit permissions checks for each component on the page template and unless the active user is a Site Manager those checks will likely fail.

Bottom Line: Always specify a site manager user when applying a Configuration Update. Technically this isn’t always required depending upon the contents of the Configuration Update, but it’s easy to do so make a habit of doing it every time.

----

More pitfall avoidance tips next time….

Cheers!

Limitations in my blogging approach, and what, if anything, to do about them

I’d like to take a minor time out from the SDLC discussion to solicit feedback on how to make this blog more useful. In my post back on March 2nd, I described that I’m actually hosting this blog at http://ResearchExtranet.blogspot.com then exposing into the Click Commerce site at Tom's Blog via an RSS Viewer component. This approach, while allowing me to use a wide array of authoring tools, does have some limitations for the reader. The two I have found the most inconvenient are:

1. BlogSpot recently decided to include an invisible pixel sized image into every post so they can track readership. This seemingly innocuous change is the cause for the Security Warning being displayed by the browser so the user sees a message that looks something like this: “This webpage contains content that will not be delivered using a secure HTTPS connection, which could compromise the security of the entire webpage.”
   
   This happens because http://research.clickcommerce.com is SSL secured for authenticated users and the source URL for the tracking image is not. Though the warning doesn’t translate into a problem actually seeing the blog post, it is annoying. The friendly people at BlogSpot have informed me that they are looking into providing better support for SSL enabled sites. I’m hopeful they will provide a solution so I’m inclined to wait this one out if you are willing to suffer the wait with me. Please let me know if this inconvenience is a major issue for you.
2. There have been times when an image would have done a better job than mere words in making my point. To allow the image to be viewable no matter where you read my blog (ClickCommerce.com, BlogSpot, or your favorite blog reader), the image needs to be available to all and not hosted. I’ve avoided using them because of the mixed content warning which results when presenting an image from a site other than the site where the blog post is viewed. So I put it to you, do you view the blog from locations other than ClickCommerce.com? Would you be willing to see and dismiss the mixed content warning in order to get the benefit of embedded images? An alternative would be for me to post knowledgebase articles and use the blog posts to introduce them. It’s not quite as convenient as having it all in one place, but would avoid the issue with the warning. Please send me your thoughts on how you’d like to see this blog move forward.

And now back to regularly scheduled  programming….

Cheers!
- Tom

The SDLC Part 2 – Process Studio and Source Control

 

Last time I introduced the notion of the recommended Software development Lifecycle (SDLC). Now it’s time to get a bit more specific. As mentioned last time, the best way to support a disciplined development process is to make use of three distinct environments: Development, Staging, and Production. Each environment can either be made up of a single or multiple servers. While there is no requirement that each environment be like the others, it is recommended that your staging environment match production as closely as possible so that experience gained from testing your site on staging will reflect the experience your users will have on the production system. It’s also useful because this will best enable you to use your staging server(s) as a warm-spare in case of catastrophic failure of the production site.

Further Reading…. FAQ: Everything You Wanted to Know about Source Control Integration But Were Afraid to Ask HOWTO: Apply Large Configuration Updates

To go into everything you can do when configuring and implementing your workflow processes would take more time than I have here and there are several good articles and online reference guides available in the knowledgebase. We also offer both introductory and advanced training courses. Instead I’ll focus on how to manage the development process.

A key principle of the SDLC is that development only takes place in the development environment and not on staging or production. The work you do in the development environment gets moved to staging so it can be tested through a configuration update. A configuration update is a zip file that includes the full set of changes made during development that need to be tested then deployed to production. In order to accurately identify the changes that should be built into the configuration update, each individual change is versioned in a repository using Microsoft Visual Source Safe.

Making a change or enhancement to workflow configuration begins by checking out the elements from the configuration repository using a tool called Process Studio. Once checked out, development takes place using the web based tools, Entity Manager, or Site Designer. Before you should consider the change complete, the changes are tested locally on the development server. If everything works as expected, the changes are checked back into source control using Process Studio. This process repeats itself for all changes.

When the development of all intended fixes/enhancements is complete, it’s time to put them to the test. While developers are expected to test their changes in the development environment before they are checked into source control, official testing should never be done on development. The reason for this is that the development environment is not a good approximation for production. Developers, in the course of their work, make changes to data and environment that make it hard to be able to use the test results as a predictor of how the changes will work on production. Instead, a configuration update is created using Process Studio so it can be applied to staging for official testing. Before applying the update to Staging, it’s a good idea to refresh the staging environment from the most recent production backup. This gives you the best chance of understanding how the changes will behave on production.

If issues are discovered during testing on staging, the process is reset.

1. The issues are fixed in development (check-out, fix, check-in),
2. A new configuration update is built,
3. Staging is restored from a production backup,
4. The configuration update is applied to staging
5. The changes are tested

If all the tests pass, the exact same configuration update that was last applied to staging is then applied to production. Though not required, it’s a good idea at this point to refresh your development environment with a new backup of production. The closer the development environment is to the real thing, the fewer issues you’ll have going forward.

At this point, development can begin on the next set of features, fixes, and enhancements. And the cycle repeats…

To learn more about the role of source control in your development lifecycle, please read the following article:

FAQ: Everything You Wanted to Know about Source Control Integration But Were Afraid to Ask

That article does an excellent job describing all core principles and processes. Of course, not everything goes as planned as you apply updates to your staging and production sites. Next time, I’ll discuss some common challenges and how to troubleshoot when issues do arise.

Cheers!

The Software Development Lifecycle – Part 1

Well, it was inevitable. My goal of posting at least weekly to this blog is being threatened. It’s been over a week since my last post so it’s time to pick it back up again.

This week at Click was certainly a busy one and it made me realize that it’s time for a refresher on our recommended Software Development Life Cycle (SDLC). All software development follows a repeated cycle, sort of like the “Wet Hair, Lather, Rinse, Repeat” instructions on your shampoo bottle – simple, but effective. Generally speaking, software development follows  a simple cycle as well:

Define –> Design –> Implement –> Test –> Deploy –> Repeat

This is true no matter the technology or tools. Working with Click Commerce Extranet base solutions is no different. Putting the cycle into practice requires discipline, familiarity with the development tools, and an ability to troubleshoot problems when they arise. Over the next couple of posts, I’ll be describing the Click Research and Healthcare SDLC. Along the way, I’ll highlight common problems and how to address them. Hopefully this will lead us to a discussion on how best to handle the concurrent development of multiple solutions, which is the topic of a panel discussion I’ll be hosting at the upcoming C3 conference. So…let’s get started.

Three Environments
To effectively practice the SDLC, three environments are required:

1. Development
   This is where all active development takes place. Developers typically will work together in this environment, benefitting from and leveraging each other’s work. All work is formally versioned through the use of Source Control integration via a tool called Process Studio. We’ll be discussing the use of Process Studio in more detail a bit later. This is the only environment where development should place.
   
2. Staging (Test)
   This environment is ideally a mirror image the production environment and is used as a place to test completed development before it is deemed ready for production use. It also can serve as a warm standby environment just in case there are issues with the production site that can’t immediately be resolved.
   
3. Production
   This is the live system and the only site end users will use to perform their daily tasks.

Work performed in the development server is packaged up into what’s called a Configuration Update which can then be applied to Staging, where it is tested, and, if all the tests pass, to Production. For more information on what is included in a Configuration Update, check out the following Knowledgebase Article:

INFO: Understanding Configuration Updates in Click Commerce Extranet

Next time, we’ll talk about how configuration updates are built and special things to consider in order to make sure they can be correctly applied.