All posts by Thomas Carlier

SCM is Complex Queue Processing

Implementation

As an SCM teams grow in size, the issue of request handling increases in importance. Small organization with a single administrator or build engineer need nothing more than simple first in, first out processing of email requests, plus an escalation process for high priority work.

When the team grows to more than tow admins, it’s time to think about help desk or service software.  Prioritizing and processing a queue full of requests with visibility and transparency will become necessary to quell the rising competition for time from development teams. This is the point where companies select and implement a software solution and call it good.  End of story, right?

Absolutely not.  An often missing element of operational planning is fully accessing the complexity of request handling as a sub-system of the configuration process. Left unmanaged, this aspect of software development results in disruptive workflows and becomes a major source of complaints against DevOps organizations. Believe me, I’ve seen my share of yelling and screaming matches in 20 years of software configuration management work.

We all know there are times when requests for one project need higher prioritization than requests for other projects. Some requests can be considered low priority to be handled when nothing important is in the queue.  I like to think of this request handling problem as similar to that of a grocery store checkout. The main two elements of a checkout are checkers and the lines of customers waiting to be checked out. Other elements include the self-checkout area and express lines; and of course, the manager.

Properly managing the store’s checkout has to include checkout training and some level of customer behavior analysis. This issue is tricky because of the human behavior involved. Some people would never dream of cutting to the front of the line at a grocery store, but unfortunately  there are a few people who actually are rude enough to push their way to the front. What should the checker do in this instance? Should they be sent to the back of the line, or should they be checked out first? If they are checked out and a complaint occurs, who is the problem?  The line cutter?  The checker?  The manager? Is the complaining customer the problem?

I think the blame could go anywhere depending on the company policy if there is one.  It could be a lack of checker training, or the checker’s judgement call.  Is flexibility allowed? . If the checker has not been trained in proper procedures for handling these situations, how can they be expected to perform correctly. Is it the managers fault for not having enough staff on duty? Imagine how it would be if the checker noticed someone she knew at the back of the line and called them up to the front to check out first? Is there a policy against this? What if the customer only has one item?  Is there an express lane?

A grocery store checkout process is probably broken if…

· customers routinely cut to the head of the line.
· managers don’t open enough check stands.
· checkers aren’t been trained properly
· express lanes aren’t available
· self-checkout isn’t possible
· checkers routinely make bad judgement calls
· doesn’t send rude customer to the back of the line.
· checkers give preferential treatment to their friends.

A funny thing I’ve noticed is people who would never dream of cutting in line at the grocery store often have no problem cutting in line at the SCM counter. I’ve seen this many times and I think lack of visibility is the cause – or enabler – of the behavior. If you can’t see the line you don’t know it’s happening. If the people in line don’t see the lane cutters, they won’t grumble as much, and this could be used as an argument for hiding the queue, but perhaps wouldn’t be best for the organization.  The right thing to do is create visibility of the queue.

In the SCM world, there are other factors impacting the request handling process. For one thing, it may not be very transparent even if there is help desk software. Unlike the grocery store checkout, it may still be hard to see how many people are on duty or how many people are in line. Members of a development team may discover their requests are handled faster when they drop by the DevOps person’s cube since they don’t have to open a ticket that way, and this makes some of the worst offenders of the ‘queue cutting.’ I think it happens more in the software development than in the grocery stores since lane cutters don’t hear the gasps or see the appalled looks on the faces of other customers waiting in line.

My best advice is to recommend setting policies for prioritizing and processing requests for service, and also training and empowering your team to enforce the policy. Does your development team respect your prioritization policy? Did you remember that SCM documentation is useful?  There is no easy, one size fits all solution to request handling.

SCM is Complex Queue Processing

SCM Documentaion is Useful

Design

It’s very common for software engineers to document procedures. Perhaps it should be required for everyone. For some, it’s always been this way. To others, this is new. I’m not sure why it’s not always done right up front since documentation makes us all more efficient. But what makes good documentation?

First, let’s talk about the most common type of documentation: the data dump. This is the kind of documentation that barely meets the requirements of ‘it’s documented.’ Technically it might be documented, but since there is too much information presented it may not be useful.   This is the kind of documentation you get from people who don’t believe in documentation, but are forced to do it – created when the writer’s time is deemed more important than the reader’s time.

I like to follow the principles as outlined in the book, Presenting To Win: the art of telling your story by Jerry Weissman.  It means looking over the pile of information and deciding how it needs to be presented. It means putting the user’s time ahead of the writer’s time. After having done this for many process documentation projects, I now have a pattern to share that improved my documentation.   Hopefully, it will improve yours.

Start with the different types of information contained in the documentation, such as domain information and technical information.   An example of domain knowledge is a specific servers name and port, or the policies regarding naming conventions, such as “all build machines are on the build.xxx.com sub net.   Technical knowledge is the command used to change the name. So if the documentation includes changing a host name to meet the standard convention, there is some domain knowledge and some technical involved. Start by separating the two types.

When considering the user of the software engineering documentation, the user needs can be found in the four quadrants of the matrix below, based on domain vs technical knowledge.

Junior Software Engineer Senior Software Engineer
new to domain both domain information
domain veteran technical instructions none

 

Since you won’t always know which quadrant the user is in, it’s best to write documentation with the varying user needs in mind. Assuming senior engineers are more highly paid than junior engineers, domain information should come first. A new senior level system administrator will have the technical knowledge but not the domain knowledge, so make it easy for to find the domain knowledge without having to read through all the technical details. Here is are a couple examples

  1. Change the system name to meet the standard naming convention
    1. Naming conventions are found here…
    2. Instructions for changing system names are found here…
  2. Change the IP address
    1. Available IP address are found here…
    2. Instructions for changing system IP addresses are found here…

The senior engineer who has been with the company can stop at reading the numbered step instructions. The senior engineer who just got hired will read the top level steps, plus dive into the second level sections as needed. The junior engineer who has been with the company will read the top level steps, possibly will skip some second level sections, but will read others.   The junior engineer who just got hired will probably need all sections of the documentation.

The goal here it so make it easy for the end user to find just the right amount of information needed. Eliminate wasted time reading stuff already known by the user. At the same time, be sure to include all of the information needed so the SCM team isn’t needed.

SCM Documentation is Useful