What does this mean?

In terms of a concept a baseline is a snapshot of the configuration which is validated and locked. It is also a verification of the current configuration. What this means conceptually is that by having established a baseline you can spot gaps or variations to the last known state or baseline.

Nice theory, but often considered hard to do and repeat in many IT environments. To be effective, baselines must be deliverable quickly and be repeatable without incurring high data collection costs and without scrimping on data quality.

With Tideway Foundation (TWF) baselines are very easy to generate and simple to repeat. Baselines can also be established for different information needs and some examples are given below.

Server inventory and CPU count: This level of baseline is useful as a starting point for establishing what you have when negotiating service contracts, for example.

Detailed Inventory of Servers and Software: useful when establishing or redefining IT strategy, searching for areas of technology risk or identifying where further cost control may be beneficial – a real area of value when responding to the current economic climate and taking cost out of IT. This level of baseline is also equally useful in establishing whether you have the right technology to support the business.

In terms of spotting the gaps, a detailed inventory of servers and software baseline makes an ideal starting point, and the automatically discovered information can be used to feed data center move projects, disaster recovery planning and testing, mapping power and BTU consumption to technology, planning for consolidation, standardization and virtualization, assessing EOL/EOS scenarios, and so on,

Actual configuration data: When undergoing internal IT reviews or external SAS-70 audits, you may have to prove that you know your IT configuration, and that you have control over changes in the environment. You may also need to understand the type of changes and their frequency in order to put in place the correct level of control and governance across IT.

Application Dependency Maps: A baseline of Application Dependency Maps will underpin application stabilization, or the analysis required to support a move toward shared IT services, or indeed provide a starting point in planning which business IT services to keep in a merger and acquisition scenario.

These are just a small subset of possible baselines uses. Tideway delivers baselines through a packaged service approach, often delivering the results during the first few weeks of the engagement and enabling the customer to spot changes to the baseline while they continue to benefit from using Tideway Foundation. Tideway Foundation dashboards then make reviewing the summary baseline information a simple task while facilitating a launch point for a deep dive into the underlying data.

The key point to remember about Tideway Foundation baselines is that the package service approach is results driven. They are delivered quickly and provide a repeatable IT baseline solution without incurring those high data collection costs and without scrimping on data quality.

The November release includes 9 new products and 2 enhancements extending Tideway’s coverage of:

• IBM products, taking the total to 43 products across the Information Management, Tivoli and Websphere brands.
  
• Open source products to include PostgreSQL, Lighttpd, Red Hat and Sun in response to the growing market trend towards open source products.
  
• Server virtualization platforms to include HP-UX nPartitions which builds on our existing coverage for Citrix, IBM, Sun, VMware and Xen.
  

You can visit the ‘In the Spotlight’ page on Configipedia for show case examples of how TKU release can help customers report on HP-UX nPartitions, IBM Tivoli Composite Application Manager and Sun Application Server.

Software Product Patterns in TKU November include:

• IBM Tivoli Composite Application Manager Response Time Tracking
  
• IBM Tivoli Composite Application Manager for Websphere and J2EE
  
• IBM Websphere Application Adapters
  
• Lighttpd Webserver
  
• PostgreSQL
  
• Red Hat JBoss Hibernate
  
• Red Hat JBoss JacORB
  
• Sun Java Application Server / GlassFish
  
• HP-UX Partitions (Beta Release Pattern) allows you to identify partitions, number of processors allocated to a partition and hardware container for partition
  

The HP-UX Partitions pattern is NOT included as part of the TKU release package as it requires discovery extensions to function. The pattern and discovery extension can be downloaded from ‘Configipedia’

How on earth could that happen?

Computing is a funny business. Some of the things that sound hard to do end up being relatively easy, others that sound like they should be straightforward end up being horribly complicated.

One of the many things that Tideway Foundation does for our customers is tell them what servers they have in their data centres, what software is installed and running on them, and how the business services (which is, after all, what the data centre is there for) map on to that infrastructure. The first part of this, knowing what servers you have in your data center, sounds pretty trivial. Surely you can just go and count them?

Well yes, you can. The problem is that counting them doesn’t actually help you understand what the server is doing, whether it is important to the business, or indeed whether it is doing anything useful at all.

Unfortunately the data centre is full of anonymous racks of computers which all look much the same. If you can find your way into the data centre, you can see and touch the boxes, see the cables that supply them with power and those which connect them to the network, but it won’t help you understand whether the server you are looking at is running part of the company’s latest and greatest Web 3.0 social networking project or the remnants of a long abandoned CRM system.

One way of thinking about this problem is to understand it as the divide between a physical view of the infrastructure, stuff you can see and touch, versus the logical view, which involves understanding what software is running where and how it interacts with other bits of software running elsewhere.

A systems administrator can gain a logical view of a server by logging in to it over the network and executing commands which will tell him or her what software is running at the time. But repeating this process several thousand times to build a complete picture of the data centre simply takes too long. By the time the systems administrator has finished, the world has changed. And the resulting view is likely to be limited to those servers that you knew existed. How does the systems administrator know to log in to a server if nobody knows it exists? This is more more likely than you might think since a typical systems administrator in a large organisation may never even have visited the data centre.

How does Tideway Foundation help?

We automate much of this process, providing a detailed view of what servers are present, how they are configured, the software that runs on them, and the business services they support. It might sound easy, but is harder than it looks.

One challenge is that most data centres have accreted many layers of technology and complexity over the years, so servers are often of many types with many different operating systems and many different versions of those operating systems. A complete picture may require hundreds of distinct software products and versions to be identified. Being able to cope with this heterogeneity is complicated, but essential to provide a complete picture.

The good news is that we regularly find servers that are not being used or which can be retired for other reasons. Given that it can cost $10K-$20K per year to run each server in the data centre, and that large data centres may contain many thousands, the potential for cost saving is huge, even if you don’t find the 30% of unused servers that the Economist article suggests may be present.

Given the variety of CM tools available to this organization, the configuration manager has no single source of truth, and internal customers and consumers of this data wrestle daily with spreadsheets, emails, meetings and word-of-mouth comments in order to collect their facts. This leads to the fact gap. To use Rudolf Melik’s definitions (in “The Rise of the Project Workforce”) data in context provides us with information, while information in context provides us with knowledge. In other words, actionable information comes from knowledge. For a DC move project, the data about the servers, applications, and technologies being moved needs to be accurate, and correct. Knowledge about each component also needs to be presented in the context of all related components which make up each end-to-end business application. Otherwise the required knowledge to make the move a success is incomplete.

When planning a DC move, the end-to-end component relationships need to be known. From this we can deduce which components should move, which should be replicated, which should undergo some form of technology refresh, and which are actually redundant. When move time comes, the relocation team needs to know which physical systems to ‘lift & shift’ and which configuration changes need to be made in the process. The test and application teams on the receiving end need to know how the end-to-end dependencies should now map in order to check that they now do. Meanwhile, the configuration manager is between a rock and a hard place. Firstly, collecting the required data takes hours/days of manual audits. Secondly, what data is available is not in a single source, so when everything is moved, it is unlikely that these disparate sources of data will be correctly updated to reflect the results of moves and changes.

Those managing the moves face triple trouble. Firstly, they make decisions based on incomplete evidence. Then they make resource planning decisions based on assumptions that their data is correct. Then they have to compensate with additional work and correction activity each time an incorrect configuration is encountered. Finally, configuration errors which have the potential to be repeatable are difficult to search and check for because there is no one source of truth to check, even though the potential for the error is known about. Liken this to knowing there is a banana skin ahead of you, and then stepping on it.

Jim Carroll, in his book “What I Learned from Frogs in Texas” suggests that almost a third of a person’s working week is spent helping others to answer questions. Over half of these questions have been asked before, and for our configuration manager, not having a single source of truth means that configuration knowledge can never become institutionalized. Interestingly, in the same study, Carroll noted that 81% of respondents believed that it was important to share knowledge. If we park the consideration that knowledge is power and the political implications that this carries, there is no logical reason to continue provisioning the same knowledge over and over again to the same consumers. Instead, what makes more sense is a focus on automating the data collection process, and allowing others to become self-sufficient in accessing and using that single source of truth. This single source of truth can then act as a common vocabulary which in the case of IT flattens technology silos and brings back context to the available configuration data across the whole of the IT organization.

The net result of having a single source of trusted relationship and attribute data which flattens the technology silos and leads to a greater degree of collaboration amongst the different technology teams is simple. The looming DC move project is far more likely to be successful.

It’s true, and probably says a lot about the missing links between the parts of my brain which control fun, and the parts that deal with risk. To rectify this I probably need to go through an intensive set of applied behavioral analysis sessions to reset the neural links in my brain – resetting the patterns which associate fun with gambling. Of course, identifying patterns is something which is very important in Vegas – and the Retail world for that matter. A famous US economist once quipped that there are 3 rules in physics which describe 99% of the behavior of inanimate objects while there are 99 rules in business which describe only 3% of buyer behavior.

Walk into any bookstore – Barnes & Noble, Borders, Waterstones, etc and we find many business books which either attempt to identify the patterns in buyer behavior or attempt to reset our patterns of behavior. Much of the advice makes sense, or at least we can see how the conclusions on particular scenarios were reached by the authors. Unfortunately, and the reason why I found it so easy to resist blowing a few dollars in Vegas, change requires effort, and as one of those 3 laws of physics tell us, change doesn’t happen without an equal and opposite force.

Las Vegas airport is an amusing place. It has slot machines. It also has a continuous set of people arriving intent on beating the system and ‘winning big’. As the gambling newbies arrive, they file past the current crop of losers – both groups thinking about how to spot winning patterns – much the same, I imagine, as traders on Wall Street. So patterns are everywhere. We can’t escape them. They even exist in IT – in the systems we deploy, and the way we manage those systems.

The problem with patterns is that they are sometimes difficult to spot. How hard can it be to count 52 cards – quite hard apparently? We know patterns exist, but it takes effort to collect the data and keep track of it. Now, here is a good bet. When a data center manager or an application owner bets you s/he knows how many servers they have, take the bet – they won’t get it right – unless of course their data center is the size of a cupboard, and even then virtualization can throw them a joker.

Once we wake up to reality and realize that we don’t know what is in our data centers it’s easier to understand how we can make mistakes on cost and risk when it comes to managing IT. If we miss a count of just one card in a 52 card deck our game is lost. Likewise, if we forget to include just one system in a data center move, or a patch rollout, or a software license count, the consequences can be expensive.

Enter stage left, auto-discovery and application dependency mapping. Our counting is done for us, the pattern identification and mapping is taken care of. All we have to do is think about what we can do with the information at our fingertips, and last time I checked, thinking didn’t require as much effort as resisting change. Except, digesting the new information may require effort, and it’s a bit like going on a diet – nice in principle, but we always resort back to what we feel comfortable with.

That’s why applied behavioral analysis is so important. If we take new information, now readily available through our auto-discovery and application dependency mapping system, and we are given actual examples of how that data is useful for us and how we benefit from it – ok, so you should be thinking about paid TV advertisements for abdominal exercisers at this point – the actual act of physically experiencing something new helps to embed the change and its benefits into our current set of beliefs.

Tideway’s Configipedia website (http://www.tideway.com/configipedia/Main_Page) does just that. Via these webpages we show our customers how to use the data that Tideway Foundation collects, and demonstrate just how easy it is to benefit from it in their own IT environments. Some of our existing and prospective customers have been learning about how to establish a baseline of policies around Standard Operating Environment (SOE); how to gather license intelligence for a forthcoming Oracle Audit; how to set up a Change Intelligence monitor on critical JAR files underpinning J2EE apps; and so on. During subsequent blogs, I propose to explore some of these areas in detail, and sharing our recipes for success.

In the meantime, and getting back to Vegas, I feel confident that if someone were to send me just a few dollars, I’d be able to practice gambling sufficiently to see the patterns and begin experiencing gambling as a fun activity…..

The lessons learnt below apply to the deployment of our product across an IT estate, but some clearly have relevance to all projects.

Lessons Learnt:

1. Provide regular dashboard reports & WebEx sessions to demonstrate discovered data. This builds confidence in the discovery process and the data captured, while encouraging customers to assist in achieving deployment success. There is always something very compelling about seeing continuous progress which, in the end, tends to encourage and drag us all to the finish line – who want to only get to 80%, for example.

2. Choose and develop early champions on the customer team who naturally exude confidence and trust in their colleagues. It is amazing what an employee badge can do for our sense of belonging. When that employee badge is carried by someone who believes in the success of our engagement, and can influence others, expanding the deployment into new areas becomes so much easier. As a famous 49er once said, “Leadership is like gravity…” it doesn’t matter how we define it – just use it.

3. Now for the technical lesson. Progress will be slow without technical support. Have committed technical resources available from the customer’s team, and make sure they have equally committed air-cover. Report technical progress in detail – point out precisely where the issue is, and what the issue is. Better still, change the way the product works or reports its progress so that it can self-diagnose its own issues. Making it clear what the problem is makes the solution clearer. Some specific points pertinent to Tideway Foundation:

 Implement deployment reports to help remediation and track progress.

 Gain permission and leverage the potential to roll out a package with ssh key, required access, and any required tools and permissions (specifically lsof & ndd). Especially if these are not already resident on target servers.

 Ensure DNS or a similar tool is available to indentify hostnames of discovered servers for which access is not yet available.

4. Learn who does what. Identify the tasks and who has responsibility for them on the customer team. Building on this point, use the customer’s additional information systems which might provide valuable ‘Let’s get this done’ data, such as who owns which system, and who is authorised to make changes to them. More specifically, learn the art of the mash-up, and know how to pull related data from different systems into a single view. For example, asset systems may contain useful ownership and location reference information.

5. Be a teacher. Encourage the customer team with early coaching & mentoring to enable them to takeover basic ‘business as usual’ product tasks (remember to get sign-off on your deliverables first though). Don’t fish for the customer – show them how to fish for themselves. After all, our customers have a greater appreciation of the value of automatically discovered application dependency mapping information as it applied to their systems, than we do. What we can provide, though, are those nuggets & gems about the success other customers have had elsewhere. Be the customer’s guide.

6. Maintain data quality. Measure quality using an appropriate yardstick, and report change against this. Aim to show positive progress on data quality, so that any regression becomes something customers feel compelled to do something about. In the case of our product, quality can be maximised by continually aggregating daily scan results during deployment.

7. Finally, beware local public holidays and general national vacation periods, when delivering small engagements to global customers. In the US the workforce likes to work ‘summer hours’ in July and August, while in Europe August is a popular vacation month too.

…and remember, keep learning!

This has happened to me when I needed to upgrade to newer versions of the VPN software, and it’s very annoying. The time when I’m upgrading my VPN software is by definition when I need to connect to the VPN. Finding myself unable to connect to any network at all is painful to say the least.

So, after the last time this happened to me, I decided to hunt around for an alternative. I found vpnc, a Unix implementation of the Cisco VPN software maintained my Maurice Massar. A little while ago a clever bloke named Paolo Zerpellon did some work to get this working over Cygwin under Windows, using a TAP driver from the OpenVPN project.

TAP driver is a pseudo network interface that permits applications to tunnel things over Ethernet. An application can write an ethernet frame to the TAP device, and the Windows networking stack will receive it as if it came from a normal network interface. Vpnc uses this to sit between the Windows kernel and the VPN.

So, the steps you need to follow are:

1. Get Cygwin going.
2. Download, install and build vpnc.
3. Install the OpenVPN distribution to get the TAP driver.
4. Get your Cisco VPN connection file and translate it into a vpnc connection file.
5. Tweak the connection file a bit.
6. If you’re using vpnc 0.5.1, fix a little bug in it.
7. Connect!

So, on with the show:

Installing Cygwin

Cygwin is a bit of Windows software that provides a Linux emulation layer. It means that you can compile Linux software to run on Windows. I’m not going to document it here, obviously, but suffice it to say that it installs direct from the internet using a program called “Setup.exe” that you download from their site (http://www.cygwin.com/). Keep setup.exe safe – it’s how you install additional packages in future.

During the installation phase, it gives you the chance to select any additional packages you want. The packages are in a tree view, and you expand the top node to find the ones you’re after. Packages that are not selected have the value “Skip” in the “New” column (obviously). To install them, click the “Skip” and it’ll change to the version that will be installed. The non-default packages that you want are:

• Devel -> make (to make vpnc)
• Devel -> gcc-core (to compile vpnc)
• Libs -> libgcrypt-devel (headers needed by vpnc)

Download and build vpnc

• Download the latest vpnc package from here to somewhere obvious.
• Untar it:
  tar -xzf <vpnc.tar.gz you downloaded>
• Build and install it:
  cd <dir that it was untarred to>; make install

You’ll get some warnings and stuff, and it’ll build a few executables. Those exes will be copied into the right places on your system, and a default (but useless) config will be created in your /etc/vpnc/default.conf directory. The executables you’re interested in are:

• vpnc.exe: the main vpnc executable, installed into /usr/local/sbin
• pcf2vpnc.exe: a utility to translate Cisco configs into vpnc ones (see below), installed into /usr/local/bin

Download and install OpenVPN

Download the MSI installer from here and install it on your machine. During the installation, you can choose (if you wish) to uncheck all the components of it except for the Win32 Tap Driver, which is the only bit you need.

After the installation, you’ll see that you have a new network connection in your Network Connections that’s got an adapter of type “TAP-Win32 Adapter V8” or some such. It’ll be called something really useful like “Network Connection 4”, so rename it to “TAP Driver”, or just “TAP”, or “Binky” – whatever you like.

Translate your Cisco VPN connection file into one for vpnc

For vpnc to connect to your VPN and log in with the correct details, it needs loads of connection info. The way it gets this is either from the command line or from a config file. You can create multiple configs and tell vpnc which one to use on the command line when you run it. The configs you create need to be named like “something.conf”, and they must be placed in /etc/vpnc/.

Your network admin will have given you a file with a “.pcf” extension to use with the Cisco VPN software when connecting to your VPN. You might find it in under your home directory – do a search for “.pcf”. It’s a text file that contains details like your VPN concentrator’s DNS name, the group name, the encoded group password and so on. You need the details from this file in vpnc’s config format, and luckily enough, vpnc comes with a utility to translate it.

Run the utility, specifying the output file (default.conf):
pcf2vpnc MyCompanys.pcf default.conf

Move the resulting file into the place where vpnc looks for its configurations:
mv default.conf /etc/vpnc

The reason that you call this file “default.conf” is that if you run vpnc with no arguments then it’ll look for the “default.conf” file and use that.

Tweaking the resulting file

Your default.conf file contains most of the info that you need, but it doesn’t yet tell vpnc to use the TAP driver. The following lines do that:

# This next line contains the name you renamed 
# your TAP network connection to:
Interface name TAP_Driver
# This line tells vpnc that you're doing TAP, not TUNnelling.
Interface mode tap
# For some reason that I don't understand, vpnc doesn't work under cygwin
# unless it's still attached to the console it was running from,
# so don't detach:
No Detach

Something else that you need to do is to tell vpnc which local port to listen on. Windows boxes already have LSAS (Local Security Authentication Server) listening on UDP 500 (which is vpnc’s default), so you have to tell vpnc to use something else.

# Tell vpnc to select a random free port instead of using 500
Local Port 0

There’s some more that you can put in there for ease of use. The following lines are useful, but note that your password is in there in plain text – you MUST restrict access to this file on your Windows system to keep it safe.

# Login credentials
Xauth username Somebody.Clever
Xauth password biggusbrainus

Fixing the bug in the routing script

When vpnc starts up it needs to add some entries to your routing tables. It needs to redirect all traffic to the VPN concentrator through one of your normal internet gateways, and then it needs to redirect all traffic to the addresses on the VPN through the vpnc interface. It manages this by executing the file in /etc/vpnc/vpnc-script.

This file is a little shell script that actually does the changes in the Unix world, but in the Windows world it delegates the work to a javascript script in /etc/vpnc/vpnc-script-win.js. If you look at this script, you’ll see it’s just a hundred lines or so of code that adds some entries to your routing tables. Bizarrely, in the version of vpnc that I downloaded (0.5.1), there is a bug in this file: it generates broken “route” command lines, and so the second set of routing entries (redirecting all VPN addresses down the TAP interface) don’t get added. Look at line 80 – the bug should be fairly obvious!

This is the diff of the fix:

$ diff vpnc-script-win.js /etc/vpnc/vpnc-script-win.js
80c80
<                       run("route add " + network + " mask " + netmask +
---
>                       run("route add " + network + " mask " + netmask

As you can see, there’s an extra “+”. I’ve told the author about this, but I can’t believe nobody else has already – this release of vpnc just doesn’t work for cygwin.

Finally – start the thing already!

Okay, you’re now ready to kick it off. From your cygwin session, run:
$ /usr/local/sbin/vpnc

You should get output that looks like this:

Microsoft (R) Windows Script Host Version 5.7
Copyright (C) Microsoft Corporation. All rights reserved.

Device: TAP_Driver
TAP-Win32 Driver Version 8.4 
Warning: using insecure memory!
Microsoft (R) Windows Script Host Version 5.7
Copyright (C) Microsoft Corporation. All rights reserved.

VPN Gateway: 83.244.123.33
Internal Address: 192.168.100.109
Internal Netmask: 255.255.255.0
Interface: "TAP_Driver"
Configuring "TAP_Driver" interface...
done.
Configuring networks:
Route configuration done.
--------------------------------------------------
Your company's banner text will go here!
--------------------------------------------------
VPNC started in foreground...

…and you’ll see in your Windows UI that the disconnected TAP interface is now connected. It should resolve you an address over DHCP and then you’re away – you’re on the VPN. To exit, just kill the process in cygwin (bring your cygwin window to the front and hit ctrl-C). This won’t remove any of the routing entries from your routing tables, but they won’t do any harm and they’ll disappear the next time you reboot.

The annoying bit…

Just as I’d finished getting all of this working on my machine at home (and writing this long winded howto), a colleague of mine pointed out that there is another Windows based OSS project that will do this with a lot less hassle. I’m not going to tell you which one because I was sulking too much to take notice, and maybe he’s fibbing. I checked, and OpenVPN can’t do it, so I’m hopeful. Anyway, this was still fun and hey… Cygwin’s cooler than a Windows UI based thing any day. Right?

…right?

If we can crack how to choose the next problem to work on, our engineers, designers and developers can then come up with wonderful and innovative solutions for them. Solving them in a way that is compelling and easy to use is hard, but is at least a problem that is tangible: how to best solve a particular problem within a certain set of constraints.

Somewhat surprisingly, it is much less easy to decide what to do in the first place, particularly if the market moves quickly, competitors are trying to catch up, and resources are tight. And in which software company is this not the case?!?

The framework promoted by Pragmatic Marketing goes some way towards answering the problem, but isn’t the complete answer either. They suggest that the business goes through several stages to identify what it needs to do:

• First, determine what the Distinctive Competence of your business is. Whatever it is, this is the basis for having a business that cannot easily be undercut or overtaken by another company that lacks your particular competence.
• Next, based on the DC, try to find out what problems exist in the market that you have or could build a solution for. The problems you want to identify are the ones that meet a few key criteria: a) They have to be common, b) They have to have budget/money available to solve them, and c) They have to have a deadline by which they must be solved.
• The research your Product Managers to do get here is likely to throw up a list of potential problems that can be ranked by pseudo-scientific factors such as urgency, uniqueness, number of reachable customers, value to customers, etc.
• At some point, R&D gets involved to determine whether the topmost really-valuable-problem has a solution that it’s actually possible to build in a reasonable amount of time. Yes, everyone would really like to buy a time machine, but that doesn’t mean that is where we should devote all of our R&D resources…

The above is neat, and certainly is heads and shoulders above other commonly used methods, such as when the development organization itself decides the priorities: old code needs to be refactored, libraries need to be updated, code streams need to be integrated, new platforms need to be supported, user interface needs to be redesigned use a common framework, etc. Even though all of these could be real and urgent priorities, they of course can’t stand alone. A great and well-maintained code-base by itself doesn’t make for a sustainable business.

The most common real source of priorities is customers and existing users: just build the features or fix the defects that your customers ask for, on the premise that since they pay your bills and have already bought the product, they are able to spot where it is weak or otherwise could benefit from more functionality.

The problem with this is that these people already have bought your product; by meeting their demands for more functionality, you may not be focusing your efforts on those areas that would be most effective in getting new customers to join the existing ones. At the same time, if you never do any of the things your customers want, your customer satisfaction and reputation as a company that is receptive to customer needs are likely to nosedive – so there clearly needs to be a balance.

Pragmatic also realizes that there are lots of actors involved in the process of buying and using software: the Buyer has the budget and decision power, the Technical Evaluator decided whether it meets the requirements, and the User is often stuck with what the Buyer and Evaluator between them decide is good enough.

Looking at it from the user’s perspective, it isn’t just solutions to problems that makes for great products. What problem does the iPhone solve that other phones don’t? It is unique in that it feels good, and is pleasant to look at – but I don’t think market research could have found that people thought their Nokias were particularly deficient in this regard. What we have here is a set of requirements that all relate to creating a product that it’s a pleasure to use. How many pieces of software are both helpful and pleasant to look at while being genuinely useful to the user of it? Most do what they need to do, while coming across as obtuse, unhelpful and annoying – Microsoft Project is a great example of software that checks all of the feature boxes, but in practice is horrible to work with.

To further complicate matters, your customers hopefully consists of more than just those that have paid for your product and expertise, but also includes a larger community of users that make use of your community or free versions. Having a strong community around your product offering is a great way of letting people benefit from the value of your product well in advance of actually paying anything for it – at Tideway, we have recently launched a full-featured Community Edition of Foundation for just this purpose.

Since the barrier of entry for a free version is a lot lower than when you have to pay a lot of money for it, the number of customers gets exponentially larger with a community edition – which is fantastic and great stuff! On the other hand, it also means that the list of features and suggestions that come from your customers gets very long indeed, and probably contains a LOT of useful nuggets. How do we prioritise these along with all of the other things we need to do???

Oh, and the list doesn’t end here. The Community demands and needs a place to go in order to function properly; the expectations for how well the community web site (with forums, feeds, user profiles, rating systems, etc) needs to work is high. And it should look and feel similar to the product, ideally functioning as a natural extension of the product: a dynamic repository of ideas, tips, best practices, questions, how-tos, FAQs and other enablers that is easily usable from and blends into the product.

To get this experience, the same designers and developers that develop the product should therefore work on the web site, blurring the lines between the two. And so, here is another huge source of requirements that again needs to be balanced against the others in the ongoing struggle for priority.

To recap, there are a lot of possible ways to spend your R&D budget, and you need to spend some on several of them:

• Build solutions to new market problems
• Build solutions to paying-customer problems
• Build solutions to free-version problems
• Build solutions to allow your community to thrive
• Make the product a pleasure to work with
• Evolve the architecture of the product
• Fix defects in older versions

The last two are necessary to make sure the product is maintainable and doesn’t degenerate into a quagmire of buggy, old, spaghetti code. But how much time do we devote to each of them, and how do we decide what specifically to work on in a given iteration or release? The items are of vastly different scope and are all important in their different ways! Argh!

I believe the answer requires two different aspects: A way to manage and show all of the things that are candidates, and a way to decide on their prioritisation in small increments. How to go about actually doing that is the subject of Part 2 of this topic. Stay tuned!

Borrowing Michael Porter’s thinking from ‘What is Strategy’, (1996), successful IT departments will be those that can capture a unique competitive position. Porter goes on to say that “A period of imitation may be inevitable in emerging industries, but that period reflects the level of uncertainty rather than a desired state of affairs. In high-tech industries this imitation phase often continues much longer than it should…” Unfortunately, for many IT departments, replication and imitation of the strategies and technologies used by competitor companies is the norm. In fact, while many CIOs and their strategic advisors focus on operating in step with industry standard performance metrics, this me-to approach to IT management steers them clear of ever reaching a unique competitive position within their own organisations.

I’m reminded of the impact of replication and imitation when changing employer. While I worked at Sun Microsystems (UK) as a technical project manager I was part of a group which made use of process, procedure, some technology, and the strengths of our relationships throughout the organisation. As a group, we were very successful. We got things done. However, when I changed employer, and began replicating the same tools, procedures and approaches in the new company I wasn’t able to imitate the success I had experienced previously. One principle cause of this may have been that in the new company, the relationships and knowledge I had to help me get things done – all part of my ‘system’ for the role – were all new to me. I was left replicating very well, but imitating very badly. Certainly within the UK banking IT industry, whole teams have been known to hop from one employer to the next. Each time a team hops, it may try to replicate what it knew to be a successful strategy elsewhere, but inevitably this approach relies on process, procedure and technology to be replicated alongside the team. At best this may succeed in introducing improved efficiency and effectiveness, but sadly, while both are very healthy attributes for an IT department to have, they are not strategy.

Everything in the IT department exists to support the provision of services to the business. This includes all the technology, the people, and the process and procedure that the department defines as valuable. For each business organisation, creating a unique and valuable position means creating a fit among all the activities of the organisation which sets that organisation apart from its competitors. Ditto for the IT department, and since the successful strategy of the IT department has a dependency on what it can achieve uniquely for the business, its value chain in the organisation should focus on activities which support the business’ chosen strategy, and not the other way round.

Competitive strategies for the business include: low-cost provider; broad differentiation; best-cost provider; niche focus based on low costs; and niche focus based on differentiation. How many IT departments can talk about their IT strategy with the business in these terms? How many IT departments can determine that the business services IT underpins facilitate corresponding offensive or defensive strategies in the market place? These are important questions for an IT department to ask since in turn they guide and help formulate the IT strategy in supporting the business. In other words, rather than simply apportioning IT effort to business services which generate the most revenue by volume, a better focus would be to review the generic strategy of the organisation and align IT behind it.

Having a focus on generic strategies which fit with the strategy the company is pursuing; inline with the mission and vision, requires a very different alignment of IT strategy than most IT departments are used to. Doing more with less and becoming more efficient are not enough, and is not really strategy anyway. Instead, truly understanding the organisation’s business, the business services which support the business model (how the company generates revenue – converting product/service to cash), the underlying technology and its relevance to those business services, is a clear starting point. By understanding the complete end-to-end value chain in the provision of IT to support each business service, IT departments can and will become better than competitors in measurable ways. Why, because few of their competitors can do this.

To build on strategic offensives, IT departments will need to base their approach on maximising the value of the organisation’s competitive IT assets and strengths, whatever these may be. These will not be well understood without a full view of the IT estate, the underlying components and all the dependencies in between. As an exception, in emerging industries and technologies, it may not always be easy to determine the chosen generic strategy, nor the supplementary or complementary strategy being adopted. This makes reciprocal offensive strategies ineffective since it becomes difficult to determine if they are truly impacting competitor’s weaknesses. Similarly, with a blue ocean strategy – where the market and industry are too new to characterise – the focus for the company may well be to simply create demand for their product. The IT strategy then has less need for being offensive, but still has a dependency on fully understanding the IT estate and the value to the business.

Strategic decisions around whether to partner or form a strategic alliance, or whether to outsource an element or portion of the value chain, will also impact how successful IT departments become. For example, if a decision is made to outsource elements of IT because the department is weak in managing or supporting that area, while competitors are building strength into theirs, they need to choose their outsourcer carefully. The also need to now exactly what it is that they are outsourcing. Again the starting point is having a complete map of the IT environment, and then knowing the strengths and weaknesses apparent across the IT estate.

One observation we might make about businesses today, especially in the technology sector, and in those industries dependent on IT, is that the only strategy in the beginning is to find new customers and visionaries to promote the product among their industry peers. With a new technology product, or a market ready for a new technology, there needs to be knowledge flow between the market and the company to share information on customer requirements and product function. The same is true of the IT department. New technology is introduced because it can be, seems cool, and keeps IT interested, but without that complete 360-degree view of the IT estate, IT can’t share the intent behind its strategy, if indeed it has one.

Many IT departments have an unfailing belief in how successful they are going to be and how valuable they are to the organisation just because they provide technology. In some cases, these same IT departments exist despite not having a real strategy to support the business. An expectation exists that simply introducing new technology leads to business success whether there is strategy behind the technology drive or not. It may be that this drives IT departments to focus on two things: how to win favour among the lines-of-business heads, and therefore gain budget; and how to support lines-of business heads who shout loudest for new technology. Trading an IT strategy on the back of early praise fro technology simply means that sustainable competitive advantage becomes the next CIOs problem.

For many CIOs and IT managers, it may be easier to focus on technology product features, and become lost in decision making issues relating to price battles and industry consolidation as and when strategic convergence occurs. In such circumstances, thinking about IT strategy may not be immediately convenient, and these IT departments, especially for those with entrepreneurial characteristics rather than managerial skills, it may seem easier to jump between generic competitive strategies in the hunt for technology success, rather than focussing on one generic strategy and benefiting from longer term rigour in pursuing business alignment with a suitable fit of activities tuned for competitive advantage. In this scenario of focussing on implementing technology via any strategy, benchmarking and continuous comparison between products and emerging suppliers may just lead to all IT departments doing the same activities and missing opportunities to create unique value for their business.

If we take Google as an example of a company which built a unique competitive position, the company’s approach to the technology it used – it built is own – demonstrates a willingness to be different. As a company, Google had followed a broadly differentiated competitive strategy with its alternative technology for web page ranking. At the same time Google picked off its competitors with complementary offensive strategies, targeting its competitor’s weaknesses by attacking the market leaders using its company resources – great brains behind the product, and a new approach to using cheap computing power to do its search engine number crunching. In other words, how Google used IT was different to its competitors – different enough, and sufficiently aligned to the business strategy to be useful and effective.

The difference between Google and the majority of IT departments is that Google was starting from scratch. Most IT departments have already made technology investments; are already stepping through the life-cycle of technology upgrades; are already services providers to their respective businesses; and are already bound by service levels to differing degrees. To be different, strategic and successful in their use of IT they need to first understand their starting point; what technology do they have; where is it used; who uses it; how is it used; what dependencies exist; and what areas of the business benefit from which technology components.

The 5 generic competitive strategies and supplementary strategic actions which can be taken seem both broad and extensive – certainly enough to cover most competitive strategy combinations. The danger for most IT departments is that their IT strategy is limited to determining how they can imitate and replicate technology, drive efficiency and effectiveness, and ultimately fail to appreciate how what they have aligns to business services. Instead, for IT departments to contribute in creating a unique competitive position for their organisations they might focus on being different. They might focus on gaining a deep understanding of the technology they already use; understanding how this technology supports the business processes, relationships between key components and dependencies between software, hardware and the underlying infrastructure technology; and of course aligning what they have in support of the organisation’s chosen strategy.

These factors bring us to a point of imperfection which hinders strategic analysis. We have a tendency to complicate issues with both irrelevant information, and a lack of focus on what we actually need to know. However, theory on strategic thinking, planning and execution offers tools for industry and competitor analysis and techniques for evaluating resource capabilities, relative cost position and competitive strength. These tools and techniques, allow managers to zero in on what is important, and these same tools can be used by IT management to kick start their own strategic thinking, but first they will need to know what they don’t know.

Analysis is extremely important for strategic thinking. Without analysis of the information available, strategic options cannot be defined, and the most appropriate option cannot be chosen. No analysis, no plan. No plan, no action. Similarly, since analysis is likely to be an ongoing activity, the use of focused thinking on the important external factors influencing the company – competition, market forces and competitor strategies – all set the scene for thinking through how the organisation can use its balance sheet of strategic assets and liabilities. For an IT department, the strength of its IT resources – assets and relationships between those assets in supporting business process – becomes the anchor stone.

Textbooks on strategy discuss methods to address the external analysis required for strategy analysis and offer approaches to provide solid answers to the questions an organisation needs to ask of itself. These methods are activity based and include identifying an industry’s dominant Economic Features, a Five-Forces Model of Competition analysis, the identification of an Industry’s Driving Forces, Strategic Group Mapping, Monitoring Rivals, identifying Key Factors for future success, and an analysis of Industry outlook in terms of the attractiveness of the opportunity.

For IT departments the internal strategic analysis can include finding answers to questions on how well the current strategy is working, strengths, weaknesses, opportunities and strengths, price and cost competitiveness relative to 3rd party resources, relative competitive strength, and immediate issues to fix. In other words IT management can look at the internal environment of the IT organisation using tools commonly known as SWOT analysis, value chain analysis, benchmarking and competitive strength assessment. These tools help IT management analyse the company, helping to match the external requirements to the internal capabilities. One question remains though – how do we really analyse the IT assets.

Using Application Dependency Mapping (ADM) to begin the assessment of how well the current IT strategy is working is a valid approach. Firstly, analysis of the existing ADM for an IT environment produces facts about how IT is configured in the collections of services that underpin business processes. Understanding the patterns of IT configuration relative to industry standards and competitor approaches (where known) is important. Secondly, when assessing the effectiveness of the current strategy, we need to understand that strategic plans take time to implement and time to take effect. There may be a tendency to analyse the ADM models and asset attributes on a quarterly basis, to see if the strategy is being effective, when in fact it may take a year or more to implement some IT initiatives. Tweaks in the strategy from quarter to quarter may not produce visible results until several quarters later. What is important though is that by analysing change in the ADM models and asset attributes through change analysis and node comparison, it is possible to both demonstrate that any given IT strategy is being implemented and adhered to, and to show the impact of changes as they are made in the environment. This then becomes the source of truth which triggers action and reaction to strategy success and failure, and can occur daily when using automated IT discovery solutions.

Like SWOT analysis, the result of interpreting an IT team’s design and build documentation can lead to subjective findings. For example, concluding that an IT team has strong process and procedures to help it achieve operational excellence is an easy conclusion to draw. It is also a statement often made by CIOs to the board. However, ADM, auto-discovery and indexing of IT assets may show a very different story where perhaps patch and build management are shown to be inconsistent, and where versioning data highlights the disparate editions of the same software package in use across the organisation. What we claim as strength of the IT department may not actually be one. Conversely, ADM and automated discovery of IT assets can clearly show the weaknesses embedded in the IT infrastructure and these weaknesses can be dealt with in the strategic plan.

In today’s world of IT knowledge workers we might want to include resource flexibility as a ‘strength measure’ – how quickly can the IT organisation grow or shrink its resource pool of human capital. Qualitative observations are often used to assess the size and skills of the resource pool required to operate an IT environment and complete or the strategic initiatives required. These qualitative observations are sometimes based on ‘gut feel’ rather than fact. Again, with ADM we can quickly assess the complexity of the IT environment and complete quantitative assessment of what resources are required to support the current front-burner issues and future IT strategic plans.

Defining front-burner issues requiring attention, draws managers in to dealing with aspects of the company strategy they can address now. The danger with this is that they keep on dealing with front-burner issues and don’t spend enough time on the wider strategic issues – witness the fire-fighting approach of many IT teams. That said, zero-ing in on the most important strategic issues is probably the most important aspect of the strategic analysis and provides the key input to deciding on the strategy to follow. ADM and auto-asset discovery provides a view of which potential strategic issues the IT organisation should focus on first, not just the ones currently on fire.

In conclusion we can say that not knowing what strategy is and how to use the models and theory available will prevent us from correctly starting the analysis process. The analysis itself is important in helping us to understand the macro-environment in which the IT environment operates and in knowing its own strengths and weaknesses compared to competitors and other internal departments. However, the analysis should be focused in order to be useful and achievable. From this analysis we can begin to map out the strategic vision of where the IT department needs to be in the future and what its initial direction is going to be. Once the strategic vision is understood, possible strategic options can be identified and assessed. While implementing the best strategic option, IT organisations should continue strategic analysis and make corrective actions as required, remembering the ‘rubbish in, rubbish out’ rule, and remembering that strategic analysis is about creating options. When the analysis is based on current, accurate data about the IT services the organisation supports, IT can mature to the next level – drawing conclusions from proven, actionable data to make strategic improvement.