Blog Administration
Wednesday, February 23. 2005
Pay No Attention to That Man Behind the Curtain
The Navy Marine Corps Intranet (NMCI) is a mandatory, omnibus information technology (IT) contract run by the U.S. Department of the Navy (DoN). The current prime contractor is EDS Corporation. All active duty military and DoD civilians within the Department of the Navy (U.S. Navy and U.S. Marine Corps) are obligated to use this contract for current and future technology needs such as networks, computers, servers, software and IT infrastructure.
Awarded in the year 2000, the first "seat" was activated in September of 2001. Over 500,000 users will be converted over from legacy systems by the end of fiscal year 2005. At least that’s the plan. Initial plans for the contract - or so it was rumored - included all voice and data requirements of the DoN. This would include management, design, acquisition, maintenance and operation of a huge host of services including voice telephone service, cellular phone and pager service, satellite connectivity, tactical communications, microwave and voice radio connectivity and, of course, anything in the computer and IT realm.
This "super contract" was quickly scaled down to focus on IT.
EDS Corporation won the bid for the contract that, at the time, was awarded for $6.9 Billion. One of the largest contracts of its kind in the history of the DoD. Mr. Robert Cringely has an excellent article on the finer points of the bidding process in his article which includes a great analysis of why and how EDS under-bid the contract.
The Emperor Has No Clothes!
Funding for the contract has bloated up from the initial $6.9 Billion to over $8.8 Billion as of 2004. The DoN continues to pour money into the contract in an effort to resuscitate both EDS and the ideal of outsourced IT. It is unclear what impact this expenditure -- the initial costs and the over-run -- will have on future DoN budgets as the contract was never approved by the U.S. Congress.
Navy officials continue to tout the success of the contract despite huge outcries from the actual users who have to suffer with the equipment and services provided by the contract. In Cringely's second article he points out that one size does not fit all and it is folly to attempt this with such a large, diverse and technically skilled workforce.
Mr. Cringely also highlights a trend that goes beyond the Navy and beyond NMCI. That is, the changing role of "outsourcing" in America. It used to be that a company who made Widget A decided that they wanted or needed to make Widget B. So they outsourced the production of Widget B so they could focus on what they do best: making Widget A. The trend in outsourcing now is different. Companies who make both Widget A and Widget B decide that they are doing a bad job with Widget B so, through outsourcing, they try to find someone else who screws it up less than they do.
The Navy can do IT. EDS is not better at IT than the Navy. Someone just needed a plan. Instead they roped up $8.8 Billion and gave the job to someone else, hoping they would screw it up less. The Navy is better than EDS and any leader who doesn’t believe that should not be a leader.
User "satisfaction surveys" are whitewashed or gilded to show what a great success the effort has been. A 55% satisfaction level was touted as a testimony for the great level of worker satisfaction with NMCI performance.
In 2003 and 2004 there were a large number of laptop computers that began experiencing hard drive failures. NMCI refused to release the actual numbers of failures. Data began to be collected anecdotally at each facility in an effort to gain some visibility into the problem. Why would NMCI conceal the failure rates for equipment and force the Government to waste time trying to recreate statistics? One word: SLA. Well, that's an acronym... but it's kind of like a word.
Service Level Agreements (SLA) are how EDS gets paid. It is how they make bonus. It is simply a metric used to determine if the contractor is meeting, failing or exceeding expectation thresholds. Obviously he who controls the data, controls his own fate. If you don’t release the results of surveys and tests, nobody can say you are not meeting your minimums. Additionally, the contract has an up-to-speed clause that states EDS is not required to meet any SLAs until they can demonstrate, for 90 consecutive days, that they can accomplish the minimum requirements of the contract. So as long as things don’t function correctly for 90 days at a time, they are not responsible for meeting SLAs. Excellent planning.
Service response times are one of the most common complaints of NMCI users. The admirals can't figure this one out. Every time they call the help-desk on a computer issue, the technician is standing at the door before the admiral can put the phone back in the cradle. No service problems here.
Imagine the sailor, underway on a Navy ship. He has no administrator privileges on his computer and his IT support staff is 300 miles away over open water. Giving the sailors the tools they need… or not.
Come On In. The Water is Fine.
In an effort to bolster support (or spread the misery) the Navy has tried to convince other organizations, from the U.S. Coast Guard to NASA, to adopt either the NMCI contract or a similar omnibus vehicle. Officially, these other departments are adopting a wait-and-see attitude. Unofficially they are running for the hills.
The U.S. Air Force has a huge technology core to support such things as its fighter/bomber aircraft and its enormous space-based (satellite) groups. It has yet to belly up to the bar to drink from the bitter cup of NMCI despite the salesmanship of senior Navy officials.
Snake Oil! Get Your Snake Oil here!
When the work force was first told about NMCI, it was touted as a move towards a leaner, more efficient, standardized approach that was going to save the Navy money. A cost-cutting measure. In fact, that was the justification given for spending $6.9 Billion with little or no oversight. The funds would be repaid with the savings from leaning out the IT. Distributed help desks and network gurus could be removed from the hundreds of military bases and field sites and centralized into a core. Efficiency would skyrocket.
After the initial "rollout" of seats began (and of course the contract had been awarded), people started to question the cost savings being touted. Inquiries to produce and explain the accounting behind these supposed cost savings were met with hand-waving and throat-clearing. EDS, from a business perspective, was nearly crushed. Huge cost overruns began to surface and the Navy and EDS started running out of fingers to plug holes in the dike. The impact to the Navy is incalculable in terms of lost work hours, reduced capability and general disruption. These affects are still being felt by many users trying to bring their NMCI computers up to speed just to do their job.
Time to change gears. Forget that cost savings stuff. Advancement can be expensive. NMCI will be a secure network. Ya, that's it. It will be more secure than the existing, diverse legacy networks and equipment. Except for the fact that in 2003, thousands of supposedly secure machines and networks were infected with exactly the same worms as millions of other Windows PCs and servers around the world. In 2005, what was only rumored as, "NMCI was hacked" led to thousands of users being forced to change their passwords under emergency-like conditions. The details of this compromise were understandably never released but we hope NMCI administrators took notice.
Additionally, one would also need to raise an eyebrow at the entire concept of allowing a contractor organization to be in control of Government data. Not necessarily from a classified/unclassified standpoint -- although that could warrant some thought also -- but more from the concept of impartiality. All the servers and desktop machines are under the control of EDS. Where does a procurement official store procurement-sensitive data? Would that procurement person even know if EDS, a contractor, was looking at their data? How are personnel, medical and financial data being secured? Since September 11th the whole concept of "sensitive" data has changed. Who has access to this data? O.k. so NMCI’s not so secure, let's try this again.
We will achieve greatness through standardization. Ya, that's it. Everyone knows standardization is a good thing. Certainly in the mid 1990s, back when well-meaning leaders were nurturing the seeds of NMCI, there was reason to be concerned with the state of standardization. Word Perfect battled Microsoft Word. Lotus and Excel shot it out in the spreadsheet arena. Novell had no use for TCP/IP. VMS and Unix flexed their muscles.... If a command or organization did adopt a standard, it was invariably a different standard from the organization down the street.
Yet as we fast forward to 2001 (and certainly 2005) we see the Navy has changed as much as every other technology-centric organization in the developed world. Microsoft suites are now standard. They are even dictated by things like the IT21 initiative. TCP/IP is the standard for networking and, again, concepts like ForceNet mandated it as a standard over technologies like ATM. PCs, running either Windows or Linux, are already standardized in the DoD and are even making their way into the traditionally proprietary world of embedded and tactical systems. How old is the "standardization" problem they are trying to solve? "Damn the torpedoes! Full speed ahead!"
So, What's the Problem?
So now that we’ve talked in generalities about unfounded claims of cost savings, security lapses and the quest for standardization, perhaps some more specific examples would be in order.
Users are subjected to a managed Windows desktop. Managed, meaning they have very little control over not only the look of their Windows desktop but the functionality of the computer as a whole. Certainly not something that is reserved solely for NMCI, many corporate IT departments limit what end-users can install on their machines. Under many circumstances this makes sense to reduce the risk of virus infections and instability caused by users installing software packages they shouldn’t. But what do the users get?
As of 2005:
Windows 2000 (Phased out with the introduction of XP in 2001)
Netscape 4.7 (Phased out in 1999)
Internet Explorer 5.0 (Phased out in 2002)
Office 2000 (Phased out in 2002)
Why are all these items so outdated? Because EDS is on a three-year update cycle, at least with the hardware. Obviously the software and OS are on even longer cycles. Understandably, if you are buying 500,000 computer suites, you can't be expected to update everything the instant the OEM changes versions. However, many Navy customers were told (either officially or otherwise) that NMCI has adopted a "one behind" policy on software. That is, users will not get Windows XP until the rest of the world has moved on to XP's successor. There is no question that NMCI made an effort to downgrade many of the PCs supplied by Dell under the contract, uninstalling EOM software. NMCI laptop users can frequently turn their laptops over and find the OEM sticker for Windows XP on the bottom.
Part of the promise of increased efficiency included disbanding local help desks and network support centers so that these tasks could be centralized. In fact many of the legacy support personnel were promised jobs with EDS once they were removed from their positions with the Navy. That benefit not withstanding -- in fact some reports are that these people were quickly let go after being hired -- it appears that many bases have quietly recreated their Government support infrastructure even though they are converted to NMCI. They either do it unofficially, assigning people "collateral duties" in addition to their regular tasks or they simply never disbanded their legacy support systems. These "shadow help desks" create duplication of effort and negate any purported benefit of NMCI centralization. Why do the Navy personnel do this when they could simply let NMCI fail? Because they are there to get the job done. Whatever it takes.
The Dot Dev Account
So, what if the user needs special application software or newer versions of popular applications that are not included as part of the managed desktop? Well, for a price, NMCI will sell you the administration rights to your own computer. It's called a "dot dev" or S&T (science and technology) or developer's seat. But there are several catches, even with this.
As a .dev user, you do have the ability to install most software but the .dev user is still not granted full rights. A quick trip to the Windows on-line update site at windowsupdate.microsoft.com greets the .dev user with the warning: "Network policy settings prevent you from using Windows Update to download and install updates on your computer."
But wait, there's still more. Just because you have the ability to install software through the OS doesn’t mean you have the permission to install it. If the software is not on the approved list of applications, it can not be installed. Users of .dev accounts sign a contract stating that they will not install, download or compile any software that has not been approved. One could ask what good such a system would do for a programmer. Every single time he changes code and recompiles he is essentially creating an unapproved piece of software on his NMCI seat.
The .dev user has no access to E-mail while logged in to his development account. So any applications that would normally interface with Outlook (e.g. Palm Organizer or PDA software) are useless. Additionally, the .dev user has to log out of the development account and log in to his normal, non-development account several times a day just to check E-mail. For many E-mail-centric users, the .dev account is almost never used because of this reason. However he still gets charged extra, every day, just for having the privilege.
Lastly, as a developer's seat, your help-desk support also gets placed into a special category. Unless you pay for it, you are not provided software support for any application, even those that are supplied as part of the basic package (called the Gold Disk). So if your MS Word application becomes corrupt and refuses to load, even if it was not the result of anything done by the .dev user, the NMCI help desk is not required to assist with this.
We can do that! How Much Money Do You Have?
Like most Firm Fixed Price contracts (if you can call $6.9B to $8.8B "fixed") NMCI has a Contract Line Item Number (CLIN) list. Each CLIN has an item number and the user can pick from this list just like a menu. If it isn’t on the list, they can't do it. There are some changes allowed to this list over the lifespan of the contract. However, it would be incorrect to call the pricing "dynamic". If the cost of a P4 laptop next year is half the price of this year's model, don't expect to see the NMCI CLIN for a laptop go down by half. There is no explicit requirement for the contractor to keep up with technology or prevalent pricing of that technology.
So how bad is the pricing? The public CLIN list is available at http://www.nmci-isf.com/clinlist.pdf in PDF format or http://www.nmci-isf.com/clinlist.htm in HTML [Note: As of mid-2006 these URLs have been pulled by EDS]. So for about $4000 a year you can get a fairly capable machine (although the software is three years out of date). So not too bad. Ummm, remember, you have to pay $4000 every year and when you're done with that PC, NMCI gets it back. Let's see. You need to change cubicles and want to plug your PC in to the existing wall jack in your new cube? That's an "Add/Change/Move" action and you have to pay NMCI for that.
When commands need to pay for NMCI, there appears to be no clear-cut method for funding such a transformation. There was certainly no core funding coming directly from CNO or the upper echelons of the DoN. Even commands like NAVAIR and NAVSEA left their subordinate bases to fend for themselves. Many sites chose to put a "tax" on all base funding sources in an effort to spread the misery. So Navy facilities, who were undertaking joint work with the Air Force or the Army, had to tax their Air Force customers to pay for NMCI. It is unclear how this is being reported to the Air Force or other non-DoN activities. This is especially difficult for large projects that never pre-estimated the cost of NMCI into their joint projects and are now looking at short-falls due to the unforeseen NMCI tax.
Its Ours Now
So what happened to all the PCs the Navy had before NMCI came along? They belong to EDS! As part of the contract, all existing IT assets became the property of EDS. The Navy, who had acquired billions of dollars worth of computers, networking equipment and infrastructure with taxpayer money, simply gave everything to a contractor. So if a tech-savvy command had just purchased the latest "loaded" PCs with huge amounts of memory, large hard drives and state of the art processors, it didn't matter. The computers were turned over to EDS and replaced with five-year old technology and six-year old software at twice the cost.
Conversely, of course, if the command had been limping along with outdated, inadequate technology, NMCI could offer something better. Although people who felt their NMCI machines were "upgrades" are few and far between, if you trust many of the surveys in publications like Federal Computer Week. It would appear that administrative and secretarial users are happier with their NMCI seats than the engineers, scientists, analysts and project managers. One could ask which segment we are really trying to support with NMCI.
Stop Your Whining. What's the Solution?
The proponents of NMCI can point to some successes brought about by the NMCI contract. There is no doubt they exist. Yet if one looks closely, you can see that many of these "successes" are the result of Navy employees, not EDS and certainly not anything that NMCI brought to the party. Hard working Sailors and Marines using USB memory sticks, swapping hard drives and sneaking legacy laptops into their labs so they can do the work they need to do. Why don't they just let the mission fail and have NMCI take the justified heat? Because they care more about the Navy and Marine Corps than the NMCI leadership does.
EDS is not a criminal. There is nothing, as of yet, to indicate corruption within the NMCI contract. EDS made some incredibly stupid mistakes. The Navy made some incredibly stupid mistakes. Unfortunately the American Taxpayer is now left to pay for those mistakes.
A solution to the NMCI debacle could only come from a clear understanding of the initial problem that NMCI was introduced to solve. The problem statement that spawned NMCI has not been clearly communicated to the rank and file Navy employees. It certainly has not been communicated to Government leaders outside of the DoN – if they even knew to ask. Intentional obfuscation or negligent research, the outcome is the same. NMCI is a solution to a problem that doesn't exist. It is now a problem in its self.
Make the NMCI contract optional. Let commands prove themselves without being forced to foot the bill for some else's ideal of standardization and efficiency.
The foggy misdirection created by claims of improved efficiency do nothing to point to NMCI as a solution. In fact there is no evidence that NMCI is more cost effective or more efficient than anything the Navy could have done organically. It may require Congressional pressure to find out exactly what is right and what is wrong with NMCI. We can only hope that such an inquiry would go beyond the, "Pay no attention to that man behind the curtain!" shouts from the Navy leaders. The working Sailors, Marines and civilians know better.
Awarded in the year 2000, the first "seat" was activated in September of 2001. Over 500,000 users will be converted over from legacy systems by the end of fiscal year 2005. At least that’s the plan. Initial plans for the contract - or so it was rumored - included all voice and data requirements of the DoN. This would include management, design, acquisition, maintenance and operation of a huge host of services including voice telephone service, cellular phone and pager service, satellite connectivity, tactical communications, microwave and voice radio connectivity and, of course, anything in the computer and IT realm.
This "super contract" was quickly scaled down to focus on IT.
EDS Corporation won the bid for the contract that, at the time, was awarded for $6.9 Billion. One of the largest contracts of its kind in the history of the DoD. Mr. Robert Cringely has an excellent article on the finer points of the bidding process in his article which includes a great analysis of why and how EDS under-bid the contract.
The Emperor Has No Clothes!
Funding for the contract has bloated up from the initial $6.9 Billion to over $8.8 Billion as of 2004. The DoN continues to pour money into the contract in an effort to resuscitate both EDS and the ideal of outsourced IT. It is unclear what impact this expenditure -- the initial costs and the over-run -- will have on future DoN budgets as the contract was never approved by the U.S. Congress.
Navy officials continue to tout the success of the contract despite huge outcries from the actual users who have to suffer with the equipment and services provided by the contract. In Cringely's second article he points out that one size does not fit all and it is folly to attempt this with such a large, diverse and technically skilled workforce.
Mr. Cringely also highlights a trend that goes beyond the Navy and beyond NMCI. That is, the changing role of "outsourcing" in America. It used to be that a company who made Widget A decided that they wanted or needed to make Widget B. So they outsourced the production of Widget B so they could focus on what they do best: making Widget A. The trend in outsourcing now is different. Companies who make both Widget A and Widget B decide that they are doing a bad job with Widget B so, through outsourcing, they try to find someone else who screws it up less than they do.
The Navy can do IT. EDS is not better at IT than the Navy. Someone just needed a plan. Instead they roped up $8.8 Billion and gave the job to someone else, hoping they would screw it up less. The Navy is better than EDS and any leader who doesn’t believe that should not be a leader.
User "satisfaction surveys" are whitewashed or gilded to show what a great success the effort has been. A 55% satisfaction level was touted as a testimony for the great level of worker satisfaction with NMCI performance.
In 2003 and 2004 there were a large number of laptop computers that began experiencing hard drive failures. NMCI refused to release the actual numbers of failures. Data began to be collected anecdotally at each facility in an effort to gain some visibility into the problem. Why would NMCI conceal the failure rates for equipment and force the Government to waste time trying to recreate statistics? One word: SLA. Well, that's an acronym... but it's kind of like a word.
Service Level Agreements (SLA) are how EDS gets paid. It is how they make bonus. It is simply a metric used to determine if the contractor is meeting, failing or exceeding expectation thresholds. Obviously he who controls the data, controls his own fate. If you don’t release the results of surveys and tests, nobody can say you are not meeting your minimums. Additionally, the contract has an up-to-speed clause that states EDS is not required to meet any SLAs until they can demonstrate, for 90 consecutive days, that they can accomplish the minimum requirements of the contract. So as long as things don’t function correctly for 90 days at a time, they are not responsible for meeting SLAs. Excellent planning.
Service response times are one of the most common complaints of NMCI users. The admirals can't figure this one out. Every time they call the help-desk on a computer issue, the technician is standing at the door before the admiral can put the phone back in the cradle. No service problems here.
Imagine the sailor, underway on a Navy ship. He has no administrator privileges on his computer and his IT support staff is 300 miles away over open water. Giving the sailors the tools they need… or not.
Come On In. The Water is Fine.
In an effort to bolster support (or spread the misery) the Navy has tried to convince other organizations, from the U.S. Coast Guard to NASA, to adopt either the NMCI contract or a similar omnibus vehicle. Officially, these other departments are adopting a wait-and-see attitude. Unofficially they are running for the hills.
The U.S. Air Force has a huge technology core to support such things as its fighter/bomber aircraft and its enormous space-based (satellite) groups. It has yet to belly up to the bar to drink from the bitter cup of NMCI despite the salesmanship of senior Navy officials.
Snake Oil! Get Your Snake Oil here!
When the work force was first told about NMCI, it was touted as a move towards a leaner, more efficient, standardized approach that was going to save the Navy money. A cost-cutting measure. In fact, that was the justification given for spending $6.9 Billion with little or no oversight. The funds would be repaid with the savings from leaning out the IT. Distributed help desks and network gurus could be removed from the hundreds of military bases and field sites and centralized into a core. Efficiency would skyrocket.
After the initial "rollout" of seats began (and of course the contract had been awarded), people started to question the cost savings being touted. Inquiries to produce and explain the accounting behind these supposed cost savings were met with hand-waving and throat-clearing. EDS, from a business perspective, was nearly crushed. Huge cost overruns began to surface and the Navy and EDS started running out of fingers to plug holes in the dike. The impact to the Navy is incalculable in terms of lost work hours, reduced capability and general disruption. These affects are still being felt by many users trying to bring their NMCI computers up to speed just to do their job.
Time to change gears. Forget that cost savings stuff. Advancement can be expensive. NMCI will be a secure network. Ya, that's it. It will be more secure than the existing, diverse legacy networks and equipment. Except for the fact that in 2003, thousands of supposedly secure machines and networks were infected with exactly the same worms as millions of other Windows PCs and servers around the world. In 2005, what was only rumored as, "NMCI was hacked" led to thousands of users being forced to change their passwords under emergency-like conditions. The details of this compromise were understandably never released but we hope NMCI administrators took notice.
Additionally, one would also need to raise an eyebrow at the entire concept of allowing a contractor organization to be in control of Government data. Not necessarily from a classified/unclassified standpoint -- although that could warrant some thought also -- but more from the concept of impartiality. All the servers and desktop machines are under the control of EDS. Where does a procurement official store procurement-sensitive data? Would that procurement person even know if EDS, a contractor, was looking at their data? How are personnel, medical and financial data being secured? Since September 11th the whole concept of "sensitive" data has changed. Who has access to this data? O.k. so NMCI’s not so secure, let's try this again.
We will achieve greatness through standardization. Ya, that's it. Everyone knows standardization is a good thing. Certainly in the mid 1990s, back when well-meaning leaders were nurturing the seeds of NMCI, there was reason to be concerned with the state of standardization. Word Perfect battled Microsoft Word. Lotus and Excel shot it out in the spreadsheet arena. Novell had no use for TCP/IP. VMS and Unix flexed their muscles.... If a command or organization did adopt a standard, it was invariably a different standard from the organization down the street.
Yet as we fast forward to 2001 (and certainly 2005) we see the Navy has changed as much as every other technology-centric organization in the developed world. Microsoft suites are now standard. They are even dictated by things like the IT21 initiative. TCP/IP is the standard for networking and, again, concepts like ForceNet mandated it as a standard over technologies like ATM. PCs, running either Windows or Linux, are already standardized in the DoD and are even making their way into the traditionally proprietary world of embedded and tactical systems. How old is the "standardization" problem they are trying to solve? "Damn the torpedoes! Full speed ahead!"
So, What's the Problem?
So now that we’ve talked in generalities about unfounded claims of cost savings, security lapses and the quest for standardization, perhaps some more specific examples would be in order.
Users are subjected to a managed Windows desktop. Managed, meaning they have very little control over not only the look of their Windows desktop but the functionality of the computer as a whole. Certainly not something that is reserved solely for NMCI, many corporate IT departments limit what end-users can install on their machines. Under many circumstances this makes sense to reduce the risk of virus infections and instability caused by users installing software packages they shouldn’t. But what do the users get?
As of 2005:
Windows 2000 (Phased out with the introduction of XP in 2001)
Netscape 4.7 (Phased out in 1999)
Internet Explorer 5.0 (Phased out in 2002)
Office 2000 (Phased out in 2002)
Why are all these items so outdated? Because EDS is on a three-year update cycle, at least with the hardware. Obviously the software and OS are on even longer cycles. Understandably, if you are buying 500,000 computer suites, you can't be expected to update everything the instant the OEM changes versions. However, many Navy customers were told (either officially or otherwise) that NMCI has adopted a "one behind" policy on software. That is, users will not get Windows XP until the rest of the world has moved on to XP's successor. There is no question that NMCI made an effort to downgrade many of the PCs supplied by Dell under the contract, uninstalling EOM software. NMCI laptop users can frequently turn their laptops over and find the OEM sticker for Windows XP on the bottom.
Part of the promise of increased efficiency included disbanding local help desks and network support centers so that these tasks could be centralized. In fact many of the legacy support personnel were promised jobs with EDS once they were removed from their positions with the Navy. That benefit not withstanding -- in fact some reports are that these people were quickly let go after being hired -- it appears that many bases have quietly recreated their Government support infrastructure even though they are converted to NMCI. They either do it unofficially, assigning people "collateral duties" in addition to their regular tasks or they simply never disbanded their legacy support systems. These "shadow help desks" create duplication of effort and negate any purported benefit of NMCI centralization. Why do the Navy personnel do this when they could simply let NMCI fail? Because they are there to get the job done. Whatever it takes.
The Dot Dev Account
So, what if the user needs special application software or newer versions of popular applications that are not included as part of the managed desktop? Well, for a price, NMCI will sell you the administration rights to your own computer. It's called a "dot dev" or S&T (science and technology) or developer's seat. But there are several catches, even with this.
As a .dev user, you do have the ability to install most software but the .dev user is still not granted full rights. A quick trip to the Windows on-line update site at windowsupdate.microsoft.com greets the .dev user with the warning: "Network policy settings prevent you from using Windows Update to download and install updates on your computer."
But wait, there's still more. Just because you have the ability to install software through the OS doesn’t mean you have the permission to install it. If the software is not on the approved list of applications, it can not be installed. Users of .dev accounts sign a contract stating that they will not install, download or compile any software that has not been approved. One could ask what good such a system would do for a programmer. Every single time he changes code and recompiles he is essentially creating an unapproved piece of software on his NMCI seat.
The .dev user has no access to E-mail while logged in to his development account. So any applications that would normally interface with Outlook (e.g. Palm Organizer or PDA software) are useless. Additionally, the .dev user has to log out of the development account and log in to his normal, non-development account several times a day just to check E-mail. For many E-mail-centric users, the .dev account is almost never used because of this reason. However he still gets charged extra, every day, just for having the privilege.
Lastly, as a developer's seat, your help-desk support also gets placed into a special category. Unless you pay for it, you are not provided software support for any application, even those that are supplied as part of the basic package (called the Gold Disk). So if your MS Word application becomes corrupt and refuses to load, even if it was not the result of anything done by the .dev user, the NMCI help desk is not required to assist with this.
We can do that! How Much Money Do You Have?
Like most Firm Fixed Price contracts (if you can call $6.9B to $8.8B "fixed") NMCI has a Contract Line Item Number (CLIN) list. Each CLIN has an item number and the user can pick from this list just like a menu. If it isn’t on the list, they can't do it. There are some changes allowed to this list over the lifespan of the contract. However, it would be incorrect to call the pricing "dynamic". If the cost of a P4 laptop next year is half the price of this year's model, don't expect to see the NMCI CLIN for a laptop go down by half. There is no explicit requirement for the contractor to keep up with technology or prevalent pricing of that technology.
So how bad is the pricing? The public CLIN list is available at http://www.nmci-isf.com/clinlist.pdf in PDF format or http://www.nmci-isf.com/clinlist.htm in HTML [Note: As of mid-2006 these URLs have been pulled by EDS]. So for about $4000 a year you can get a fairly capable machine (although the software is three years out of date). So not too bad. Ummm, remember, you have to pay $4000 every year and when you're done with that PC, NMCI gets it back. Let's see. You need to change cubicles and want to plug your PC in to the existing wall jack in your new cube? That's an "Add/Change/Move" action and you have to pay NMCI for that.
When commands need to pay for NMCI, there appears to be no clear-cut method for funding such a transformation. There was certainly no core funding coming directly from CNO or the upper echelons of the DoN. Even commands like NAVAIR and NAVSEA left their subordinate bases to fend for themselves. Many sites chose to put a "tax" on all base funding sources in an effort to spread the misery. So Navy facilities, who were undertaking joint work with the Air Force or the Army, had to tax their Air Force customers to pay for NMCI. It is unclear how this is being reported to the Air Force or other non-DoN activities. This is especially difficult for large projects that never pre-estimated the cost of NMCI into their joint projects and are now looking at short-falls due to the unforeseen NMCI tax.
Its Ours Now
So what happened to all the PCs the Navy had before NMCI came along? They belong to EDS! As part of the contract, all existing IT assets became the property of EDS. The Navy, who had acquired billions of dollars worth of computers, networking equipment and infrastructure with taxpayer money, simply gave everything to a contractor. So if a tech-savvy command had just purchased the latest "loaded" PCs with huge amounts of memory, large hard drives and state of the art processors, it didn't matter. The computers were turned over to EDS and replaced with five-year old technology and six-year old software at twice the cost.
Conversely, of course, if the command had been limping along with outdated, inadequate technology, NMCI could offer something better. Although people who felt their NMCI machines were "upgrades" are few and far between, if you trust many of the surveys in publications like Federal Computer Week. It would appear that administrative and secretarial users are happier with their NMCI seats than the engineers, scientists, analysts and project managers. One could ask which segment we are really trying to support with NMCI.
Stop Your Whining. What's the Solution?
The proponents of NMCI can point to some successes brought about by the NMCI contract. There is no doubt they exist. Yet if one looks closely, you can see that many of these "successes" are the result of Navy employees, not EDS and certainly not anything that NMCI brought to the party. Hard working Sailors and Marines using USB memory sticks, swapping hard drives and sneaking legacy laptops into their labs so they can do the work they need to do. Why don't they just let the mission fail and have NMCI take the justified heat? Because they care more about the Navy and Marine Corps than the NMCI leadership does.
EDS is not a criminal. There is nothing, as of yet, to indicate corruption within the NMCI contract. EDS made some incredibly stupid mistakes. The Navy made some incredibly stupid mistakes. Unfortunately the American Taxpayer is now left to pay for those mistakes.
A solution to the NMCI debacle could only come from a clear understanding of the initial problem that NMCI was introduced to solve. The problem statement that spawned NMCI has not been clearly communicated to the rank and file Navy employees. It certainly has not been communicated to Government leaders outside of the DoN – if they even knew to ask. Intentional obfuscation or negligent research, the outcome is the same. NMCI is a solution to a problem that doesn't exist. It is now a problem in its self.
Make the NMCI contract optional. Let commands prove themselves without being forced to foot the bill for some else's ideal of standardization and efficiency.
The foggy misdirection created by claims of improved efficiency do nothing to point to NMCI as a solution. In fact there is no evidence that NMCI is more cost effective or more efficient than anything the Navy could have done organically. It may require Congressional pressure to find out exactly what is right and what is wrong with NMCI. We can only hope that such an inquiry would go beyond the, "Pay no attention to that man behind the curtain!" shouts from the Navy leaders. The working Sailors, Marines and civilians know better.
Saturday, February 19. 2005
Some Prophetic Quotes
"...it's much better than what we had before, and we could not function without NMCI."
-- FCW Interview, 17 September 2004. Mr. Gordon England, Secretary of the Navy
"People forget where we were before NMCI — we've come a long way. If you measure where we are compared to where we were, you see just how far that is."
-- 22 June 2004. Mr. Gordon England, Secretary of the Navy
"That's the way it is with my own personal [America Online] account."
-- 22 June 2004. Mr. Gordon England, Secretary of the Navy, dismissing what he called a "few bugs" in the system.
Resistance is futile. You will be assimilated.
"If you don't like it, leave. Because we're going to do this. Resistance to it is costing me money and costing me time, and I won't stand for it. I'll plow through or over anybody and do whatever it takes. We're not doing NMCI because it's a cute idea, but because it will provide a bridge — a road — to the efficiencies we want to achieve."
-- 24 March 2004. Adm. Michael Mullen, Vice Chief of Naval Operations
"We didn't have 100,000 apps, as the Navy did. I wish we did, because that would mean that we had the money to buy them!"
-- 23 March 2004. Brig. Gen. John Thomas, Marine Corps' CIO
"I don't know why you replace something that is free, simple and easy with something that is expensive, complicated and shackled,"
-- 15 August 2004. Unnamed Navy Captain in Government Week Magazine interview
"NMCI will come on this base over my dead body."
-- 18 Feb 2004. Anonymous Commanding General, California Marine Corps Base
Let's give the Airlines $8.8 Billion and see if they can tell the difference between Seattle and San Diego
"Thanks to NMCI for delivering my e-mail to San Diego this morning, as opposed to the airline, which delivered my luggage to Seattle last night."
-- 5 Feb 2004. Capt. Kevin Uhrich, director of the Naval Networks Division of the Navy Network Warfare Command
"NMCI is a six year-old solution to a ten year-old problem that hasn't existed for eight years."
-- Overheard an an IT conference in reference to the outdated technology supplied as part of the contract.
Someone hits the nail on the head!
"Part of the problem is standardization, which looks great if you are part of a command that hasn't had much in the way of IT resources, but looks terrible if you are with a leading-edge unit that had all sorts of toys, only to have NMCI take most of them away. One size definitely does not fit everyone, and it was a grave mistake on the part of the Navy to even pretend that it could."
-- 25 March 2004. Mr. Robert X. Cringely
Thursday, December 2. 2004
NMCI and GCCS
Federal Computer Week reports that NMCI failed to support the Common Operating Environment (COE) which is the backbone of DOD IT and several enterprise-wide applications including Global Command and Control System (GCCS) and other afloat tactical software. The Defense Information Infrastructure (DII) COE was developed in the early 90's and codified into a set of guidelines and standards for software including the OS (kernel), data exchange services and support (messaging) applications.
The shortcoming came to light at Hawaii's new Nimitz-MacArthur Pacific Command Center. Navy officials had to stand up a work-around GCCS network to meet the needs of their joint partners. Simply one example of the hundreds of networks and equipment suites erected (or never removed) to compensate for shortcomings in NMCI. Despite the failings of NMCI to support command and control systems, the Pacific Command reports that they are "generally happy" with NMCI as it allowed them to supply desktop PCs to the support staff. Although to read the other report, it sounds as if NCMI solved a procurement problem, not an IT problem.
Is this a Navy problem or an EDS problem? Yes. Did the Navy ask EDS to support DII COE before the contract was bid? Does the EDS bill of goods actually have the infrastructure necessary to support DII COE? Did anyone think about this before a multi-million dollar facility had to punt to compensate for failure?
The shortcoming came to light at Hawaii's new Nimitz-MacArthur Pacific Command Center. Navy officials had to stand up a work-around GCCS network to meet the needs of their joint partners. Simply one example of the hundreds of networks and equipment suites erected (or never removed) to compensate for shortcomings in NMCI. Despite the failings of NMCI to support command and control systems, the Pacific Command reports that they are "generally happy" with NMCI as it allowed them to supply desktop PCs to the support staff. Although to read the other report, it sounds as if NCMI solved a procurement problem, not an IT problem.
Is this a Navy problem or an EDS problem? Yes. Did the Navy ask EDS to support DII COE before the contract was bid? Does the EDS bill of goods actually have the infrastructure necessary to support DII COE? Did anyone think about this before a multi-million dollar facility had to punt to compensate for failure?
Monday, November 1. 2004
Wolf Guarding the Hen House
"EDS stands to reap financial rewards when NMCI customer service levels reach certain benchmarks, beginning at 85 percent. As customer satisfaction levels rise, EDS has the potential to earn as much as $100 per NMCI 'seat' for each financial quarter, according to program officials."
A follow-up article in Government Executive Magazine sheds (a little) more light on the NMCI "survey results" released by EDS Corporation earlier this year. An important take away from the piece is the fact that NMCI stands to make money -- a welcome change for them and NMCI -- if the satisfaction level reaches 85 percent. They just reached 80 percent satisfaction and are closing in on the 85 percent threshold. Wow, this NMCI thing must really be working well!
Hmmmm, let us look at the finer points of the "survey":
1. The survey was designed, constructed, distributed, collected and analyzed by EDS Corporation, the same company that stands to gain financially from a high satisfaction percentage.
2. The questions have not been released for review.
3. The sample size and response rate of the survey has not been released.
4. The detailed results (e.g. response statistics for individual questions) have not been released.
5. The Navy has expressed confidence in the survey results and EDS.
Interesting. Not sure which Navy officials GE Magazine interviewed to determine that they had confidence in the results but it was apparently someone who hasn't taken a high-school science class.
The article goes on to offer several quotes from national pollsters and statisticians such as the Gallup Organization, stating how there could be an appearance of impropriety in the fact that EDS did their own survey and now refuses to release details. Really? Wow.
Unlike the previous GE article on the survey, (see reference elsewhere on nmcistinks.com) for this article GE actually managed to track down someone who took the survey. The user reported that he was asked questions like, "Was the help desk person polite?" to which he was forced to respond honestly in the affirmative. However he said there was no opportunity for him to footnote that response with, "However, the help desk person failed to do anything about my problem." Another satisfied user!
Again, perhaps the most disturbing aspect of this entire episode is the Navy's overt support for the "survey" process and results. The official position of the Navy is that releasing the details of the study would compromise their ability to successfully conduct future surveys. How profound.
A final note. We should thank the folks at publications like Government Executive Magazine and Federal Computer Week for digging these stories out of the back rooms. I can only hope that congressional staffers and watchdog organizations are reading them as fervently as we are.
Saturday, October 30. 2004
Can You Hear Me Now
Federal Computer Week and other sources are reporting that EDS Corporation is planning to roll out "enterprise-based" cellular phone service as part of the NMCI contract. The contract would be managed by FISC San Diego. The move is expected to go forward by the end of 2004.
NMCI officials are touting the move as a cost savings measure, claiming savings can be realized by economies of scale. EDS is saying this would provide the DoN with a contract vehicle to negotiate for the best deal possible on cellular voice service and mobile data for Research In Motion BlackBerry devices.
Once again we hear the famous "economies of scale" argument -- the ultimate in sophomoric, restaurant and hotel management double-speak -- and as usual we hear it with no data to back up the claims. It seems that this tactic works consistently because nobody actually bothers to do postmortem analysis of these claims to see if they ever hold true. Hence the reason we have the NMCI debacle in the first place, now crawling out from a $2 Billion-plus overage.
It is unclear what advantage there is to adding EDS and NMCI into the process of obtaining cellular provider contracts. The economies of scale argument has merit (e.g. pooling of minutes, large user-base contract breaks, etc.) but these advantages are independent of the profit-skimming, middle man aspect of NMCI. Navy and Marine Corps facilities already have enterprise-wide contracts with the likes of AT&T Wireless, Nextel and others. The Navy is free to contractually pool these users between bases and commands already, as they always were, without the "help" of NMCI. Large corporations like Motorola and Ford manage to negotiate cellular contracts every day without the help of NMCI.
What infrastructure does EDS and NMCI bring to the party? How many cellular towers and backhaul circuits do they have to support cellular users?
If NMCI chooses to flex its muscle and interpret the contract as an enabler for this type of service, that's fine. If they would like to interpret the contract even more liberally and deem themselves as the required source for this type of service, that's fine. But don't blow smoke up our legs and tell us we are going to save money by outsourcing this. We've heard that somewhere before.
NMCI officials are touting the move as a cost savings measure, claiming savings can be realized by economies of scale. EDS is saying this would provide the DoN with a contract vehicle to negotiate for the best deal possible on cellular voice service and mobile data for Research In Motion BlackBerry devices.
Once again we hear the famous "economies of scale" argument -- the ultimate in sophomoric, restaurant and hotel management double-speak -- and as usual we hear it with no data to back up the claims. It seems that this tactic works consistently because nobody actually bothers to do postmortem analysis of these claims to see if they ever hold true. Hence the reason we have the NMCI debacle in the first place, now crawling out from a $2 Billion-plus overage.
It is unclear what advantage there is to adding EDS and NMCI into the process of obtaining cellular provider contracts. The economies of scale argument has merit (e.g. pooling of minutes, large user-base contract breaks, etc.) but these advantages are independent of the profit-skimming, middle man aspect of NMCI. Navy and Marine Corps facilities already have enterprise-wide contracts with the likes of AT&T Wireless, Nextel and others. The Navy is free to contractually pool these users between bases and commands already, as they always were, without the "help" of NMCI. Large corporations like Motorola and Ford manage to negotiate cellular contracts every day without the help of NMCI.
What infrastructure does EDS and NMCI bring to the party? How many cellular towers and backhaul circuits do they have to support cellular users?
If NMCI chooses to flex its muscle and interpret the contract as an enabler for this type of service, that's fine. If they would like to interpret the contract even more liberally and deem themselves as the required source for this type of service, that's fine. But don't blow smoke up our legs and tell us we are going to save money by outsourcing this. We've heard that somewhere before.
Sunday, October 24. 2004
NMCI Runs Afoul Of DISA
In what can only be viewed as an effort to expand their empire, EDS and NMCI have been bragging about their efforts to utilize voice-over-IP technology to provide voice (telephone) service to its customers. NMCI has not yet convinced anyone that they are contractually entitled to take over traditional switched telephone service on Navy and Marine Corps facilities. VoIP offers them an avenue to carry voice services over their existing IP-based infrastructure.
Except for one problem. They might be breaking DoD regulations by doing it.
A Bit of Background
Voice-over-IP popularity exploded in 2002 or even earlier. Dozens of vendors offered hundreds of solutions from full-blown IP PBXs to desktop phones to box-to-box solutions. The salivating communications guru was greeted with nearly unlimited options to solve his voice requirements. VoIP was an elegant solution because of several reasons.
It was designed from the ground up to interface with traditional analog and digital phone switches so the engineer could create hybrid systems. It was easy to encrypt as individual circuits or in bulk. It offered a scalable solution where hundreds of users could be carried on a single CAT-5 cable as opposed to the huge bundles of copper required for traditional phone systems. It was extremely efficient in terms of bandwidth usage not only because of the packetized nature but because it did away with the 64 kbps DS0 barrier. It allowed "toll skipping" which could potentially save thousands of dollars on long distance charges.
Yet VoIP was not without its drawbacks. In an organization as technically and geographically diverse as the DoD, there are bound to be incompatibilities between VoIP systems and traditional phone systems (or even other VoIP systems). This problem was not helped by the VoIP industry that -- like nearly every other standards-based organization in the world -- failed to agree on universal protocols and interfaces. There was also the question of security. Who was making sure that these new-fangled VoIP boxes hitting the market didn't have security flaws or even malicious back door compromises in them? This was certainly a large concern for any organization but especially the US Military.
A Big Hand of Guidance
Enter DISA. The Defense Information Systems Agency is tasked with ensuring Information Assurance within the DoD and have, for better or worse, built an empire controlling information and communications systems within the Department of Defense. Their charter is beyond the scope of this posting but their web site has lots of information.
In April 2004 the Field Security Operations Division of DISA released the Voice over Internet Protocol Security Technical Implementation Guide (VoIP STIG) which gives DoD facilities guidance and best practices for implementing VoIP. Of course the focus of this document is on security and Information Assurance. This document is based on the overarching principles and authority of DoD Directive 8500.1.
The VoIP STIG does an excellent job of pointing out the vulnerabilities of VoIP (which will be left as an exercise to the reader) but more importantly identifies the urgent need for interoperability and security testing of any IP-based system. The need for this, especially in a command and control (C2) environment, should be self-evident.
Who does this interoperability and security testing? The Joint Interoperability Test Command out of Fort Huachuca, Arizona is the primary command currently allowed to "bless" these technical solutions. Simply put, if a VoIP solution has not been tested and approved by JITC, it is not supposed to be connected to a DoD phone network. And certainly not to a Defense Switched Network (DSN) backbone.
NMCI Says It Isn't Us
So who would dare violate a DISA directive? Who would subvert DoD 8500.1? Certainly not NMCI. Or would they?
Apparently NMCI and possibly others have run afoul of the DISA directive because an April 2004 advisory from the Navy's Network Warfare Command (NETWARCOM), in cooperation with the Marine Corps Network Operations and Security Command (MCNOSC), notified several commands that they were in non-compliance with DISA directives.
It states that at least 16 DoN commands have uncertified VoIP systems. This number resulted from a telephone switch inventory done as part of the National Defense Authorization Act FY2003 (Public Law 107-314). Although it does not state categorically that these non-certified systems are NMCI offspring, the advisory makes direct reference to the EDS VoIP solution and states that commands are not allowed to implement any VoIP solution without JITC certification.
It would be interesting to know how much JITC/DISA oversight was done on the VoIP solutions that NMCI is bragging about. Apparently it was not as complete as it should have been or NETWARCOM would not have felt the need to remind Navy and Marine Corps users of the certification and approval process. The rules are there for everyone.
Capt. Chris Christopher, deputy Director of future operations for NMCI has stated that making telephone calls over VoIP was "inevitable." Certainly it is. Let's just hope that NMCI is not usurping DISA in its effort to hasten the inevitable.
Except for one problem. They might be breaking DoD regulations by doing it.
A Bit of Background
Voice-over-IP popularity exploded in 2002 or even earlier. Dozens of vendors offered hundreds of solutions from full-blown IP PBXs to desktop phones to box-to-box solutions. The salivating communications guru was greeted with nearly unlimited options to solve his voice requirements. VoIP was an elegant solution because of several reasons.
It was designed from the ground up to interface with traditional analog and digital phone switches so the engineer could create hybrid systems. It was easy to encrypt as individual circuits or in bulk. It offered a scalable solution where hundreds of users could be carried on a single CAT-5 cable as opposed to the huge bundles of copper required for traditional phone systems. It was extremely efficient in terms of bandwidth usage not only because of the packetized nature but because it did away with the 64 kbps DS0 barrier. It allowed "toll skipping" which could potentially save thousands of dollars on long distance charges.
Yet VoIP was not without its drawbacks. In an organization as technically and geographically diverse as the DoD, there are bound to be incompatibilities between VoIP systems and traditional phone systems (or even other VoIP systems). This problem was not helped by the VoIP industry that -- like nearly every other standards-based organization in the world -- failed to agree on universal protocols and interfaces. There was also the question of security. Who was making sure that these new-fangled VoIP boxes hitting the market didn't have security flaws or even malicious back door compromises in them? This was certainly a large concern for any organization but especially the US Military.
A Big Hand of Guidance
Enter DISA. The Defense Information Systems Agency is tasked with ensuring Information Assurance within the DoD and have, for better or worse, built an empire controlling information and communications systems within the Department of Defense. Their charter is beyond the scope of this posting but their web site has lots of information.
In April 2004 the Field Security Operations Division of DISA released the Voice over Internet Protocol Security Technical Implementation Guide (VoIP STIG) which gives DoD facilities guidance and best practices for implementing VoIP. Of course the focus of this document is on security and Information Assurance. This document is based on the overarching principles and authority of DoD Directive 8500.1.
The VoIP STIG does an excellent job of pointing out the vulnerabilities of VoIP (which will be left as an exercise to the reader) but more importantly identifies the urgent need for interoperability and security testing of any IP-based system. The need for this, especially in a command and control (C2) environment, should be self-evident.
Who does this interoperability and security testing? The Joint Interoperability Test Command out of Fort Huachuca, Arizona is the primary command currently allowed to "bless" these technical solutions. Simply put, if a VoIP solution has not been tested and approved by JITC, it is not supposed to be connected to a DoD phone network. And certainly not to a Defense Switched Network (DSN) backbone.
NMCI Says It Isn't Us
So who would dare violate a DISA directive? Who would subvert DoD 8500.1? Certainly not NMCI. Or would they?
Apparently NMCI and possibly others have run afoul of the DISA directive because an April 2004 advisory from the Navy's Network Warfare Command (NETWARCOM), in cooperation with the Marine Corps Network Operations and Security Command (MCNOSC), notified several commands that they were in non-compliance with DISA directives.
It states that at least 16 DoN commands have uncertified VoIP systems. This number resulted from a telephone switch inventory done as part of the National Defense Authorization Act FY2003 (Public Law 107-314). Although it does not state categorically that these non-certified systems are NMCI offspring, the advisory makes direct reference to the EDS VoIP solution and states that commands are not allowed to implement any VoIP solution without JITC certification.
It would be interesting to know how much JITC/DISA oversight was done on the VoIP solutions that NMCI is bragging about. Apparently it was not as complete as it should have been or NETWARCOM would not have felt the need to remind Navy and Marine Corps users of the certification and approval process. The rules are there for everyone.
Capt. Chris Christopher, deputy Director of future operations for NMCI has stated that making telephone calls over VoIP was "inevitable." Certainly it is. Let's just hope that NMCI is not usurping DISA in its effort to hasten the inevitable.
Monday, August 30. 2004
Silent Majority or Manipulation
"Several service personnel, however, said they do not know anyone who has taken the survey."
A recent article in Government Executive Magazine reports that the satisfaction level is "near 80 percent and rising" within the ranks of Navy Marine Corps Intranet users. The accuracy of these numbers will be left to the reader to ponder but there is a very important threshold not far from this 80 percent number. When the "satisfaction level" reaches 85 percent, NMCI contractor EDS Corporation is entitled to bonus payments for every NMCI seat, regardless of the satisfaction level of that seat user or the base where it is installed.
The article goes on to discuss the fact that a short and informal survey by GE staff could find no NMCI users who had actually taken the survey. More importantly NMCI officials have not released the details of the survey including number of surveys sent, number returned (sample size), the questions on the survey and a myriad of other factors that could lead to an honest, stoichiometric analysis of the vaunted survey.
Perhaps most disturbing, Navy officials have stood by quietly accepting, if not supporting, this junk science being proffered by EDS.
« previous page
(Page 5 of 5, totaling 35 entries)
