NMCI and Other Nefarious Adventures - NMCI Follies

Big Brother or Just Plain Incompetence

nospam@example.com (Charlie) — Sat, 03 Feb 2007 09:43:56 -0800

Some days it's hard to tell if the back-room folks at NMCI are doing something intentionally or if they are just bumbling around and do something by accident. Let's just say the latest networking oddity wasn't preannounced, unless we missed that memo.

It appears that they are blocking Google cache. Again, this "blocking" appears to the user as simply a network error. There is no announcement screen stating that the requested Google cache pages are blocked by policy. This effect also appears to come and go. Some days the cached information is available, some days it isn't.

The purpose for such a Big Brother Block is obvious. The cached pages on Google (other than imagery) allow users to circumvent any site-specific blocking done by NMCI administrators. Our site, for example, is very much available in Google cache, even to an otherwise blocked NMCI host.

Of course the legitimate uses for Google cache make this latest draconian restriction more than just a little annoying. Some sites are extremely slow or have recently gone down for some reason. Or some times all you need is the text from a site and don't want to wait for pictures to load -- the "cached text only" option on Google cache is great for this. Of course none of these things are available if the cache is blocked.

What's the solution? Simple. Find a different way to skin that cat! Use another search engine and access the cache information there. A great example is MyWay.com which provides a front-end to Google, Yahoo, Ask.com and others without annoying banner ads and pop-ups. Simply enter your search terms and let it do a default search (currently defaults to ask.com). When it pulls up the information, choose "Google" from the tabs at the top. This is the same information available at the regular Google.com page. Now all your cache information is available. Or at least the text is.

You could be reading this page right now from your NMCI seat! Happy Googling!

Ya Who Not You

nospam@example.com (Charlie) — Fri, 14 Oct 2005 01:25:00 -0700

Once again we see fallacy in action. NMCI has apparently started blocking web access to Hotmail, Yahoo Mail, Google Mail and other on-line, web-based mail sites. So what could their reason be for such a drastic measure?

Very simply this is DoN policy. That's right folks, don't blame the good folks at EDS for this one. The Navy has deemed access to unofficial E-mail as an unacceptable risk to the integrity of their internal networks. Certainly users could unsuspectingly download a virus, worm, Trojan, malware or other Nasty in the form of an E-mail or attachment.

Yet one needs to ask, how is downloading such a threat from an E-mail site any more likely than acquiring one from a regular website? What makes web E-mail any more of a threat?

In fact, there are several things that make such a stance even more illogical and lay fallacy to the idea that web-based E-mail is a higher threat.

First, most large web-based E-mail services do scanning on attachments. Yahoo, AOL, Juno and others all scan their user's E-mail for suspicious items. So most of these sites are probably less likely to have dangerous code on them than millions of other regular websites.

Secondly, in the same vein, how are the file links on a regular website checked when a user clicks on them? Perhaps in the NMCI HTTP proxy system, but does this checking occur for SSL sites? Point is, if there is a threat on web mail sites, then there is a threat on all web sites. Maybe they should block access to everything except .mil and .gov sites. I know. Don't give them any ideas.

Third, many of the recent vulnerabilities publicized for Microsoft products do not need a user-executed code vector to do their business, they exploit flaws in the browser itself. There are several easily exploited weaknesses that Microsoft has been quick to address in its current Internet Explorer products and Windows XP-SP2. Once again we see the problems with having an outdated OS and application software on NMCI seats. So if the exploit is on the webpage itself, and has nothing to do with a user downloading a malicious attachment, some of the safest sites on the net would be reputable sites like Hotmail, Yahoo and AOL.

Lastly, we wonder where the NMCI proxy administrators got their list of web-based E-mail sites. How complete could it be? How many thousands of Squirrelmail sites and small ISP sites are out there?

Their method for "blocking" these sights appears to be little more than DNS modification. So someone at the central site has gone through the domain name servers and put manual entries in for things like mail.yahoo.com and www.hotmail.com. These new entries redirect the user to a warning banner instead of taking them to the requested mail web page. Of course this means that any local user (S&T .dev account not required) can modify their own hosts table to override these same DNS entries. For Windows 2000 users this means simply going into c:\winnt\system32\drivers\etc\hosts and placing a few entries like:

66.218.75.184 mail.yahoo.com
64.233.185.83 mail.google.com

In reality it would prove a little more difficult as many mail sites use many different host names and jump around during the user session. It all comes down to following the rules -- the admins can make things more difficult and keep out the casual user but if someone insists on breaking the rules, they will.

There was also widespread speculation that the stance resulted from (or the fear of) users forwarding Government E-mail to non-Government servers -- for whatever reason. This opens up the possibility of FOUO, Privacy Act, Procurement Sensitive and even classified data (which would point to bigger problems) being sent to servers outside the Government's control. The thought of NMCI going to Yahoo and asking them to wipe their terrabytes of hard drives simply to remove one FOUO memo is not something anyone wants to deal with. It is still unclear how restricting web access to these remote sites has anything to do with someone writing an auto-forward rule in Outlook. Keeping them from reading it is different than keeping them from sending it.

At some point it all comes back to trusting the users to follow the policy. If the policy states that DoN users are not allowed to access personal E-mail from an NMCI seat, then at some point that has to be good enough. People are going to put infected floppies, USB drives and CDs into their seats. People are going to forward inappropriate mail to outside addresses. People are going to visit compromised web sites with an outdated browser. People are going to open infected attachments received through NMCI/Outlook. Web-based E-mail is probably the least of our worries. Blocking major websites is nothing more than window dressing. Once again....

The SPAM Debacle Appears To Be Over

nospam@example.com (Charlie) — Tue, 19 Jul 2005 19:50:44 -0700

"The day is ours, the bloody dog is dead." -- Mr. Shakespeare

In what can only be described as a tacit admission of utter and complete failure, NMCI administrators have disabled the tagging of spam in association with the "word lists" method described elsewhere in these pages.

You heard it here first. We told you it wouldn't work.

Not only did it not work to defeat spam, it caused horrendous disruption for thousands of Navy and Marine Corps users. Rampant false positives left a user's mailbox littered with erroneous warnings about "Sexual content", "profanity" and "Proprietary content".

The warnings were changed cosmetically early on in the debacle to include the words "May contain..." which was apparently done in an effort to make users feel better about false positives and the associated ineffective treatment of spam. Another cosmetic change was made from "sexual content" to "unauthorized content". This was not only completely useless, but was most likely technically incorrect for places like Naval hospitals where legitimate e-mails probably contain words that are both sexual in content and authorized.

One official from the East coast instructed managers to tell complaining users that, "...this is the solution to SPAM and unwanted e-mail users have been asking for." Really? We would be interested to see the requests to use such an ill-conceived method to address the spam issue.

The erroneous tagging was more than just annoying for the primary recipients. The tags, adulterating both the body of the E-mail and the subject line, had to be scrubbed before the message could be forwarded or replied to. Wasting the Government's time and resources once again, thank you NMCI.

No white-listing or "intelligent" source analysis was done on the E-mails. Simply confirming the message source (through IP address, not sender fields) as .mil or .gov addresses could have easily abated some of the mess and probably reduced loading on the systems parsing thousands of messages a day. Of course there never was any facility for the users to do custom white-lists of their own.

User's quarantined messages were to be deleted in seven days, regardless of activity from the intended recipient. So if a user was on TDY, vacation or away from an NMCI seat for more than seven days his E-mail would begin to be permanently purged having never seen the light of day. There was no provision for the user to extend this time period or disable the "feature" of quarantine all together.

We are still completely baffled why IronPort would cast its shadow on such an idiotic undertaking. SpamCop.net, Bonded Sender Program, Senderbase.org and C-Series Appliances all speak to IronPorts ability to provide very effective spam handling. What were they thinking?

Running Startup Scripts

nospam@example.com (Charlie) — Sun, 24 Apr 2005 09:03:00 -0700

Why does it take so long for an NMCI machine to boot up? Exactly what is the machine doing when it boots? Does anyone know?

We took a small sampling of boot up times for both laptop and desktop seats. Remember this is relatively current hardware running Windows 2000 -- not state of the art but certainly not something out of the 80s. These are average times.

Laptop -- From Boot to Login: 3 Minutes, 37 Seconds
Laptop -- From Login to Desktop: 2 Minutes, 18 Seconds
Total Unusable Time: 5 Minutes 55 Seconds

Desktop -- From Boot to Login: 3 Minutes, 30 Seconds
Desktop -- From Login to Desktop: 2 Minutes, 12 Seconds
Total Unusable Time: 5 Minutes, 42 Seconds

Of course NMCI attempts to mitigate these times by instructing users to not turn off their machines at night. Of course this flies in the face of the energy saving initiatives at many sites. More importantly, many laptop users have discovered -- the hard way -- that their laptop hard drives are not rated for continuous duty. Leaving these machines on 24 hours a day is the kiss of death for the older Dell hard drives.

It should be noted that some machines took significantly longer to boot, so much so that they were not even recorded in this sample and dismissed as aberrant. One desktop machine took over 10 minutes after log in to present a usable desktop. Several laptops on the same network at the same facility took an extra minute beyond the times shown here.

As a data point, the following times were taken from several legacy Windows XP machines running similar hardware.

From Boot to Login: 41 Seconds
From Login to Desktop: 17 Seconds
Total Unusable Time: 58 Seconds

Certainly all of these times, NMCI and Legacy, are the result of dozens of variables. It appears that the NMCI machines are interacting heavily with Domain Controllers or something else on the network during boot.

Just as long as people are counting the extra five minutes of wasted employee time (several times a day for thousands of employees) when they tout the improved efficiency of NMCI.

Never Heard Of Him

nospam@example.com (Charlie) — Wed, 06 Apr 2005 18:10:29 -0700

The Navy Marine Corps Intranet appears to be simply the Navy Intranet or the Marine Corps Intranet depending on what side of the Pentagon you salute. Navy users can plow through literally thousands of E-mail addresses in the MS Outlook global address book without seeing a single Marine Corps E-mail address. Marine Corps users have no access to the Navy address list either.

NMCI has stated that it has "no plans" to merge the two databases. We believe that is perhaps a bit overstated (and certainly came from an unofficial and non-authoritative source). Perhaps they mean they have no immediate plans to provide a single address list. Perhaps they can not do it because of technical limitations in their chosen database method.

Whatever the reason, the purple color of NMCI appears to be fading a bit more every day.

Netscape to Be Updated

nospam@example.com (Charlie) — Mon, 21 Mar 2005 17:03:00 -0800

The updates are coming fast and furious. NMCI users on the west coast were notified today that their Netscape browser will be "updated to Netscape Version 4.76."

It is not clear if this was a typo or if there are other updates within Version 4.76 because most users already had Version 4.76 on their machines.

Several historical sites like this one and this one show this version as being released around late 1999 or early 2000.

Again, the concept of supplying unsupported and highly outdated software is beyond understanding.

Yowza

nospam@example.com (Charlie) — Thu, 17 Mar 2005 06:22:00 -0800

From Patuxent River, MD we have a report of a battery entering thermal run-away. If this is a fake, its a good fake.

NMCI Laptop on Fire at PAX River

The lower resolution image is here (click on thumbnail to see 640x480). Close examination of the full size image (4 Mpixel) shows no signs of Photoshop magic. If you'd like to see a larger version, be sure to check out the Moldy Oldies posting.

The owner was seen running out the door of his building trailing smoke behind him. He quickly tossed the laptop on the ground and someone snapped this picture.

Enjoy!

A Rumor From the East Coast

nospam@example.com (Charlie) — Mon, 14 Mar 2005 22:19:00 -0800

A reader from Norfolk writes to say that NMCI is considering disabling the USB ports on NMCI seats.
This is supposedly in response to users attaching "thumb" drives and external hard disk drives in an effort to increase storage space without paying exorbitant NMCI lease fees for larger hard drives.
Although this would not be surprising, one would have to ask how they are going to deal with the USB mice if they completely disable the ports. Perhaps a restricted driver subset. Or, perhaps its just a rumor. You heard it here first.

Wolf Guarding the Hen House

nospam@example.com (Charlie) — Mon, 01 Nov 2004 19:35:00 -0800

"EDS stands to reap financial rewards when NMCI customer service levels reach certain benchmarks, beginning at 85 percent. As customer satisfaction levels rise, EDS has the potential to earn as much as $100 per NMCI 'seat' for each financial quarter, according to program officials."

A follow-up article in Government Executive Magazine sheds (a little) more light on the NMCI "survey results" released by EDS Corporation earlier this year. An important take away from the piece is the fact that NMCI stands to make money -- a welcome change for them and NMCI -- if the satisfaction level reaches 85 percent. They just reached 80 percent satisfaction and are closing in on the 85 percent threshold. Wow, this NMCI thing must really be working well!

Hmmmm, let us look at the finer points of the "survey":

1. The survey was designed, constructed, distributed, collected and analyzed by EDS Corporation, the same company that stands to gain financially from a high satisfaction percentage.
2. The questions have not been released for review.
3. The sample size and response rate of the survey has not been released.
4. The detailed results (e.g. response statistics for individual questions) have not been released.
5. The Navy has expressed confidence in the survey results and EDS.

Interesting. Not sure which Navy officials GE Magazine interviewed to determine that they had confidence in the results but it was apparently someone who hasn't taken a high-school science class.

The article goes on to offer several quotes from national pollsters and statisticians such as the Gallup Organization, stating how there could be an appearance of impropriety in the fact that EDS did their own survey and now refuses to release details. Really? Wow.

Unlike the previous GE article on the survey, (see reference elsewhere on nmcistinks.com) for this article GE actually managed to track down someone who took the survey. The user reported that he was asked questions like, "Was the help desk person polite?" to which he was forced to respond honestly in the affirmative. However he said there was no opportunity for him to footnote that response with, "However, the help desk person failed to do anything about my problem." Another satisfied user!

Again, perhaps the most disturbing aspect of this entire episode is the Navy's overt support for the "survey" process and results. The official position of the Navy is that releasing the details of the study would compromise their ability to successfully conduct future surveys. How profound.

A final note. We should thank the folks at publications like Government Executive Magazine and Federal Computer Week for digging these stories out of the back rooms. I can only hope that congressional staffers and watchdog organizations are reading them as fervently as we are.

NMCI Runs Afoul Of DISA

nospam@example.com (Charlie) — Sun, 24 Oct 2004 17:16:00 -0700

In what can only be viewed as an effort to expand their empire, EDS and NMCI have been bragging about their efforts to utilize voice-over-IP technology to provide voice (telephone) service to its customers. NMCI has not yet convinced anyone that they are contractually entitled to take over traditional switched telephone service on Navy and Marine Corps facilities. VoIP offers them an avenue to carry voice services over their existing IP-based infrastructure.

Except for one problem. They might be breaking DoD regulations by doing it.

A Bit of Background
Voice-over-IP popularity exploded in 2002 or even earlier. Dozens of vendors offered hundreds of solutions from full-blown IP PBXs to desktop phones to box-to-box solutions. The salivating communications guru was greeted with nearly unlimited options to solve his voice requirements. VoIP was an elegant solution because of several reasons.

It was designed from the ground up to interface with traditional analog and digital phone switches so the engineer could create hybrid systems. It was easy to encrypt as individual circuits or in bulk. It offered a scalable solution where hundreds of users could be carried on a single CAT-5 cable as opposed to the huge bundles of copper required for traditional phone systems. It was extremely efficient in terms of bandwidth usage not only because of the packetized nature but because it did away with the 64 kbps DS0 barrier. It allowed "toll skipping" which could potentially save thousands of dollars on long distance charges.

Yet VoIP was not without its drawbacks. In an organization as technically and geographically diverse as the DoD, there are bound to be incompatibilities between VoIP systems and traditional phone systems (or even other VoIP systems). This problem was not helped by the VoIP industry that -- like nearly every other standards-based organization in the world -- failed to agree on universal protocols and interfaces. There was also the question of security. Who was making sure that these new-fangled VoIP boxes hitting the market didn't have security flaws or even malicious back door compromises in them? This was certainly a large concern for any organization but especially the US Military.

A Big Hand of Guidance
Enter DISA. The Defense Information Systems Agency is tasked with ensuring Information Assurance within the DoD and have, for better or worse, built an empire controlling information and communications systems within the Department of Defense. Their charter is beyond the scope of this posting but their web site has lots of information.

In April 2004 the Field Security Operations Division of DISA released the Voice over Internet Protocol Security Technical Implementation Guide (VoIP STIG) which gives DoD facilities guidance and best practices for implementing VoIP. Of course the focus of this document is on security and Information Assurance. This document is based on the overarching principles and authority of DoD Directive 8500.1.

The VoIP STIG does an excellent job of pointing out the vulnerabilities of VoIP (which will be left as an exercise to the reader) but more importantly identifies the urgent need for interoperability and security testing of any IP-based system. The need for this, especially in a command and control (C2) environment, should be self-evident.

Who does this interoperability and security testing? The Joint Interoperability Test Command out of Fort Huachuca, Arizona is the primary command currently allowed to "bless" these technical solutions. Simply put, if a VoIP solution has not been tested and approved by JITC, it is not supposed to be connected to a DoD phone network. And certainly not to a Defense Switched Network (DSN) backbone.

NMCI Says It Isn't Us
So who would dare violate a DISA directive? Who would subvert DoD 8500.1? Certainly not NMCI. Or would they?

Apparently NMCI and possibly others have run afoul of the DISA directive because an April 2004 advisory from the Navy's Network Warfare Command (NETWARCOM), in cooperation with the Marine Corps Network Operations and Security Command (MCNOSC), notified several commands that they were in non-compliance with DISA directives.

It states that at least 16 DoN commands have uncertified VoIP systems. This number resulted from a telephone switch inventory done as part of the National Defense Authorization Act FY2003 (Public Law 107-314). Although it does not state categorically that these non-certified systems are NMCI offspring, the advisory makes direct reference to the EDS VoIP solution and states that commands are not allowed to implement any VoIP solution without JITC certification.

It would be interesting to know how much JITC/DISA oversight was done on the VoIP solutions that NMCI is bragging about. Apparently it was not as complete as it should have been or NETWARCOM would not have felt the need to remind Navy and Marine Corps users of the certification and approval process. The rules are there for everyone.

Capt. Chris Christopher, deputy Director of future operations for NMCI has stated that making telephone calls over VoIP was "inevitable." Certainly it is. Let's just hope that NMCI is not usurping DISA in its effort to hasten the inevitable.

Silent Majority or Manipulation

nospam@example.com (Charlie) — Mon, 30 Aug 2004 20:10:00 -0700

"Several service personnel, however, said they do not know anyone who has taken the survey."

A recent article in Government Executive Magazine reports that the satisfaction level is "near 80 percent and rising" within the ranks of Navy Marine Corps Intranet users. The accuracy of these numbers will be left to the reader to ponder but there is a very important threshold not far from this 80 percent number. When the "satisfaction level" reaches 85 percent, NMCI contractor EDS Corporation is entitled to bonus payments for every NMCI seat, regardless of the satisfaction level of that seat user or the base where it is installed.

The article goes on to discuss the fact that a short and informal survey by GE staff could find no NMCI users who had actually taken the survey. More importantly NMCI officials have not released the details of the survey including number of surveys sent, number returned (sample size), the questions on the survey and a myriad of other factors that could lead to an honest, stoichiometric analysis of the vaunted survey.

Perhaps most disturbing, Navy officials have stood by quietly accepting, if not supporting, this junk science being proffered by EDS.