Team,
It’s been almost two years since I visited the USS OAK HILL (LSD 51) to experience a “day in the life of a JO” and get a true deckplate perspective on the impact the administrative programs we use on our ships have on our LPOs, LCPOs, DIV (O)s, and DHs. That visit, and subsequent discussions on other Fleet units, prompted me to direct a comprehensive review of all software applications being used in the Fleet today. Additionally, I directed CYBERFOR to take a good, hard look at how we could establish a much more robust governance model to ensure we deliver effective tools that meet the standards our Sailors need and expect from us.
This comprehensive review, the “Fleet FAM effort,” was initially led by RADM Meek and now continues under RDML Herbert’s leadership at Navy Cyber Forces. We had some initial setbacks as we tried to get our arms around this complex problem, but the Fleet FAM team has since made steady and meaningful progress toward unwinding the effects of over two decades of undisciplined software management in the Fleet. I want to share some of the updates with you that I recently received from RDML Herbert and get your feedback on what you think we can or should be doing differently.
First, her team is making good progress with identifying and fixing the applications that are currently fielded in the Fleet. They’ve already assessed 568 of the 945 applications originally identified in the Fleet FAM baseline software application list. Each application is thoroughly reviewed for a valid requirement before a final determination is made to keep, kill, fix or sunset the program. The team is on pace to finish the application review by the end of this year.
Second, to assist with the review, RDML Herbert assembled a Fleet Application Solutions Team (FAST) of IT experts to visit ships and hear firsthand from our Sailors about any problems or difficulties they’re experiencing with the applications they’re required to use. This team is probably the most critical piece in this effort because they are pulling in feedback directly from our end-users on the deckplates.
Third, RDML Herbert is holding a combined Numbered Fleet N6 and Fleet FAM conference this month to bring together the experts in the Fleet to discuss many of the big issues we’ve discovered so far and how we can most effectively employ best practices to manage the tools we have today. Now, RDML Herbert fully understands my concerns about conferences and off-sites (these events are not simply a vacation from the office) and she has assured me she will make full use of everyone’s time and drive the team to produce tangible and relevant outputs that we can implement today to ease the burden on the Fleet and help our Sailors do their jobs.
Finally, RDML Herbert recently released several messages, the first addressing program management responsibilities for shipboard application configuration management and control, and the second providing Fleet best practices for maintaining baseline configuration control. In addition, a Fleet FAM instruction is in the works now and should be released later this month. These messages are a big step in the right direction, but we need to ensure the intended audiences are in receipt of, and acting on, the guidance provided (more on this in a future post!).
I told you when we started this effort that it would take a long and steady commitment from all of us to reverse the damage caused by so many years of undisciplined software management in the Fleet and by the many entities who were able to deliver software applications to the Fleet. There is still much to do, but I’m encouraged by our team’s progress and look forward to hearing more about what they learn from our Sailors during their ship visits.
All the best, JCHjr
ADM J. C. Harvey, Jr.
16 comments:
Admiral,
The first program that should be reviewed is Automated Work Notification (AWN), the several-years-overdue replacement for OMMS-NG. Over $100 million dollars has been spent since 2007 and several hundred more have been allocated for future development and management. AWN is designed to improved readiness by automating the reporting process of the OPNAV 4790 2K for equipment discrepancies. Unfortunately, AWN does nothing to ease the burden on the Fleet and help Sailors to do their jobs. In fact, AWN increases the burden on the Fleet and Sailors. In an effort to align deficiencies against objects in the Maintenance Figure of Merit (MFOM), AWN does not align with the master ship configuration database (CDMD-OA) used on ships. Sailors and TYCOM 2K administrators must spend multiple hours hand-tailoring each 2K written in AWN that must be loaded onto the ship’s CSMP. Any ship that receives the post-INSURV MI 2K bulk load (INSURV uses AWN) and LCS 1 and LCS 2 can describe the requirements in detail. AWN has limited opportunities for expansion to portable devices and lacks common features that are taken for granted in modern software applications, such as time-saving universal search. The omission of RSupply integration until the later stages of AWN development is a perfect example of software development gone horribly wrong.
On the other hand, the Navy has created a 95% solution "in-house" called REDI that is data and platform agnostic, has universal search built in, talks to all Navy authoritative databases via Navy-managed APIs (eliminating the need to upgrade server architectures), can be configured quickly to suit the needs of its customers, and is ready for mobile device expansion. REDI is software that can and does ease the burden on the Fleet and help Sailors to do their jobs. Best of all, REDI is essentially free since ONR already paid for its development for use by LCS Reliability Engineers. Finally, REDI has already been successfully piloted during two recent RMC TSRA visits (http://www.navsea.navy.mil/CNRMC/Lists/News/Article.aspx?ID=7).
REDI is not only a tool for managing maintenance data (ability to write new 2Ks and view the existing CSMP), but also a logistics and Supply support tool. It allows the user to access tech manuals through TDKM, talk to PMS SKED, present EOSS and CSOSS documents, and pull data from any other authoritative Navy sources. It also allows the user the ability to redline documents and diagrams and submit tech manual and PMS feedback reports (TMDERs and TFBRs). Using the REDI RSupply link, users can order parts right on the deckplate.
There is nothing that AWN does that REDI cannot do since REDI 'talks to' the same servers that AWN does to feed real-time MFOM values. Properly-designed and validated use cases with Sailors can accelerate the development of a tool that would be embraced by Sailors. It can be used now on desktops, but can be expanded for use on any device with a browser capable of rendering HTML5 (e.g, smart phone, tablet). The benefits of REDI are many – Sailor efficiency, improved 2K quality, on-demand technical reference use, improved Zone Inspection management, low-cost software management and enhancement, preservation of Navy authoritative databases, and re-distribution of AWN development funds to ship material readiness accounts.
As an officer who has completed post-graduate studies in Computer Science, I am perpetually frustrated by our inability to field and manage solid software applications. REDI is software that fulfills the promise of so many failed software projects that have fallen to the inefficiencies of the Mythical Man-Month.
Admiral, I applaud your willingness to try to correct the mistakes of 20 years of poor software engineering and governance. Please consider stopping AWN development and deployment and accelerating the deployment of REDI instead.
V/r,
Concerned Sailor
Admiral,
Most of things that "Concerned Sailor" said are true (never heard of REDI though). But, he/she must be corrected - $175M+ has been spent since 2005. While some funding (few million) of this program was as a result of congressional earmarks, some 95% was directly from your ship's maintenance account. So everytime you hear about the poor material condition of a ship, you can thank your very own N43 staff for using money on developing software instead of fixing ships. What did $175M buy you? Pretty much nothing, just a bunch of software that is riddled with security vulnerabilities (hundreds, thousands of security findings). The first thing some might say, "there goes SPAWAR, developing another sub-par software application." But truth be told, Admiral, SPAWAR had nothing to do with spending this money OR developing this software. This was all driven by USFF N43 up until late last year when it was finally turned over to the SPAWAR folks for some "acquisition management." By then, the program has been heavily scrutinzed, but millions of dollars (@$30M this year) are still being spent. Rumor has it that OPNAV didn't even fund the entire cost requirement this year so USFF N43 had to reach into the ship's maintenance line account (AGAIN) to fund the shortfalls. Again, don't pin this one on SPAWAR, start with your N43 staff. They know the truth, just not sure you're going to want to hear it, or if the truth actually makes it to your desk.
v/r,
Anonymous (of course, who wants the wrath of reprisal?)
Admiral,
The beat goes on. Back when we were trying to tie all the maintenance systems into a "new and improved" Maintenance system, the charge was led by NAVSEA to 'save money'. All the CLF Tycoms who were involved in the decision-making were underwhelmed with the idea of buying SAP software and tailoring to fit a majority of the assorted 143+ systems at the time. CNAL just flat said no (good move) but CNSL was forced to take the ride despite a chorus of naysayers. Our input was list all the desired requirements of a system and bid it to software developers, like microsoft and build it from ground up. 10-12 years later and a billion dollars expended and we have a system that requires more work inputting than any of the old systems and still don't get the required outputs, we're about to revert to new systems similar to what we had in the 1990's. We generally do penny-wise, but pound foolish decisions across the board. The last place an organization like N43 in FFC (CLF) should be putting any dollars is computers and software,thats a SPAWAR function. The ship's repair funds are already being squandered on legions of Government Service assessors(...and we're still hiring more!!!) yet there is no investment in the actual fleet's folks who will actually perform those hands-on repairs. I guess we need huge new software systems to track all the assessments, pre-INSURV Grooms, Pre-INSURV walk-thru's and underway practice rides, and work processes and mounds of WAFs being generated, so when we actually get some qualified repairmen onboard they have a super-duper reference system.
Retired 0-6
In addition to the AWN debacle your N43 staff also is promoting the Validate, Screening, and Broker (VSB) as a replacement for the RMAIS peroduct that is ably serving the fleet. VSB is to have all the capability of RMAIS, but is not there 5 years in. This is antoher 100 plus million of maintenace funds squandered to put the MFOM value into software. BTW RMAIS already is capable of displaying the MFOM value. CACI apprreciates the business.
Admiral,
It's obvious the comments regarding REDI were written by someone attempting to promote REDI over AWN. I've witnessed demonstrations of both regarding their capabilities. Why doesn't the Navy stop pouring millions of dollars down a rat hole? We have an existing tool (Fleet Assessment Support Tool) that with some minor improvements is more than adequate tool for providing CSMP uploads of identified deficiencies. Wouldn't it make more sense to spend all this money (in the case of AWN over $100 million dollars) adding additional subject matter experts at our RMCs to provide assessment and technical assistance support to our fleet?
Very respectfully,
Concerned Retired Sailor/Civil Servant
I believe the BLUF is that the Sailor on the deckplate doesn't care how much money we spend but rather that we provide him/her with the tools that enable them to do thier job with greater ease...not greater frustration. The REDI initiative is the closest thing in a very LONG time that I have seen that will truly be a force enabler in that it will allow Sailors to utilize ATIS as it is meant to be...a mobile repository of publications/drawings/IETMs/TMs/POGs/SIBs etc. that they can access with ease. Not to mention the utilities it will provide for searching those publications for specific troubleshooting/reference material.
v.r.
LT Greg Rodriguez
This blog concerning the Fleet FAM initiative has sparked much interest by many with new software programs that are all out there to solve the Navy’s need or maybe thirst for using new technologies via hardware and software. It is essential that everyone support this initiative along with the IT EXCOMM because the reality of major budget cuts in the IT arena mandates we become smarter in how we spend those dollars. If we truly take on this project it will only be successful if the outcome decisions are made without bias or prejudice towards such things as home grown applications, not invented here syndrome as well as we have always done it that way in the past and finally we don’t want to change our current business process. The significant point that was missed in the previous blogs was the cry from the fleet that they were looking for someone, anyone that would listen to their immediate needs and provide them with a solution. Not a solution 2 or 3 years down the road but a solution now, like 60-90 days. When that is not done then you begin to see the home grown solution crop up build via ACCESS or the use Excel with built in macros or local contractors hunger for government business anxious to help the Sailor out. There are hundreds of these home grown applications in the fleet today. No one wants to admit that and OPNAV will tell you since they can’t trace money back to them then they must not exist. When the IT EXCOMM began its initial work back in 2009 they took a look at traceable maintenance application that could be identified and found more than 640 such applications.
It was readily apparent someone was looking to sell one application over another. This comparison was like comparing apples and oranges. One application developed for a single platform to solve a single issue, not designed to replace any existing application, and is currently around pre-milestone A. The other application was developed as a family of systems supporting all afloat and ashore instances, designed to replace more than 60 applications, currently being deployed and coming around post Milestone C. Just some of the significant issues with the first blog not to mention a number of reported inaccuracies. Both applications should be included in the fleet FAM and IT EXCOMM and the decision made concerning future utility. The second blog attempted to correct some of the errors reported in the first blog. The dollar figures reported appear to be even more exaggerated. The comments about funding sources were incomplete and possibly misrepresented. As talked about in the blog the program has been downsizing and consolidating efforts. It’s these types of efforts that will cause emotions to run high as less effective participants are left behind. OPNAV did not fully fund MFOM. This is a daily tough challenge for OPNAV. Most know there are always more requirements that there are funds to cover.
Everyone seems so concerned about dollars but at the end of the day, think about what the fleet is going to pay to conduct a full investigation to respond to blogs that were full of miss information, bias and emotion when there is a process in place called the Fleet FAM to weed out the non competitive applications. Get behind the FAM process and hope that it also addresses major programs like ERP, $1B, 7 years of effort and what maintenance solutions have been delivered to the fleet. Even NIAPS, 2 years late with the most recent release 2.4 schedule for this summer. The entire IT business from the acquisition process DoD 5000 to the smallest development effort needs a complete review and significant changes if we are to really make a difference.
Admiral,
I think the maintenance community as a whole understands the requirement and benefits of transitioning towards a single database solution. But we need to be honest with ourselves if we’re to improve maintenance processes overall, specifically the shipboard and ashore systems.
Our problems today can be attributed to a lack of training, support and accountability of our current shipboard tools and processes. Insight and guidance on the use of these tools is a key factor in the management of ships CSMP. New software doesn’t solve this problem, just like it hasn’t solved this issue using the legacy software. Holding personnel accountable for the health and management of the CSMP along with proper review and approval of data before it leaves the ship, is a first step and it doesn’t cost us a dime.
The new MFOM applications incorporate spiral down development. Our current programs meet our functional requirements, and are mature in their development, but don’t meet the future OPNAV vision. These legacy programs should have been used as a starting point in the development of the replacement software.
It seems that we could have taken the front end of our legacy software and changed the back end to meet the future reporting and database requirements. This would have required the CDA to be involved earlier into the development process, since they own the legacy software, resulting in a smoother path towards development and fleet implementation. In the case of AWN, a commercial off the shelf deficiency writer, which was a basic shell of a program, was used, and not until the TYCOMs had a chance to review the program, was the functional requirements taken into consideration.
Bashing the current legacy software to promote the new MFOM applications is misleading, deceptive and a little irritating. Those of us who have attended past MFOM presentations; recall such statements as “You have to fill out 100+ fields in OMMSNG to create a Work Candidate”, ”Turbo Tax Friendly”, “Your either with us or against us”… or that the current legacy software is just too difficult for the sailor to use. Overall those that have been trained properly on the use and capabilities of the software generally don’t have a problem.
In the past, I’ve been contacted by numerous ships, which have had questions regarding our current CNAL processes and use of OMMS-NG. Since OMMS-NG is the same across all platforms, our office has been able to help. I’m amazed by the number of sailors that are out there that don’t know the current capabilities of the software. The simple function of turning on the Maintenance Module, allowing the repair organization on a big deck AMPHIB, to screen and track organizational level work is a real eye opener in most cases.
We will continue to provide support as the TYCOM representatives. It just seems like, in 5 years we should have been a lot further than we are. Not to mention the projected cost to support these applications and new processes once implemented fleet wide.
V/r
Jeff Shultz
Admiral,
I am not an expert on software but I do know when a program has alot of issues. The program that I would like to draw attention to is NAVFIT98. I know that there is an update to it but unfortunately I am a deployed IA unable to access it and see if it is any better than the previous version.
My issues and/or suggestions are:
1. Emailing the individual files. We should be able to email files without having to convert them to a .txt and then reconvert them to a .mdb file when the recipient receives it. This is especially an issue when working on 20+ evals at home and needing to email it to work address.
2. NAVFIT 98 is just that...from 1998. It is clunky and frustrating. I am assigned to a joint command and am able to see how the AF does their evals. They do it via a sharepoint. When the person is finished, they click on a button and it is sent to the next level. If the person above has issues with it, they can send it back down for corrections. But it is all done in Sharepoint. It just seems to me to be much easier than NAVFIT98.
Thank you for your time and thank you for this Blog.
Anonymous, thank you for your comments on NAVFIT98. I'll ensure we add it to the list of applications we'll try to "fix or finish."
BT-BT
For all re the comments on AWN/REDI, I've been having my IG and Comptroller doing the forensics on these programs in order to get the facts, the history of what, when and why we've spent what we did, what the outcomes of these programs were supposed to be and who was accountable for achieving those outcomes, etc, etc.
Needless to say, putting together a coherent story that explains all the "whats and whys" has been very difficult (which tells you something right there).
Initial reports to me have left me unsatisfied so we'll keep digging until I get the whole picture on this issue. More to follow. All the best, JCHjr
Dear Anonymous, With respect to the questions regarding NAVFIT's ease of use and/or potential improvements, I can tell you that NPC will soon release version 30 that offers the following:
- Expanded application functionality to support all versions of Microsoft Access and operating systems
- Incorporates the new Lieutenant Forced Distribution Policy
- Incorporates the Chief Evaluation (E7-E9) into NAVFIT98
- Adds additional lines for comments on performance blocks (block 41/43)
Additionally, NPC understands that NAVFIT98A has other issues that may require significant cost and time to correct. Instead of putting those resources toward improving NAVFIT98A NPC has initiated a project that will create a web-based solution. The technical design is not complete so we don't know the specifics of how the solution will work but it must:
- Validate on the front end of the process to reduce rejects
- Allow electronic submission and digital signatures to reduce the time to update the OMPF
- Improve data quality
- Use a web based interface that allows electronic routing
- Combine all three FITREP systems (NAVFIT 98, E7-E9 and Flag FITREPs) into one system
A solution will be determined by PMW240, who should explore other service solutions during the design process.
Making a significant change like this will bring challenges particularly with making sure the solution works effectively in the Afloat environment. To help reduce the friction, NPC has asked USFF and CPF to be involved - and we will!
With Respect, RADM M. Guadagnini
Admiral, I am sure that your team has already thought of this but I wanted to ensure that someone is ensuring this web-based Eval/FITREP program is will reside on the NIAPS servers and be part of the replication push vice a direct www solution.
v.r.
LT Greg Rodriguez
LT Rodriguez,
I will ensure that we relay your recommendation to NPC as we interface with them in the process of getting modern, improved, evaluation system tools.
With Respect, M. Guadagnini
To: "Anonymous (of course, who wants the wrath of reprisal?)" and "Concerned Sailor"
I am currently serving aboard the USS FREEDOM as the MPA and can tell you that from the deckplates everything you say regarding AWN is EXACTLY right. Managing AWN is my full time job because my Engineers simply do NOT have the time to navigate the complicated and poorly supported APL structure to place jobs under the right APL for conversion to OMMS-NG on the shore side. I spend up to an hour for each job navigating an outdated COSAL (most recently updated in 2009), WEBFLIS, AWN, and various other programs simply to identify the proper APL under which to enter an AWN. If I use the wrong APL, the shore side must cancel the AWN and find one that will convert to OMMS-NG consuming even more man-hours.
AWN fails in the following:
1) Lack of ability to view the OMMS CSMP. S/F must be sent a text file from N43 to view the CSMP.
2) Lack of reliable communications between AWN and OMMS results in many jobs showing open in AWN, but really closed in OMMS, requiring constant email updates between N43 and the Ship to clarify what jobs are still open and what is not.
3) Lack of ability to track parts, results in parts arriving to the ship without any concurrent JCN and wasted man-hours trying to search the database for what the part was intended for.
4) Lack of proper APL/AEL database, requiring specialized training to validate AWN APL's against the COSAL and WEBFLIS to validate those APL's for use. If the wrong APL is used in AWN, the job will not replicate to OMMS, requiring further work on the part of N43 to research and re-write the job.
5) Lack of ability for S/F to close jobs or place in archive folder to signal N43 to close. All jobs current and closed reside in a single folder which makes it very difficult to find new jobs or determine what jobs should be closed.
6) Lack of transparency between approving authorities. Once a job is submitted from the ship, there is no way to track what level of authority the job has been approved by and when it is slated to be completed. I maintain a separate excel spreadsheet to ensure jobs make it from the AWN afloat to AWN ashore to OMMS to REMAIS to MST and MAXIMO.
7) Lack of ability to close jobs from the bottom up. The port engineer must close the job in REMAIS, N43 in OMMS, and Lockheed Martin in MAXIMO before a job will close. If any one is missed a job will be rescreened to the next availability, resulting in money wasted for a sub-contractor to ship check and discover the job is closed.
8) Lack of ability for division officers to screen jobs written by Sailors prior to being sent off the ship, resulting in a poor first-pass yield JCN acceptance.
9) Lack of first pass yield metrics for program.
10) Lack of ability to group jobs by dates scheduled to be completed, i.e. what is slated for upcoming Maintenance Periods.
I am gathering data for my point papers against this program and for the NS5 like system (such as REDI, with is BADLY needed). NS5 is a system that does everything REDI does and is utilized by the merchant marine to maintain incredibly efficient repair processes, material history files, and technical documentation.
Please contact me at navy.mil for further discussion. I am resigning from the Navy in the next two months, but I hope to make some impact to improve the lives of those I leave aboard FREEDOM.
Thank you for commenting and bringing this to light. It is a topic that must be addressed if we are to effectively manage maintenance utilizing the Sea Swap model.
Admiral,
You should also look into CNSP's Maintenance Support Tool (MST). This is another unnecessary computer program that is a decade
behind the times, endangers valuable maintenance information (by putting that information onto a notebook computer, then into a
180 degree vehicle trunk), presents a security risk (accumulation of ships' maintenance history being left unprotected in hotel rooms)
and was developed with the wrong color money (millions of maintenance dollars). CNSL refused to participate until they fell under
CNSF and were ordered to do so.
Admiral,
I spent over 20 years in the Navy and have over 20 years experience since retiring with Navy Maintenance Programs. I have been involved with the MFOM project since it started and it has NEVER lived up to any delivery dates nor more importantly, promised functionalty. So now it seems that in order to justify the ridiculous amount of money spent, there are plans from FFC N43 to continue to field this family of faulty programs. The plan (FFC N43) is to "allow" the TYCOMs & their users to "look over the shoulder" during testing/demo of the upcoming patch 2.0.1 by the technical developers. This is outrageous, when the personnel that will be required to try and accomplish their daily work with these tools are NOT allowed to test the functionality in order to try and meet some fictional date/reason. Additionally, there has been no satisfactory test of the ability to revert back to the current operating software, RMAIS, OMMS-NG, ETC. If this is allowed to continue, I believe the Navy maintenance community is setting its self up for failure. This will cause a tremendous burden on the personnel that will have to "Un-screw" this mess being made. We, the taxpayers and the Navy, should not be paying (ridiculous amounts as stated in other replies) for a contractor to continue to be allowed to hand us this garbage and then be expecited to carry on our work with a shrinking budget, less personnel, increasing demands for time and an inferior product, especially given we have software that WORKS today and with minimum investment could be enhanced to continue to serve the maintenance community well into the future. FIX SHIPS NOT BLOW MONEY ON FAULTY SOFTWARE!.
Post a Comment