EPA SWMM - priorities
It is clear when we look at the EPA SWMM offerings.
We see the SWMM Applications manual for Users and practicing civil engineers that it has not been updated in over 5 years.
We see the SWMM User’s manual for practicing civil engineers has not been updated in over 4 years.
When we look at the SWMM Engine interfacing, GUI programming manuals, Delphi/C++ Compilers and Libraries for high level private software programmers we find they were updated again ….just last month.
Furthermore, the only dual drainage analysis example we have ever had from EPA SWMM development is flawed. The example provides the City of Fort Collins street capacity standards then proceeds to assess if the example system meets the City criteria. By neglecting to model the street inlets and bypass flows, this method of analysis would significantly under estimate the impact of the Major Storm on the surface, which would lead to undetected flooding hazards to nearby structures, and roadway flood depth exceedance affecting driver and pedestrian safety that is addressed by the City criteria.
The example goes on to say:
“..As this example shows, dual drainage system models require a significant amount of additional effort to set up. …”
That was over 5 years ago. That was a very simple system… We still do not have built in tools to help the practicing engineer efficiently set up the dual drainage analysis model. The example goes on to effectively mislead and confuse the new user:
“..The details of the gutter inlet and drop structures that make the actual connection with the manholes are not important for our purpose…”
The practicing engineer is struggling due to a conspicuous lack of tools in EPA SWMM. Compare the built-in modeling tools in other publicly funded software: HEC-RAS, HY8, the GIS capabilities of HEC-HMS…The EPA SWMM cupboard is barren in contrast.
The practicing US engineer, who funds EPA SWMM with their taxes, needs tools immediately in SWMM to make themselves and their companies effective, viable and profitable to be sustainable in the economy.
Practicing engineers continue to struggle setting up dual drainage analysis models primarily because they have no street inlets, dedicated roadway/gutter specific conduit objects, nor a built-in table editor to effectively manage the number of objects involved.
Practicing engineers are inefficient in the setup of multiple detention pond systems modeling backwater effects. Engineers need tools such as a pond outlet object that has at least 3 orifices and 3 weirs nested into it that they can then simply copy from pond to pond and adjust. At least one of the orifice structures should be a multiple orifice plate where the user can enter the number of rows, columns, spacing and diameter.
Engineers need the Application Manual updated with more examples for practicing engineers. Engineers need the User’s Manual updated and expanded with a complete list of the equations used and theory used. Civil engineers do not need to QC the intimacies of the loop structures and C++ Library calls in the SWMM source code as some would like to argue, that is what we pay EPA SWMM development for. Practicing engineers can monitor their results and report any issues or defects in the output in relation to the equations and theory used, to be addressed and fixed by EPA development.
When we look at the lack of tools in EPA SWMM provided for the practicing US engineer, and contrast that with excerpts taken from the manuals created at taxpayer expense for the private high level software programmers below, we see it is clear EPA SWMM has seriously lost sight of its mission, focus, and customer obligation, which is to support the practicing taxpaying US engineer.
It is time to request a moratorium on allocating further US taxpayer resources towards the development and maintenance of the high level interfacing programming for private software developers, until there is seen a significant increase in the development of built-in tools for the practicing engineer as mentioned above. Then further discussion should occur if allocation of our tax resources to support this practice should allowed to continue.
If practicing engineers continue to be significantly inefficient in setting up these typical engineering models that are required daily in the industry, then those companies, and the civil engineering industry suffer, and that is not good for the tax base that ultimately supports EPA SWMM.
Engine and GUI support excerpts for the high-level software programmer:
“..This guide describes what a programmer needs to know to successfully link SWMM 5's engine with other applications..”
“..Before the Epaswmm5 project can be loaded into Delphi's Integrated Development Environment (IDE), several special components must be installed into the IDE's component pallette….”
“… The source code for these components is contained in the COMPONENTS.ZIP file in this
archive. Consult the Delphi Users Guide for instructions on how to install these components..”
“..some useful interfacing functions for C/C++… swmm5.iface.h Basic and Delphi Pascal…”
“..The interface was written using Embarcadero's Delphi XE2 (http://www.embarcadero.com)...”
“..This archive contains the source code for several 32-bit Delphi custom components. The components and the files used to install them are listed below…”
I'm a Canadian and none of my tax dollars go towards the EPA so perhaps I should keep my yap shut. However, I'm a heavy user of all things open source including the free US government engineering software packages so feel that an alternate opinion is warranted.
Having used many of the engineering packages, I greatly prefer the open source software, open text file input and output (and exposed format binary output) and programmers toolkit approach of the EPA's software. This provided the greatest amount of flexibility to users of all skill levels. Even moderately skilled users can figure out methods of streamlining input or summarizing output using spreadsheets. More skilled and computer savvy users can make greater use of automation provided by the engineering toolkits to simplify their workload, or to use this saved time to deliver a higher level of analysis or optimization for the same cost. The most skilled will analyze the techniques and algorithms, and extend their use or substitute for their own purpose.
I've also used both the EPANET and SWMM 5 engineering toolkits extensively since their inception. They have allowed me to produce models quicker, provide a higher level of system optimization, and ultimately to deliver more to my clients and the public. More personally, I've found that developing my own add-on tools had provided a great deal of satisfaction, allowed me to tailor the software to the ways I prefer to work, expand the 'what if' portion of my design and analysis, and to keep things interesting.
Some may argue that the EPA's approach is idealistic or academic. However, the actual effort to expose the source, input formats, etc is fairly low once the software has been written. Really only the work of a few (very knowledgeable and skilled) people in an organization of thousands as part of the largest national government on the planet.
The closed source but open input file of HEC offerings provide some opportunity to automate at a low level.
The closed source and obfuscated binary input file format provided by the FHWA's "Traffic Noise Model" makes the model tedious to use for even the simplest analysis, and offers no means of streamline analysis for the experienced user. I cannot justify charging more than a junior tech when using this software because I really cannot get any quicker or more efficient using it.
As for the rapidity of updates, a software code change and recompilation is always much quicker to do than rewriting or updating a manual or report and in any event must be completed before the manual can be updated.
My suggestion is to embrace these tools to better yourself and your work, rather than wishing for someone to develop exactly the software black box you need today.
Very well said, Kirby
Not only the fact that it's open source but that it allows use of ascii text files has kept SWMM at the top of my usability list . Lew, and now his protégés have made a point of NOT "holding their cards close to the vest".
As for manuals, you're right, its way more time-consuming to write a manual than update software. More years ago than I care to admit I wrote a multibasin bi-directional flow program that, for one major application, needed input from a crew of assistants. I found it a lot easier to write a front-end file builder that had them answer questions to populate data files than write detailed instructions.
But that was taking the easy way out like the "help" features provided with most proprietary software instead of good manuals. But even though they're still working on new versions, the user's manual and applications manual are still excellent and don't leave very much out.
I'm not sure what the definition of a "high level" private software programmer is. However it's my opinion that the SWMM5 developer's toolkit is useful to both practicing engineers and graduate students in many critical ways:
All of these tasks (and others) require an API and support for such endeavours should be encouraged by the EPA. SWMM5 is applied daily to an incredibly wide variety of water management problems and, as such, a closed version of it would never be able to satisfy all users, no matter how much development money is thrown at it. Providing an open platform has been shown time and again to foster innovation in unexpected ways, boost the economy, and accelerate the standards of practice in any discipline in which it has been adopted. In this case, it's also encouraged universities to adopt SWMM in their curriculums and graduate-level research.
The API is also effectively a crowd sourcing tool. There are many examples of SWMM community customizations influencing the official SWMM release. The new ground water custom equations is one such example. This mechanism serves as an excellent and cost-effective grass-roots way of keeping SWMM relevant as the needs of the modeling community evolve. I think you could argue that SWMM's openness has largely contributed to its popularity and longevity.
The age of the SWMM5 User's Manual and Applications Guide is not out of line with the other closed software packages you mention. The HEC-RAS documentation (including the User's Guide and Applications Manual) is also 4-5 years old, and HEC-HMS's Applications Guide hasn't been updated in over 12 years. Heck, RAS itself hasn't officially been updated in almost 5 years (couldn't resist that pun!). On the other hand, SWMM5 development libraries are obviously re-posted each time a new SWMM engine is released, and require minimal effort to do so. I don't believe the developer's toolkit documentation has changed much at all since it was first released.
Given your stance, I'm curious as to why you approve of the "GIS capabilities" of HEC-HMS, which I believe were written exclusively for, and require, a very expensive commercial software package (ArcGIS), and were funded by the US taxpayer.
While I may disagree with your strategy for securing additional development funds, I do think you make some good suggestions for improving SWMM5 and do hope the EPA continues to provide excellent support for its development.
Thanks, you make my point explicitly how off course EPA SWMM is being pulled.
"..The EU Water Framework Directive calls for... We (who's we?) should not be moving SWMM in the opposite direction..."
The taxpaying US civil engineer, who is among the sole funding support for EPA SWMM along with their fellow US citizenry, is still struggling after all these years to find ways to cost effectively get basic design data such as basin, roadway and pond structure information from their designs into EPA SWMM, and you want to assert that EPA SWMM should be more concerned about what the EU wants to do now...
Again to be clear, the EU and Canada pay nothing into EPA SWMM development and maintenance.
This highlights the conflict of interest surrounding EPA SWMM, and the influential oligarchy that is around SWMM, that has established a firm pattern that continually neglects the daily needs and welfare of practicing US civil engineers and their engineering companies as they try to cost effectively implement EPA SWMM.
The conflicts of interest go further where these certain few have private software they want to sell at a premium to the US civil engineer, that runs off the EPA SWMM UI/GUI, that many engineers and their departments do not have near the budget to purchase. They seem to fully expect the US taxpayer to support and fund their software development and maintenance operations.
It's very clear that the development of any straight forward EPA SWMM built-in tools for the practicing civil engineer, has been purposefully suppressed by this influential conflict of interest. This has been a detriment to the US civil engineer.
The EPA SWMM development team can no longer continue to freely support Engine and GUI programming for private software developers, and stay on its mission of supporting the US civil engineering infrastructure . It has proven to have a negative effect on the practicing civil engineer here. The lack of viable model setup tools severely inhibits the use of EPA SWMM in the US, which in turn is directly prohibiting the EPA from attaining their mission goals. EPA SWMM is fully mandated in our municipalities to be used for approval of projects.
We need EPA SWMM development to focus on how to be more efficient and cost effective to use for everyday civil engineering in the US.
Again, let's look at the EPA Mission Statement, and the #1 priority statement of the EPA:
MISSION OF THE EPA:
The mission of EPA is to protect human health and the environment. EPA's purpose is to ensure that: all Americans are protected from significant risks to human health and the environment where they live, learn and work;
THE EPA's #1 Priority as stated in their doctrine:
Making a Visible Difference in Communities across the Country: EPA must work each and every day - hand-in-hand with other federal agencies, states, tribes and local communities - to improve the health of American families and protect the environment one community at a time, all across the country.
We must expand the work we do to enhance the livability and economic vitality of neighborhoods in and around brownfields sites; strengthen our relationship with America's agricultural community; support green infrastructure to manage urban waters; reduce air pollution along roadways, railways and at ports; and take into consideration the impacts of our decisions on environmental justice communities through increased analysis, better science, and enhanced community engagement to ensure the protection of basic fundamental rights.
*If the Canadian and European engineers require things that are not in conformance with the EPA's Mission, but in conformance with the EUWI, then it makes sense they should pursue the EUWI to fund and procure those needs. Needs such as their continued software support.
[NOTE: Not sure if this is relevant, but for disclosure purposes I am Australian, and live/work in Australia. My father is a US permanent resident, and (currently) a practicing/tax-paying engineer in the United States.] While charging for EPA SWMM might seem like a good idea in principle, there are numerous issues that making it (partially-)paid that I feel should be considered.
1 - Charging for EPA SWMM means it would have to be closed source. If EPA is charging for SWMM, it would not make commercial sense to make the source code downloadable to be compiled by anyone, anywhere. So by enacting a payment scheme would mean either the model by which people pay would not work (as they could still essentially get the program for free), or, we lose open access to peer-review the algorithms and calculations that EPA SWMM use. Accreditation for companies that utilize or are designed to match EPA SWMM results would be meaningless, as without the source code they are unable to 'match' whatever changes or new features EPA might release. Many research engineers who use EPA SWMM as a base analysis tool, or who edit it for specific purposes, would also be severely disadvantaged by this 'closed source' model.
2 - If companies had to pay for EPA SWMM/EPA SWMM compatibility, they would almost certainly pass this cost onto consumers. Now, while I cannot speak on behalf of any company (nor even the one I work for), my experience in the past has been that if a cost is passed onto a corporation, that cost is almost always passed on to customers - plus a mark-up. I could easily envision companies, including ones based in America, passing on any costs of acquiring EPA SWMM to all users, regardless if the company had to pay for the software or not.
3 - Depending which direction EPA SWMM is developed into, it risks stepping on other departments toes. For example, suppose EPA SWMM decided to replicate many features of HEC-RAS. Should this become known to the powers that be, they could very easily cut funding (or worse, the entire project altogether), or merge the projects in order to reduce 'government waste'.
4 - Using EPA SWMM as a revenue raiser would almost certainly reduce funding allocated to the project. Governments have a habit of taking in funds from a particular source, and then depositing it in general revenue, rather than earmarking it for that particular purpose. I could again envision a reality where (and given my father has worked extensively for many Australian and US government departments, he has seen this happen numerous times) governments cut funding from a project, claiming they are 'self-funded', but then later on take the revenue from the project for other areas of government. This leaves the original project underfunded.
5 - Increased, possibly almost exclusive, focus on international standards. If international users have to pay for EPA SWMM, there would then create a sense of expectation that they get first priority to new features, or increasing numbers of features that cater to their local standards. At current, while EPA SWMM does have many features for international users, it is primarily designed for North America. If international users have to pay to use the software, the focus of development would most likely shift to ensuring that international users get value for money, rather than American engineers. While this makes good business sense, and would give the American Taxpayer best value for money, but it might not be the most helpful/useful direction for American engineers.
Now, while I have no idea how likely any of these scenarios are, any one of them would be sufficient reason in my mind not to risk the future of EPA SWMM with a 'user funded' model. EPA SWMM is a gift to the engineering community on behalf of the American people, and that it is used worldwide is a testament to both how well it is designed, and practicality. Many engineers the world over use EPA SWMM as a benchmark upon which to base their own engineering models or software, and to impose a fee for those outside of the USA could severely jeopardize the project for everyone.
Apologies for using the word "we". To clarify, my second point was that it would be a shame to move SWMM away from facilitating the integration of other computer models and processes. IMHO, it's an important capability for US practicing engineers solving US concerns.
Further to you concerns, in my 20 years on this list I've never heard any discussion of supporting any other country's needs. Also, I don't know about the EU, but while Canada may not pay the EPA, it certainly contributes to SWMM maintenance and support.
You are making some fairly far out claims that I assume are offensive to past and present EPA staff, some of whom participate in this list. Would it be possible to switch gears and entertain a more technical and constructive discussion on SWMM's current shortcomings?
I very much doubt that any EPA or federal past/present employee can comment on these topics. I don't in any way to presume to suggest what they might say, but I will perhaps unadvisedly join the discussion. Statistics on downloading of the EPA program by US vs. other users don’t really matter regarding what I want to say.
I basically agree with Rob James and Jonathan Klaric and some others in the gist of their counter to some of Fred's issues. But more to the point, I think the value of Fred's posts is in dealing with technical improvements for SWMM, not the mechanism of its support and dissemination. I too would like to see improved hydraulic/hydrologic capabilities, and I expect these to be provided, gradually, as time and resources permit. This has been the glory of SWMM's history and the proactive response of EPA personnel to the needs of the profession -- at least as expressed on this invaluable list-serve provided by our Canadian neighbors. The programming and computer code aspects are important, as described by others, for those who need to automate batch runs and adapt the model to their own particular needs. I'm grateful that the tools for that purpose are provided (through open code and ample documentation). I am pretty sure that better documentation of engineering procedures/algorithms will follow in the not too distant future, as Fred desires.
Regarding some useful ancillary features, such as more direct linkages to GIS features, more convenient input options, etc., it may interest readers to know that some obviously useful GUI linkages, such as convenient incorporation/access to Arc and other GIS images/maps/spatial data, "model setup tools," etc., have not been included because of high-level policy decisions: to not foreclose on options for American private software developers and to not require purchase, by the SWMM user, of expensive GIS software in order to use these options in SWMM. This is to say that several "built in tools for the practicing civil engineer" (notwithstanding the marvelous GUI we now have) are not included in SWMM deliberately because of protection for private enterprise in the U.S., not because of efforts to serve extraterritorial interests. This explanation is perhaps apocryphal, but I believe it is true. Regrettably for many, this means that practicing engineers may choose to spend money for third-party enhancements probably (I don't code anymore!) easily programmed, because of policy at some EPA echelon to not compete with private enterprise beyond some level in the U.S. Let me be clear: this has nothing to do with the EU or Canada.
I simply cannot believe that SWMM capabilities or dissemination policies are in any way hindered by free distribution world-wide. I could list a multitude of advances in stormwater, combined sewer, and nonpoint source pollutant management in the United States that we owe directly to learning from our European, Canadian and Asian colleagues. I think funding/pricing mechanisms for SWMM are a red herring: instead, listen to some of Fred's ideas for technical improvements, as I hope the silent EPA readers of these I don't care where a useful idea originated, nor do I count pennies (figuratively) spent supporting SWMM acquisition in other countries. A better SWMM model serves all of us and satisfies the mission of the USEPA.
I ask this directly, why are there no Dual Drainage tools in the tax funded EPA SWMM by now for the practicing US engineer?
EPA SWMM 5 has been mature for quite a while now.
EPA SWMM is an urban drainage modeling software application. The street system is the prime focal point in urban stormwater modeling. This is where runoff and pollutants are collected and concentrated before discharge into our receiving waters.
We need help in accurately setting up and modeling the volumetric distribution between the two storm conveyance systems in order for our models to have any true relevance.
It is conspicuous at this time that there are no inlet and street modeling tools available in the software.
There are dual drainage tools in EPA SWMM 5, they are called:
It is not true that there are NO inlet and street modeling tools i SWMM 5 - there are many many tools. They are not called by their HEC-22 names and need to adjusted to fit the words of HEC-22. This is true of many features of SWMM 5 - we do not have the Pond Risers of a model like ICPR but have to make the pond outlets out of storage/weirs and orifices in SWMM5.
The state of affairs you mention using the primitive SWMM objects to model our street flows is not acceptable at this point in the development cycle of EPA SWMM. You well know how difficult it is for modelers to set up a dual drainage model in EPA SWMM in order to model flow reversals between the minor and major systems.
The combination of outlets and orifices required for each inlet location to simply model flow reversals is overwhelming to implement and manage across a system. Especially without a built-in table style editor.
Developing Outlet inlet rating curves for all the inlet location permutations of : gutter slope, street cross slope, approach flow, approach flow depth, trajectory flow dynamics, for each inlet size is prohibitive. We need to leverage the FHWA HEC 22 empirical research and equations in order to make inlet modeling viable to the practicing engineer. The difficulties the engineer is currently faced with by the lack of these Minor/Major system tools directly contributes to inaccurate models being produced that ultimately affect our communities, and the mission of the EPA.
A street curb and gutter/ inlet system is our CORE common everyday stormwater modeling need, and EPA SWMM currently does not have the required tools to effectively implement this mission.
The US government has already spent a great deal of time and money through the FHWA researching the street inlet, manhole junction losses and manhole drop losses as found in HEC 22.
We need to properly model inlets and manhole junction losses efficiently to meet our civil engineering industry standard of care requirements here in the US.
Is there a technical software reason that efficient street section conduit objects and HEC 22 inlet methodology have not been implemented in the mature EPA SWMM software as of today?
While I concur that SWMM can do just about anything, as Bob notes, I do share Fred's exasperation of how difficult it can be to design assemblies of various nodes required. For instance, to properly model the hydraulic/hydrologic responses of say, a 2-stage bioretention cell with bypass flows and inlet capture, you can end up with over 30 nodes if you properly disaggregate the individual losses through the system.
This is why I use a proprietary DS model to design the various elements. I am continually surprised at how these elements interact under pressure flow to generate responses you wouldn’t anticipate. Plus I am constantly looking at increasing media HRT to optimize treatment while reducing bypass volumes. Once I know that all elements have the required capacity and the control outlets are not affected by insufficient capacity somewhere upstream, I then port all of the various orifices, outlets, pipes and media components into SWMM. While this effort may be tedious in the DS model, this would be nigh impossible in SWMM.
I think this situation outlines a great opportunity for the third-party developers of proprietary SWMM models. I could see them putting together assemblies that can be easily input, copied, raised or lowered as needed, and linked to the subcatchments and conveyance system wherever desired. As part of this effort, if somebody could get SWMM to simulate inertial bypass flow in a curb inlet, that would be great. Inlets are sometimes depressed, grates can be vaned etc. If I use a simple orifice from a trapezoidal channel, that undoubtedly overstates captured flow on sloping streets. That sort of relationship has to come from HEC-22 tables, no? I think this is where Fred has issues. So a node to address that would also be a good idea IMO. SWMM itself wouldn't need to add that if a third party developer can provide this.
My 0.$02. I think it's been a good discussion on the technical issues. Bravo to Wayne, Bob, Lew, and everyone else for making this all possible..
Thanks, this a good example how straightforward street/inlet/LID systems are required to morph into complex representations in SWMM using the primitive SWMM objects in order to have any hope of being accurately modeled in EPA SWMM 5. This quickly becomes prohibitive for engineers to set up cost effectively in engineering practice.
I strongly disagree with your assessment however that this provides an opportunity for the third-party software developer to create tools for EPA SWMM. There is no incentive for them to do so and this a false hope. Quite the contrary, the third-party developers have found it more profitable to create full SWMM-like platforms that run off the taxpayer funded and maintained SWMM engine and high level Delphi GUI.
The vision that Lew Rossman had for third-party developers to supply cost effective tools for EPA SWMM users has unfortunately failed. It has been over 7 years since the Add-In feature was implemented and if you do a Google search for SWMM Add-Ins all you will essentially find, after all these years, are the instructions from Lew how to add the Excel executable.
EPA SWMM itself as a taxpayer funded entity needs to take on the development of the tools mentioned previously... The resources and distraction it is for EPA SWMM to support and maintain the engine and GUI for third-party programmers should be stopped due to the failure of the Add-In platform and the glaring lack of modeling tools for civil engineers. The EPA SWMM development team now needs to focus its time and resources on making EPA SWMM more efficiently usable for practicing civil engineers.
Wayne Huber mentioned he could see a reason for EPA development to continue supporting and funding open source engine and GUI due to the need of a few for batch processing their models. I disagree here, a built-in Batch Processing routine should be implemented by EPA development and located under the File drop down menu, as is the typical case with other software that has batch processing needs. This would help to focus our EPA development resources on the EPA SWMM 5 interface.