Last month I discussed how the Tygron GPU Supercomputer can be used for a hyper-resolution flooding at maximum detail. However, in case of an acute emergency, calculation speed and the time required to setup a project is more important. In this blog I will demonstrate how the Tygron Platform can be used in such cases.
Scenario
It often starts with an early warning. For example, during a levee inspection, a severe crack in the levee is found, which can potentially lead to a breach at that location. Or perhaps there is already a breach forming. The user (e.g. a waterboard employee) now needs to do a fast impact assessment of what happens in the next hours and days and, based on several scenarios, inform the emergency services.
1 Create a Project: The Tygron Platform is cloud based and can be accessed from anywhere in the field, at home or in the office. Maybe the user already has a project pre-loaded and he/she can skip this step. Otherwise the user needs to select the impacted area (e.g. the polder). Once a selection has been made, the Tygron Platform will start an automated process in which all the latest Geo data from public sources are combined into a single 3D model of the area.
2 Setup the Breach: After the 3D model is loaded the user needs to define the breach area. This can be done by uploading a polygon of its location or by just drawing the area in the 3D map. Next, the user can add a flooding overlay and setup the parameters for the breach, either a Standard Wave (m3/sec over time) or based on the breach formula (Verheij-vdKnaap).
3 Calculation: Instead of Hyper resolution (<=1x1m) we now select a more modest 5x5m resolution. This allows for very fast results while still retaining much of the elevation map and building obstacles. The user can now run the simulation, or multiple overlays in parallel for different scenarios.
Note: in case of real emergency a user can also get priority access to all GPU clusters available in the Tygron Platform.
Bommelerwaard
Based on these use-case steps I did some tests on the Bommelerwaard on how fast you can get results on a 5x5m resolution. During project creation only the relevant data was loaded, where the elevation map was set at 2.5m.
So for the Bommelerwaard we can create a detailed project based on the latest Geo-data and do a flood impact simulation in under half an hour. Iterating on this we could now also try different scenarios (different breach location or taking preventive measures) and do a 7 min recalculation.
Comparison
First we compared the results of the 5x5m model with results from the previous 1x1m run. Here you can see that on the larger rural planes the flow pattern is almost identical. Also the time it takes for full flooding of the polder and final water depth is roughly the same.
Between buildings the 5x5m model has less flow. Whether or not this is a problem depends on the situation. When the water mainly flows through small streets this is a problem, however in the Bommelerwaard case most area is flat rural and the water can easily flow around and will eventually “surround” villages. The TU Delft study also confirmed that different grid sizes does not severely impact the amount of expected casualties in this specific case.
In case of a large metropolis getting flooded, I do expect that there will be a significant impact on e.g. the arrival time!
Next we did a preliminary comparison with a traditional CPU simulation model. For the Bommelerwaard there is already a study done by the TU Delft using D-Flow Flexible Mesh. Comparing models is always a bit tricky, so this must be viewed only as a rough estimate, with some remarks:
- The D-Flow FM model uses a 5x5m resolution near the breach and 20x20m resolution in the rest of the map (80%). However D-Flow FM also uses a flexible mesh, so I expect D-Flow FM results to be better at 5x5m resolution, however 80% of the flooding is done less accurate at 20x20m.
- Comparing the results, we see the same final flooding depth (difference is only a few centimeters). The average flow patterns around breach, underpass and central highway are also similar.
- Calculation core: both models use a finite volume schema to solve the st-venant equations.
- The D-Flow FM calculation was done on a pre-release version, so I also expect the final product to be faster.
- There is no data available about how long it takes to Create & Setup the D-Flow FM model, so we only compared the Calculation step.
- Since after 4 days the entire Bommelerwaard is flooded, we are only interested in the first 4 days. In the TU Delft study 12 days are simulated in 7.1 days (chapter 5.5 page 89), thus 7.1/3 = 2.4 days.
This is only a preliminary result and certainly needs additional testing on which I would very much like to collaborate with other researchers, so feel free to contact me.
However, I do think we can say that GPU Supercomputers are significantly faster at solving the (st-venant) equations then traditional CPU machines. Which is also the main reason why a most of the Top 10 fastest supercomputers in the world are GPU based.
Can we go faster?
1 Software: Last year we made several improvements to the underlying algorithms, data management and compiler optimizations. This resulted in ~30% speed increase.
2 Hardware: GPU power increases steadily every year and recently we have started testing the next generation Ampere GPU’s from NVIDIA. From these tests we can conclude:
- Ampere is out-of-the-box 2x faster then the previous Volta generation for Tygron. This mainly due to; faster memory, faster interconnect NVLINK, improved management and a much bigger L2 cache. Also double precision floating point (64 bit) operations have increased by a factor of 2.5.
- Ampere now also supports 64 bit Tensor Cores. Using these special cores you can do matrix multiplications in one single call. This does require however that we rewrite part of the algorithms, but if successful might also yield an additional ~2x speed increase.
- At the moment the Ampere GPU is for small scale testing only. Later this year I hope to re-run of the Bommelerwaard and we also expect to start using them in the production environment (available to all users).
