Глубокие нейронные сети в гидрологии: новое поколение универсальных и эффективных моделей
DOI:
https://doi.org/10.21638/spbu07.2021.101Аннотация
В течение последнего десятилетия глубокое обучение - область машинного обучения, относящаяся к искусственным нейронным сетям, состоящим из множества вычислительных слоев, - изменяет ландшафт развития статистических моделей во многих областях исследований, таких как классификация изображений, машинный перевод, распознавание речи. Географические науки, а также входящая в их состав область исследования гидрологии суши, не стоят в стороне от этого движения. В последнее время применение современных технологий и методов глубокого обучения активно набирает популярность для решения широкого спектра гидрологических задач: моделирования и прогнозирования речного стока, районирования модельных параметров, оценки располагаемых водных ресурсов, идентификации факторов, влияющих на современные изменения водного режима. Такой рост популярности глубоких нейронных сетей продиктован прежде всего их высокой универсальностью и эффективностью. Представленные качества в совокупности с быстрорастущим количеством накопленной информации о состоянии окружающей среды, а также ростом доступности вычислительных средств и ресурсов, позволяют говорить о глубоких нейронных сетях как о новом поколении математических моделей, призванных если не заменить существующие решения, то значительно обогатить область моделирования геофизических процессов. В данной работе представлен краткий обзор текущего состояния области разработки и применения глубоких нейронных сетей в гидрологии. Также в работе предложен качественный долгосрочный прогноз развития технологии глубокого обучения для решения задач гидрологического моделирования на основе использования «кривой ажиотажа Гартнера», в общих чертах описывающей жизненный цикл современных технологий.
Ключевые слова:
глубокие нейронные сети, глубокое обучение, машинное обучение, гидрология, моделирование
Скачивания
Библиографические ссылки
Addor, N., Newman, A. J., Mizukami, N. and Clark, M. P. (2017). The CAMELS data set: catchment attributes and meteorology for large-sample studies. Hydrology and Earth System Sciences (HESS), 21 (10), 5293-5313.
Adiwardana, D., Luong, M. T., So, D. R., Hall, J., Fiedel, N., Thoppilan, R., Yang, Z., Kulshreshtha, A., Nemade, G., Lu, Y. and Le, Q. V. (2020). Towards a Human-like Open-Domain Chatbot. ArXiv, [online] Available at: https://arxiv.org/abs/2001.09977 [Accessed Mar. 04, 2021].
Agrawal, S., Barrington, L., Bromberg, C., Burge, J., Gazen, C. and Hickey, J. (2019). Machine Learning for Precipitation Nowcasting from Radar Images. ArXiv, [online] Available at: https://arxiv.org/ abs/1912.12132 [Accessed Mar. 04, 2021].
Arazo, E., Ortego, D., Albert, P., O’Connor, N. E. and McGuinness, K. (2019). Pseudo-labeling and confirmation bias in deep semi-supervised learning. ArXiv, [online] Available at: https://arxiv.org/ abs/1908.02983 [Accessed Mar. 04, 2021].
Arsenault, R., Bazile, R., Ouellet Dallaire, C. and Brissette, F. (2016). CANOPEX: A Canadian hydrometeorological watershed database. Hydrological Processes, 30 (15), 2734-2736.
Avanzi, F., Johnson, R. C., Oroza, C. A., Hirashima, H., Maurer, T. and Yamaguchi, S. (2020). Insights into preferential-flow snowpack runoff using Random Forest. Water Resources Research, 55 (12), 10727- 10746.
Ayzel, G. (2019). Does Deep Learning Advance Hourly Runoff Predictions? In: Proceedings of the V International Conference on Information Technologies and High-Performance Computing. [online] Khabarovsk: CEUR-WS, 62-70. Available at: http://ceur-ws.org/Vol-2426/paper9.pdf [Accessed Mar. 04, 2021].
Ayzel, G., Heistermann, M., Sorokin, A., Nikitin, O. and Lukyanova, O. (2019). All convolutional neural networks for radar-based precipitation nowcasting. Procedia Computer Science, 150, 186-192.
Ayzel, G., Kurochkina, L., Kazakov, E. and Zhuravlev, S. (2020). Runoff prediction in ungauged basins: benchmarking the efficiency of deep learning. In: IV Vinogradov Conference “Hydrology: from Learning to Worldview” in Memory of Outstanding Russian Hydrologist Yury Vinogradov. [online] Saint Peters- burg: E3S Web Conf., 1-6. Available at: https://doi.org/10.1051/e3sconf/202016301001 [Accessed Mar. 04, 2021].
Bauer, P., Thorpe, A. and Brunet, G. (2015). The quiet revolution of numerical weather prediction. Nature, 525 (7567), 47-55.
Bremer, L. L., Hamel, P., Ponette-González, A. G., Pompeu, P. V., Saad, S. I. and Brauman, K. A. (2019). Who are we measuring and modeling for? Supporting multi-level decision-making in watershed management. Water Resources Research, 56 (1), e2019WR026011.
Deng, J., Dong, W., Socher, R., Li, L. J., Li, K. and Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on computer vision and pattern recognition (CVPR2009). [online] Miami: IEEE, 248-255. Available at: https://ieeexplore.ieee.org/document/5206848 [Accessed Mar. 04, 2021].
Fuhrer, O., Chadha, T., Hoefler, T., Kwasniewski, G., Lapillonne, X., Leutwyler, D., Lüthi, D., Osuna, C., Schär, C., Schulthess, T. C. and Vogt, H. (2018). Near-global climate simulation at 1 km resolution: establishing a performance baseline on 4888 GPUs with COSMO 5.0. Geoscientific Model Development, 11 (4), 1665-1681.
Gauch, M., Mai, J. and Lin, J. (2019). The Proper Care and Feeding of CAMELS: How Limited Training Data Affects Streamflow Prediction. ArXiv, [online] Available at: https://arxiv.org/abs/1911.07249 [Accessed Mar. 04, 2021].
Gunning, D., Stefik, M., Choi, J., Miller, T., Stumpf, S. and Yang, G. Z. (2019). XAI - Explainable artificial intelligence. Science Robotics, 4 (37), eaay7120.
Ho, T. K. (1995). Random decision forests. In: Proceedings of 3rd international conference on document analysis and recognition. [online] Montreal: IEEE, 1, 278-282 [Accessed Mar. 04, 2021].
Hornik, K., Stinchcombe, M. and White, H. (1989). Multilayer feedforward networks are universal approximators. Neural networks, 2 (5), 359-366.
Hinton, G. E., Osindero, S. and Teh, Y. W. (2006). A fast learning algorithm for deep belief nets. Neural computation, 18 (7), 1527-1554.
Hutter, F., Kotthoff, L. and Vanschoren, J. (2019). Automated Machine Learning. New York, USA: Springer.
Konapala, G. and Mishra, A. (2020). Quantifying climate and catchment control on hydrological drought in continental United States. Water Resources Research, 56 (1), e2018WR024620.
Kratzert, F., Klotz, D., Brenner, C., Schulz, K. and Herrnegger, M. (2018). Rainfall-runoff modelling using long short-term memory (LSTM) networks. Hydrology and Earth System Sciences, 22 (11), 6005-6022.
Kratzert, F., Herrnegger, M., Klotz, D., Hochreiter, S. and Klambauer, G. (2019a). NeuralHydrology- Interpreting LSTMs in Hydrology. In: W. Samek, G. Montavon, A. Vedaldi, L. K. Hansen, K.-R. Müller, ed., Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Bern: Springer, 347-362.
Kratzert, F., Klotz, D., Herrnegger, M., Sampson, A. K., Hochreiter, S. and Nearing, G. S. (2019b). Toward Improved Predictions in Ungauged Basins: Exploiting the Power of Machine Learning. Water Resources Research, 55 (12), 11344-11354.
Krizhevsky, A., Sutskever, I. and Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems 25 (NIPS 2012). [online] Lake Tahoe: NIPS, 1097-1105. Available at: https://papers.nips.cc/paper/4824-imagenet-classification-with-deep- convolutional-neural-networks.pdf [Accessed Mar. 04, 2021].
Laloy, E., Hérault, R., Jacques, D. and Linde, N. (2018). Training-image based geostatistical inversion using a spatial generative adversarial neural network. Water Resources Research, 54 (1), 381-406.
LeCun, Y., Bengio, Y. and Hinton, G. (2015). Deep learning. Nature, 521 (7553), 436-444.
McCulloch, W. S. and Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics, 5 (4), 115-133.
McGovern, A., Lagerquist, R., John Gagne, D., Jergensen, G. E., Elmore, K. L., Homeyer, C. R. and Smith, T. (2019). Making the black box more transparent: Understanding the physical implications of machine learning. Bulletin of the American Meteorological Society, 100 (11), 2175-2199.
Minsky, M. and Papert, S. A. (1969). Perceptrons: An introduction to computational geometry. Boston: MIT Press.
Moraux, A., Dewitte, S., Cornelis, B. and Munteanu, A. (2019). Deep Learning for Precipitation Estimation from Satellite and Rain Gauges Measurements. Remote Sensing, 11 (21), 2463.
Olah, C., Mordvintsev, A. and Schubert, L. (2017). Feature visualization. Distill, 2 (11), e7.
O’Leary, D. E. (2008). Gartner’s hype cycle and information system research issues. International Journal of Accounting Information Systems, 9 (4), 240-252.
Pan, B., Hsu, K., AghaKouchak, A. and Sorooshian, S. (2019). Improving precipitation estimation using convolutional neural network. Water Resources Research, 55 (3), 2301-2321.
Racah, E., Beckham, C., Maharaj, T., Kahou, S. E., Prabhat, M. and Pal, C. (2017). Extreme Weather: A large-scale climate dataset for semi-supervised detection, localization, and understanding of extreme weather events. In: Advances in Neural Information Processing Systems 31 (NIPS 2017). [online] Long Beach: NIPS, 3402-3413. Available at: https://papers.nips.cc/paper/6932-extremeweather-a-large- scale-climate-dataset-for-semi-supervised-detection-localization-and-understanding-of-extreme- weather-events.pdf [Accessed Mar. 04, 2021].
Ranzato, M. A., Poultney, C., Chopra, S. and LeCun, Ya. (2007). Efficient learning of sparse representations with an energy-based model. In: Advances in neural information processing systems 21 (NIPS 2007). [online] Vancouver: NIPS, 1137-1144. Available at: https://papers.nips.cc/paper/3112-efficient- learning-of-sparse-representations-with-an-energy-based-model.pdf [Accessed Mar. 04, 2021].
Rao, Q. and Frtunikj, J. (2018). Deep learning for self-driving cars: chances and challenges. In: Proceedings of the 1st International Workshop on Software Engineering for AI in Autonomous Systems. [online] Gothenburg: IEEE, 35-38. Available at: https://ieeexplore.ieee.org/document/8452728 [Accessed Mar. 04, 2021].
Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Denzler, J. and Carvalhais, N. (2019). Deep learning and process understanding for data-driven Earth system science. Nature, 566 (7743), 195-204.
Rosenblatt, F. (1958). The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review, 65 (6), 386.
Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1 (5), 206-215.
Rumelhart, D. E., Hinton, G. E. and Williams, R. J. (1985). Learning internal representations by error propagation. [report] California Univ San Diego La Jolla, Inst for Cognitive Science, San Diego.
Rumelhart, D. E., Hinton, G. E. and Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323 (6088), 533-536.
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M. and Berg, A. C. (2015). Imagenet large scale visual recognition challenge. International journal of computer vision, 115 (3), 211-252.
Samek, W., Wiegand, T. and Müller, K. R. (2017). Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. [online] Available at: https://arxiv.org/abs/1708.08296 [Accessed Mar. 04, 2021].
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D. and Batra, D. (2017). Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision. [online] Venice: IEEE, 618-626. Available at: https://ieeexplore.ieee.org/document/8237336 [Accessed Mar. 04, 2021].
Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural networks, 61, 85-117.
Shen, C. (2018). A transdisciplinary review of deep learning research and its relevance for water resources scientists. Water Resources Research, 54 (11), 8558-8593.
Shen, C., Laloy, E., Elshorbagy, A., Albert, A., Bales, J., Chang, F. J., Ganguly, S., Hsu, K. L., Kifer, D., Fang, Z. and Fang, K. (2018). HESS Opinions: Incubating deep-learning-powered hydrologic science advances as a community. Hydrology and Earth System Sciences, 22 (11), 5639-5656.
Shi, X., Gao, Z., Lausen, L., Wang, H., Yeung, D. Y., Wong, W. K. and Woo, W. C. (2017). Deep learning for precipitation nowcasting: A benchmark and a new model. In: Advances in neural information processing systems 30 (NIPS 2017). [online] Long Beach: NIPS, 5617-5627. Available at: https://papers.nips.cc/paper/7145-deep-learning-for-precipitation-nowcasting-a-benchmark-and-a-new-model [Accessed Mar. 04, 2021].
Simonyan, K., Vedaldi, A. and Zisserman, A. (2014). Deep inside convolutional networks: Visualising image classification models and saliency maps. [online] Available at: https://arxiv.org/abs/1312.6034 [Accessed Mar. 04, 2021].
Song, X., Zhang, G., Liu, F., Li, D., Zhao, Yu. and Yang, J. (2016). Modeling spatio-temporal distribution of soil moisture by deep learning-based cellular automata model. Journal of Arid Land, 8 (5), 734-748.
Song, T., Ding, W., Wu, J., Liu, H., Zhou, H. and Chu, J. (2020). Flash Flood Forecasting Based on Long Short-Term Memory Networks. Water, 12 (1), 109.
Sun, C., Shrivastava, A., Singh, S. and Gupta, A. (2017). Revisiting unreasonable effectiveness of data in deep learning era. In: Proceedings of the IEEE international conference on computer vision. [online] Venice: IEEE, 843-852. Available at: https://ieeexplore.ieee.org/document/8237359 [Accessed Mar. 04, 2021].
Sutskever, I., Vinyals, O. and Le, Q. V. (2014). Sequence to sequence learning with neural networks. In: Advances in neural information processing systems 27 (NIPS 2014). [online] Montreal: NIPS, 3104- 3112. Available at: https://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neuralnetworks.pdf [Accessed Mar. 04, 2021].
Tao, Yu., Gao, X., Hsu, K., Sorooshian, S. and Ihler, A. (2016). A deep neural network modeling framework to reduce bias in satellite precipitation products. Journal of Hydrometeorology, 17 (3), 931-945.
Trends.google.com. (2020). Google Trends’ Official Website. [online] Available at: https://trends.google.com [Accessed Mar. 04, 2021].
Wagstaff, K. L. and Lee, J. (2018). Interpretable discovery in large image data sets. [online] Available at: https://arxiv.org/pdf/1806.08340.pdf [Accessed Mar. 04, 2021].
Webofscience.com. (2020). Web of Science Core Collection’s Official Website. [online] Available at: https://apps.webofknowledge.com [Accessed Mar. 04, 2021].
Zhang, L., Zhang, L. and Du, B. (2016). Deep learning for remote sensing data: A technical tutorial on the state of the art. IEEE Geoscience and Remote Sensing Magazine, 4 (2), 22-40.
Zhang, J., Liu, P., Zhang, F. and Song, Q. (2018). CloudNet: Ground-based cloud classification with deep convolutional neural network. Geophysical Research Letters, 45 (16), 8665-8672.
Zhao, W. and Du, S. (2016). Learning multiscale and deep representations for classifying remotely sensed imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 113, 155-165.
Zoph, B., Vasudevan, V., Shlens, J. and Le, Q. V. (2018). Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. [online] Salt Lake City: IEEE, 8697-8710. Available at: https://ieeexplore.ieee.org/document/8579005 [Accessed Mar. 04, 2021].
Загрузки
Опубликован
Как цитировать
Выпуск
Раздел
Лицензия
Статьи журнала «Вестник Санкт-Петербургского университета. Науки о Земле» находятся в открытом доступе и распространяются в соответствии с условиями Лицензионного Договора с Санкт-Петербургским государственным университетом, который бесплатно предоставляет авторам неограниченное распространение и самостоятельное архивирование.
