SlideShare a Scribd company logo
1 of 31
Download to read offline
Integration of sensor networks and decision
       support tools for basin-scale,
    real-time water quality management

                                
        Nigel W.T. Quinn PhD, P.E., DWRE
    HydroEcological Engineering Advanced Decision Support
       Berkeley National Laboratory, Berkeley, CA 94720
       Division of Planning, US Bureau of Reclamation
                    Sacramento, CA 95825
                                   
                                       
                                           
         CYBERA GeoSpatial/Open Data Conference
                Banff Centre, Banff, CANADA 
                       October 6-8 2011
URBAN




     WETLANDS




AGRICULTURE
SALINITY REGULATION IN WESTERN
                SAN JOAQUIN VALLEY OF CALIFORNIA


l    The Central Valley Regional Water Quality Control Board has
      adopted an alternative stakeholder-centric approach to salinity
      planning and regulation “real-time salinity management”
l    Requires dischargers that are otherwise subject to WDR’s to
      adopt a “Board approved” real-time salinity management
      program
l    Program to include monitoring, real-time data access, modeling
      and decision support
l    High reliance on sensor networks and the development of a
      stakeholder supported sensor web
l    Compliance date in late 2014
EXEMPLAR : SEASONALLY MANAGED WETLANDS IN
        THE GRASSLANDS ECOLOGICAL AREA




                             170,000 acre wetland
                             footprint within the San
                             Joaquin Basin
DEFINITIONS


            ASSIMILATIVE CAPACITY
The mass load of a pollutant that can be safely discharged to a
receiving water without exceeding the water quality objective or
standard for that pollutant.


              REAL-TIME WATER QUALITY
                    MANAGEMENT
 A coordinated and cooperative set of actions based on real-time
 forecasts of river water quality to consistently meet water quality
 objectives
COMPARISON OF WEB-BASED SENSOR
               NETWORK TECHNOLOGIES



1.  Web-based sensor network using Campbell Scientific
    Loggernet software and Real-Time Data Management
    (RTDM) toolbox

2.  Web-based sensor data access and reporting using YSI-
    Econet and Aquatic Informatics Aquarius software

3.  Integrated web-based sensor data access, QA data
    processing and reporting using Kisters WISKI software

                   APPLICATION TO SEASONALLY
                   MANAGED WETLANDS
WEB-BASED SENSOR NETWORK USING CSI
LOGGERNET AND RTDM TOOLS
WEB-BASED SENSOR NETWORK USING
          CSI LOGGERNET AND RTDM TOOLS


ADVANTAGES
•  Capable of being customized to the application
•  Robust and easy to troubleshoot
DISADVANTAGES
•  Difficult to integrate cellular, GOES and land line telemetry
•  Time consuming to operate and troubleshoot even with automation
   offered in LoggerNet
•  Graphics from RTDM application stored daily as permanent jpeg or
   gif images – very storage intensive
•  Wetland biologists reluctant to spend time indoors doing data
   processing or system troubleshooting
•  Lag in data processing compromised effectiveness for RTDM
MONITORING
FLOW AND WATER QUALITY FORECASTING
YSI-ECONET SENSOR WEB
TOPOLOGY FOR WETLAND MONITORING




                                  !
WEB-BASED SENSOR DATA ACCESS
               USING YSI-ECONET


ADVANTAGES
•  Simple to install and become operational
•  Ability to restrict data access on public website to QA censored data
•  Web site customizable for display of sensor parameters, graphic
   visualization formats and backdrop GIS station maps
•  Rapid tech transfer among wetland community – new paradigm
DISADVANTAGES
•  Cannot download directly from either access or data nodes in network
•  Lack of integration with QA software
•  Difficult to overwrite preliminary data with QA-censored data
•  Inability to mix and match other telemetered data logging hardware
•  Excellent for small networks but expensive scale up to enterprise level
DATA QUALITY ASSURANCE :
                 DATA VALIDATION AND CORRECTION


l    Need to automatically flag suspect data and identify :
       –  Outliers
       –  Unusual rate of change
       –  Poor correlation with past or adjacent sensor time series
l    Visual flagging allows to quickly spot problems
l    Corrections should be performed either visually or numerically
l    Tracking and annotation of all corrections and changes
l    Original data must be retained
AQUARIUS DATA QA OBJECT MODEL
 DATA PROCESSING WHITEBOARD
FEATURES OF AQUARIUS SOFTWARE
                FOR REAL-TIME DATA PROCESSING


l    Over 30 toolboxes for most signal processing functions
l    Whiteboard concept allows users to easily build their own workflows
l    Simple drag and drop interface
l    Single-click visualization of data at any stage in workflow
l    Whiteboards can be saved for re-use, and can be run automatically
l    Not well integrated with sensor web for data downloading and QA
      data uploading to website
l    Excellent software help files, user online tutorials, case study
      examples
AQUARIUS FOR CONTINUOUS WEB-BASED
 DATA QA AND ERROR CORRECTING
FEATURES OF WISKI TOOLBOX FOR
                REAL-TIME DATA MANAGEMENT


ADVANTAGES
•  Fully integrated toolbox combining data downloading, data processing,
   data dissemination and modeling support
•  Installed user base within irrigation water district community and USFWS
   (Alaska)
•  Local presence within Northern California for user support and training
•  Robust system capable of handling thousands of network data nodes
•  Ability to perform low-cost SCADA control functions
DISADVANTAGES
•  Increased software functionality requires commitment for effective use
•  Significant effort required to access data from existing YSI-Econet system
AUTOMATED DRIFT CORRECTION OF
  REAL-TIME DATA IN WISKI




              Manual Readings
REVIEWING QA INFORMATION FOR
         REAL-TIME DATA IN WISKI




CONTROL BARS FOR GRAPHICAL REVIEW OF DATA QUALITY AND
                     COMMENTS
SETTING VALIDATION RULES FOR
  REAL-TIME DATA IN WISKI
STANDARDISATION OF INTEROPERABILITY
                      PROTOCOLS TO ENHANCE DATA SHARING

Data Integration with WISKI Web Services                       WISKI user can access SOS
                                                               Water ML2 Services and load
                             KITSM – scalable multi-tier       data into the WISKI database
 Time Series
    Data
               Metadata      architecture to to organize,
                             compute and share time series     Downloaded data can be
                             data                              included in further
                                                               calculations (agents) and
         KiTSM
                                                               analysis (statistics/ operations)

         KiWIS




               SOS/                          Data Consumer
               WaterML2
                                             Class Framework
One API which combines several
interoperability standards – allows
wetland data to be brought into Cloud
NEXT STEP : IMPLEMENTATION OF
                      WISKI WEB SERVICES INTERFACE

WEB PRO for Intranet                          WEB PUBLIC for Internet




                                                                   Data copied from
                                                                   screen or direct
                                                                   downloaded
   Display salinity concentration
   exceedence levels
                                    Utilizes graphical user
                                    interface to access data
FLOW AND WATER QUALITY SIMULATION MODELING – WARMF-SJR
 n
HUMAN FACTORS IN WETLAND REAL-TIME
               SALINITY MANAGEMENT ADOPTION


l    Recognize institutional constraints of participating stakeholders :
      Federal and State agencies have autonomy over their decisions :
      water districts and private wetlands answer to their Boards
l    Private entities that are not as well funded as State and Federal
      agencies. Incentive programs could be combined with existing
      habitat programs as agents of change.
l    Collaborations with regulators to develop interim salt load
      targets - creating a transition period for wetland management to
      learn by doing and improve drainage salt load scheduling
      incrementally (adaptively)
ADAPTIVELY MANAGING WETLAND
             REAL-TIME MANAGEMENT INVESTMENTS


l    Adaptive management dictates a feedback mechanism to prevent
      irreversible damage to wetland resource through real-time salinity
      management while promoting and sharing successful outcomes
l    Learning by doing develops experiential knowledge base that can
      guide future actions and operations. This is necessarily a long-term
      strategy give the inter-annual variability of climate and water supply
      allocations. Provides hedge against uncertainty.
l    By its nature a long-term planning strategy – 10 to 15 year planning
      horizon for technology transfer and institutional adoption.
l    Need to plan for long term financing of essential components such
      as enhanced data sharing and management technologies.
INSTITUTIONAL ASSURANCES TO
                       INCREASE PACE OF ADOPTION

l    California has well-financed stakeholder interest lobbies - impossible to
      satisfy all stakeholder interests. Every information management and
      decision support system is, by nature, compromised at the design phase
l    Assurances necessary to reduce perceived risk of adoption – otherwise
      easier to employ litigation to avoid change
l    Assurances can only be given by statutory bodies with institutional clout
      to make long-term promises
l    Assurances need to be backed up with data collection to better understand
      long-term trends – otherwise no proof of harm
l    Needs to be understood that system impacts can take years to develop -
      though physically reversible may be difficult to remedy institutionally
SUMMARY AND CONCLUSIONS


l    Real-time water quality (salinity) management allows greater salt export
      than traditional load-based TMDL’s.
l    For seasonally managed wetlands RTSM is the only long-term option if
      waterfowl habitat is to be sustained
l    RTSM will require integration of data acquisition, processing, model
      forecasting, information dissemination and decision support
l    Technical progression in capability of sensors and supporting software
      over past decade essential for implementation of RTSM
l    Full TMDL compliance required by 2014 – major challenge for
      cooperative data sharing and coordination of actions between
      agriculture, wetland interests, municipal and industrial stakeholders

More Related Content

Similar to Integration of sensor networks and decision support tools for basin-scale, real-time water quality management

Element Blue IOC Express Program
Element Blue IOC Express ProgramElement Blue IOC Express Program
Element Blue IOC Express Program
Steven Gerhardt
 
Monitoring and sustaining services: Lessons learned from WaterAid's post-impl...
Monitoring and sustaining services: Lessons learned from WaterAid's post-impl...Monitoring and sustaining services: Lessons learned from WaterAid's post-impl...
Monitoring and sustaining services: Lessons learned from WaterAid's post-impl...
IRC
 
Riverbed Whitewater Datasheet
Riverbed Whitewater DatasheetRiverbed Whitewater Datasheet
Riverbed Whitewater Datasheet
laurenfortune
 

Similar to Integration of sensor networks and decision support tools for basin-scale, real-time water quality management (20)

Smart Water Networks
Smart Water NetworksSmart Water Networks
Smart Water Networks
 
22 - CSIRO - Water Data Management-Sep-17
22 - CSIRO - Water Data Management-Sep-1722 - CSIRO - Water Data Management-Sep-17
22 - CSIRO - Water Data Management-Sep-17
 
Rims forum 2013 aspec data standards - george havakis gissa
Rims forum 2013   aspec data standards - george havakis gissaRims forum 2013   aspec data standards - george havakis gissa
Rims forum 2013 aspec data standards - george havakis gissa
 
Integrated Groundwater Management
Integrated Groundwater ManagementIntegrated Groundwater Management
Integrated Groundwater Management
 
VMworld 2013: From Virtualization to Cloud: How Automation Drives Agility
VMworld 2013: From Virtualization to Cloud: How Automation Drives Agility VMworld 2013: From Virtualization to Cloud: How Automation Drives Agility
VMworld 2013: From Virtualization to Cloud: How Automation Drives Agility
 
Element Blue IOC Express Program
Element Blue IOC Express ProgramElement Blue IOC Express Program
Element Blue IOC Express Program
 
Visualizing Application & Delivery Flows to Make Data-Driven Decisions
Visualizing Application & Delivery Flows to Make Data-Driven DecisionsVisualizing Application & Delivery Flows to Make Data-Driven Decisions
Visualizing Application & Delivery Flows to Make Data-Driven Decisions
 
Conversion to MapGuide® Open Source and iVAULT
Conversion to MapGuide® Open Source and iVAULTConversion to MapGuide® Open Source and iVAULT
Conversion to MapGuide® Open Source and iVAULT
 
IRJET - Implementation of VPN, AWS Cloud Enabled Real Time Water Pollution Sy...
IRJET - Implementation of VPN, AWS Cloud Enabled Real Time Water Pollution Sy...IRJET - Implementation of VPN, AWS Cloud Enabled Real Time Water Pollution Sy...
IRJET - Implementation of VPN, AWS Cloud Enabled Real Time Water Pollution Sy...
 
SMART SEWER NETWORK MONITORING USING IOT AND AI
SMART SEWER NETWORK MONITORING USING IOT AND AISMART SEWER NETWORK MONITORING USING IOT AND AI
SMART SEWER NETWORK MONITORING USING IOT AND AI
 
Water Quality - Lukas Kuenzel.pdf
Water Quality - Lukas Kuenzel.pdfWater Quality - Lukas Kuenzel.pdf
Water Quality - Lukas Kuenzel.pdf
 
EMERGENCY EVENT MANAGEMENT SYSTEM: SOLUTION ACROSS ALL AREAS OF BUSINESS
EMERGENCY EVENT MANAGEMENT SYSTEM: SOLUTION ACROSS ALL AREAS OF BUSINESSEMERGENCY EVENT MANAGEMENT SYSTEM: SOLUTION ACROSS ALL AREAS OF BUSINESS
EMERGENCY EVENT MANAGEMENT SYSTEM: SOLUTION ACROSS ALL AREAS OF BUSINESS
 
Cisco Connect 2018 Singapore - Easing the Transition
Cisco Connect 2018 Singapore - Easing the Transition Cisco Connect 2018 Singapore - Easing the Transition
Cisco Connect 2018 Singapore - Easing the Transition
 
Transforming Our Cities: High Performance Green Infrastructure and Distribute...
Transforming Our Cities: High Performance Green Infrastructure and Distribute...Transforming Our Cities: High Performance Green Infrastructure and Distribute...
Transforming Our Cities: High Performance Green Infrastructure and Distribute...
 
Monitoring and sustaining services: Lessons learned from WaterAid's post-impl...
Monitoring and sustaining services: Lessons learned from WaterAid's post-impl...Monitoring and sustaining services: Lessons learned from WaterAid's post-impl...
Monitoring and sustaining services: Lessons learned from WaterAid's post-impl...
 
Riverbed Whitewater Datasheet
Riverbed Whitewater DatasheetRiverbed Whitewater Datasheet
Riverbed Whitewater Datasheet
 
OPTIMISING & AUTOMATING THE MELBOURNE WATER TRANSMISSION NETWORK
OPTIMISING & AUTOMATING THE MELBOURNE WATER TRANSMISSION NETWORKOPTIMISING & AUTOMATING THE MELBOURNE WATER TRANSMISSION NETWORK
OPTIMISING & AUTOMATING THE MELBOURNE WATER TRANSMISSION NETWORK
 
Waters {jal , pani}
Waters {jal , pani}Waters {jal , pani}
Waters {jal , pani}
 
IRJET- Flood Alerting System through Water Level Meter
IRJET-  	  Flood Alerting System through Water Level MeterIRJET-  	  Flood Alerting System through Water Level Meter
IRJET- Flood Alerting System through Water Level Meter
 
Innovative and essential data for managing toward your DFC
Innovative and essential data for managing toward your DFCInnovative and essential data for managing toward your DFC
Innovative and essential data for managing toward your DFC
 

More from Cybera Inc.

Cyber Summit 2016: Technology, Education, and Democracy
Cyber Summit 2016: Technology, Education, and DemocracyCyber Summit 2016: Technology, Education, and Democracy
Cyber Summit 2016: Technology, Education, and Democracy
Cybera Inc.
 
Cyber Summit 2016: Research Data and the Canadian Innovation Challenge
Cyber Summit 2016: Research Data and the Canadian Innovation ChallengeCyber Summit 2016: Research Data and the Canadian Innovation Challenge
Cyber Summit 2016: Research Data and the Canadian Innovation Challenge
Cybera Inc.
 
Cyber Summit 2016: Privacy Issues in Big Data Sharing and Reuse
Cyber Summit 2016: Privacy Issues in Big Data Sharing and ReuseCyber Summit 2016: Privacy Issues in Big Data Sharing and Reuse
Cyber Summit 2016: Privacy Issues in Big Data Sharing and Reuse
Cybera Inc.
 

More from Cybera Inc. (20)

Cyber Summit 2016: Technology, Education, and Democracy
Cyber Summit 2016: Technology, Education, and DemocracyCyber Summit 2016: Technology, Education, and Democracy
Cyber Summit 2016: Technology, Education, and Democracy
 
Cyber Summit 2016: Understanding Users' (In)Secure Behaviour
Cyber Summit 2016: Understanding Users' (In)Secure BehaviourCyber Summit 2016: Understanding Users' (In)Secure Behaviour
Cyber Summit 2016: Understanding Users' (In)Secure Behaviour
 
Cyber Summit 2016: Insider Threat Indicators: Human Behaviour
Cyber Summit 2016: Insider Threat Indicators: Human BehaviourCyber Summit 2016: Insider Threat Indicators: Human Behaviour
Cyber Summit 2016: Insider Threat Indicators: Human Behaviour
 
Cyber Summit 2016: Research Data and the Canadian Innovation Challenge
Cyber Summit 2016: Research Data and the Canadian Innovation ChallengeCyber Summit 2016: Research Data and the Canadian Innovation Challenge
Cyber Summit 2016: Research Data and the Canadian Innovation Challenge
 
Cyber Summit 2016: Knowing More and Understanding Less in the Age of Big Data
Cyber Summit 2016: Knowing More and Understanding Less in the Age of Big DataCyber Summit 2016: Knowing More and Understanding Less in the Age of Big Data
Cyber Summit 2016: Knowing More and Understanding Less in the Age of Big Data
 
Cyber Summit 2016: Privacy Issues in Big Data Sharing and Reuse
Cyber Summit 2016: Privacy Issues in Big Data Sharing and ReuseCyber Summit 2016: Privacy Issues in Big Data Sharing and Reuse
Cyber Summit 2016: Privacy Issues in Big Data Sharing and Reuse
 
Cyber Summit 2016: Establishing an Ethics Framework for Predictive Analytics ...
Cyber Summit 2016: Establishing an Ethics Framework for Predictive Analytics ...Cyber Summit 2016: Establishing an Ethics Framework for Predictive Analytics ...
Cyber Summit 2016: Establishing an Ethics Framework for Predictive Analytics ...
 
Cyber Summit 2016: The Data Tsunami vs The Network: How More Data Changes Eve...
Cyber Summit 2016: The Data Tsunami vs The Network: How More Data Changes Eve...Cyber Summit 2016: The Data Tsunami vs The Network: How More Data Changes Eve...
Cyber Summit 2016: The Data Tsunami vs The Network: How More Data Changes Eve...
 
Cyber Summit 2016: Issues and Challenges Facing Municipalities In Securing Data
Cyber Summit 2016: Issues and Challenges Facing Municipalities In Securing DataCyber Summit 2016: Issues and Challenges Facing Municipalities In Securing Data
Cyber Summit 2016: Issues and Challenges Facing Municipalities In Securing Data
 
Cyber Summit 2016: Using Law Responsibly: What Happens When Law Meets Technol...
Cyber Summit 2016: Using Law Responsibly: What Happens When Law Meets Technol...Cyber Summit 2016: Using Law Responsibly: What Happens When Law Meets Technol...
Cyber Summit 2016: Using Law Responsibly: What Happens When Law Meets Technol...
 
Privacy, Security & Access to Data
Privacy, Security & Access to DataPrivacy, Security & Access to Data
Privacy, Security & Access to Data
 
Do Universities Dream of Big Data
Do Universities Dream of Big DataDo Universities Dream of Big Data
Do Universities Dream of Big Data
 
Predicting the Future With Microsoft Bing
Predicting the Future With Microsoft BingPredicting the Future With Microsoft Bing
Predicting the Future With Microsoft Bing
 
Analytics 101: How to not fail at analytics
Analytics 101: How to not fail at analyticsAnalytics 101: How to not fail at analytics
Analytics 101: How to not fail at analytics
 
Are MOOC's past their peak?
Are MOOC's past their peak?Are MOOC's past their peak?
Are MOOC's past their peak?
 
Opening the doors of the laboratory
Opening the doors of the laboratoryOpening the doors of the laboratory
Opening the doors of the laboratory
 
Open City - Edmonton
Open City - EdmontonOpen City - Edmonton
Open City - Edmonton
 
Unlocking the power of healthcare data
Unlocking the power of healthcare dataUnlocking the power of healthcare data
Unlocking the power of healthcare data
 
Checking in on Healthcare Data Analytics
Checking in on Healthcare Data AnalyticsChecking in on Healthcare Data Analytics
Checking in on Healthcare Data Analytics
 
Open access and open data: international trends and strategic context
Open access and open data: international trends and strategic contextOpen access and open data: international trends and strategic context
Open access and open data: international trends and strategic context
 

Recently uploaded

Recently uploaded (20)

Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 

Integration of sensor networks and decision support tools for basin-scale, real-time water quality management

  • 1. Integration of sensor networks and decision support tools for basin-scale, real-time water quality management Nigel W.T. Quinn PhD, P.E., DWRE HydroEcological Engineering Advanced Decision Support Berkeley National Laboratory, Berkeley, CA 94720 Division of Planning, US Bureau of Reclamation Sacramento, CA 95825 CYBERA GeoSpatial/Open Data Conference Banff Centre, Banff, CANADA October 6-8 2011
  • 2.
  • 3. URBAN WETLANDS AGRICULTURE
  • 4. SALINITY REGULATION IN WESTERN SAN JOAQUIN VALLEY OF CALIFORNIA l  The Central Valley Regional Water Quality Control Board has adopted an alternative stakeholder-centric approach to salinity planning and regulation “real-time salinity management” l  Requires dischargers that are otherwise subject to WDR’s to adopt a “Board approved” real-time salinity management program l  Program to include monitoring, real-time data access, modeling and decision support l  High reliance on sensor networks and the development of a stakeholder supported sensor web l  Compliance date in late 2014
  • 5. EXEMPLAR : SEASONALLY MANAGED WETLANDS IN THE GRASSLANDS ECOLOGICAL AREA 170,000 acre wetland footprint within the San Joaquin Basin
  • 6. DEFINITIONS ASSIMILATIVE CAPACITY The mass load of a pollutant that can be safely discharged to a receiving water without exceeding the water quality objective or standard for that pollutant. REAL-TIME WATER QUALITY MANAGEMENT A coordinated and cooperative set of actions based on real-time forecasts of river water quality to consistently meet water quality objectives
  • 7.
  • 8. COMPARISON OF WEB-BASED SENSOR NETWORK TECHNOLOGIES 1.  Web-based sensor network using Campbell Scientific Loggernet software and Real-Time Data Management (RTDM) toolbox 2.  Web-based sensor data access and reporting using YSI- Econet and Aquatic Informatics Aquarius software 3.  Integrated web-based sensor data access, QA data processing and reporting using Kisters WISKI software APPLICATION TO SEASONALLY MANAGED WETLANDS
  • 9. WEB-BASED SENSOR NETWORK USING CSI LOGGERNET AND RTDM TOOLS
  • 10. WEB-BASED SENSOR NETWORK USING CSI LOGGERNET AND RTDM TOOLS ADVANTAGES •  Capable of being customized to the application •  Robust and easy to troubleshoot DISADVANTAGES •  Difficult to integrate cellular, GOES and land line telemetry •  Time consuming to operate and troubleshoot even with automation offered in LoggerNet •  Graphics from RTDM application stored daily as permanent jpeg or gif images – very storage intensive •  Wetland biologists reluctant to spend time indoors doing data processing or system troubleshooting •  Lag in data processing compromised effectiveness for RTDM
  • 12. FLOW AND WATER QUALITY FORECASTING
  • 13. YSI-ECONET SENSOR WEB TOPOLOGY FOR WETLAND MONITORING !
  • 14. WEB-BASED SENSOR DATA ACCESS USING YSI-ECONET ADVANTAGES •  Simple to install and become operational •  Ability to restrict data access on public website to QA censored data •  Web site customizable for display of sensor parameters, graphic visualization formats and backdrop GIS station maps •  Rapid tech transfer among wetland community – new paradigm DISADVANTAGES •  Cannot download directly from either access or data nodes in network •  Lack of integration with QA software •  Difficult to overwrite preliminary data with QA-censored data •  Inability to mix and match other telemetered data logging hardware •  Excellent for small networks but expensive scale up to enterprise level
  • 15.
  • 16.
  • 17. DATA QUALITY ASSURANCE : DATA VALIDATION AND CORRECTION l  Need to automatically flag suspect data and identify : –  Outliers –  Unusual rate of change –  Poor correlation with past or adjacent sensor time series l  Visual flagging allows to quickly spot problems l  Corrections should be performed either visually or numerically l  Tracking and annotation of all corrections and changes l  Original data must be retained
  • 18. AQUARIUS DATA QA OBJECT MODEL DATA PROCESSING WHITEBOARD
  • 19. FEATURES OF AQUARIUS SOFTWARE FOR REAL-TIME DATA PROCESSING l  Over 30 toolboxes for most signal processing functions l  Whiteboard concept allows users to easily build their own workflows l  Simple drag and drop interface l  Single-click visualization of data at any stage in workflow l  Whiteboards can be saved for re-use, and can be run automatically l  Not well integrated with sensor web for data downloading and QA data uploading to website l  Excellent software help files, user online tutorials, case study examples
  • 20. AQUARIUS FOR CONTINUOUS WEB-BASED DATA QA AND ERROR CORRECTING
  • 21. FEATURES OF WISKI TOOLBOX FOR REAL-TIME DATA MANAGEMENT ADVANTAGES •  Fully integrated toolbox combining data downloading, data processing, data dissemination and modeling support •  Installed user base within irrigation water district community and USFWS (Alaska) •  Local presence within Northern California for user support and training •  Robust system capable of handling thousands of network data nodes •  Ability to perform low-cost SCADA control functions DISADVANTAGES •  Increased software functionality requires commitment for effective use •  Significant effort required to access data from existing YSI-Econet system
  • 22. AUTOMATED DRIFT CORRECTION OF REAL-TIME DATA IN WISKI Manual Readings
  • 23. REVIEWING QA INFORMATION FOR REAL-TIME DATA IN WISKI CONTROL BARS FOR GRAPHICAL REVIEW OF DATA QUALITY AND COMMENTS
  • 24. SETTING VALIDATION RULES FOR REAL-TIME DATA IN WISKI
  • 25. STANDARDISATION OF INTEROPERABILITY PROTOCOLS TO ENHANCE DATA SHARING Data Integration with WISKI Web Services WISKI user can access SOS Water ML2 Services and load KITSM – scalable multi-tier data into the WISKI database Time Series Data Metadata architecture to to organize, compute and share time series Downloaded data can be data included in further calculations (agents) and KiTSM analysis (statistics/ operations) KiWIS SOS/ Data Consumer WaterML2 Class Framework One API which combines several interoperability standards – allows wetland data to be brought into Cloud
  • 26. NEXT STEP : IMPLEMENTATION OF WISKI WEB SERVICES INTERFACE WEB PRO for Intranet WEB PUBLIC for Internet Data copied from screen or direct downloaded Display salinity concentration exceedence levels Utilizes graphical user interface to access data
  • 27. FLOW AND WATER QUALITY SIMULATION MODELING – WARMF-SJR n
  • 28. HUMAN FACTORS IN WETLAND REAL-TIME SALINITY MANAGEMENT ADOPTION l  Recognize institutional constraints of participating stakeholders : Federal and State agencies have autonomy over their decisions : water districts and private wetlands answer to their Boards l  Private entities that are not as well funded as State and Federal agencies. Incentive programs could be combined with existing habitat programs as agents of change. l  Collaborations with regulators to develop interim salt load targets - creating a transition period for wetland management to learn by doing and improve drainage salt load scheduling incrementally (adaptively)
  • 29. ADAPTIVELY MANAGING WETLAND REAL-TIME MANAGEMENT INVESTMENTS l  Adaptive management dictates a feedback mechanism to prevent irreversible damage to wetland resource through real-time salinity management while promoting and sharing successful outcomes l  Learning by doing develops experiential knowledge base that can guide future actions and operations. This is necessarily a long-term strategy give the inter-annual variability of climate and water supply allocations. Provides hedge against uncertainty. l  By its nature a long-term planning strategy – 10 to 15 year planning horizon for technology transfer and institutional adoption. l  Need to plan for long term financing of essential components such as enhanced data sharing and management technologies.
  • 30. INSTITUTIONAL ASSURANCES TO INCREASE PACE OF ADOPTION l  California has well-financed stakeholder interest lobbies - impossible to satisfy all stakeholder interests. Every information management and decision support system is, by nature, compromised at the design phase l  Assurances necessary to reduce perceived risk of adoption – otherwise easier to employ litigation to avoid change l  Assurances can only be given by statutory bodies with institutional clout to make long-term promises l  Assurances need to be backed up with data collection to better understand long-term trends – otherwise no proof of harm l  Needs to be understood that system impacts can take years to develop - though physically reversible may be difficult to remedy institutionally
  • 31. SUMMARY AND CONCLUSIONS l  Real-time water quality (salinity) management allows greater salt export than traditional load-based TMDL’s. l  For seasonally managed wetlands RTSM is the only long-term option if waterfowl habitat is to be sustained l  RTSM will require integration of data acquisition, processing, model forecasting, information dissemination and decision support l  Technical progression in capability of sensors and supporting software over past decade essential for implementation of RTSM l  Full TMDL compliance required by 2014 – major challenge for cooperative data sharing and coordination of actions between agriculture, wetland interests, municipal and industrial stakeholders