Collaborative Observatories for Natural Environments
PI: Dezhen Song (Texas A&M University) CO-PI: Ken Goldberg (UC Berkeley)

This project suggests a new model that has potential to transform conventional approaches to Field Biology. By incorporating advances in robotic cameras, long-range wireless networking, and distributed sensors, it is possible to create "Collaborative Observatories for Natural Environments" (CONEs) that allow human volunteers to observe and record detailed animal behavior in remote settings. The "Observatories" developed under this project have engaged thousands of citizen-scientists around the world who are systematically photographing and collecting data on subtropical birds to study the causes of range shifts, which may include global effects such as climate change.

The NSF project is a collaborative effort developed bycomputer scientists and engineers from Texas A&M, UC Berkeley, ornithologists from Cornell and the Smithsonian National Zoological Park.

Currently, scientific study of animals in situ requires vigilant observation of detailed animal behavior over weeks or months. When animals live in remote and/or inhospitable locations, observation can be an arduous, expensive, dangerous, and lonely experience for scientists. The project investigates a new class of hybrid teleoperated/autonomous robotic "observatories" that allow groups of citizens and scientists, via the internet, to remotely observe, record, and index detailed animal activity.

Such observatories are made possible by emerging advances in robotic cameras, long-range wireless networking, and distributed sensors. This project requires advances in the algorithmic foundations for such observatories: new metrics, models, data structures, and algorithms, that will comprise a robust, mathematical framework for collaborative observation. The project draws on computational geometry, stochastic modeling and optimization to advance fundamental understanding of networked robotics and develop efficient algorithms for collaborative observation that combines human and sensor input.

We have designed and implemented four working prototypes that have been accessible via the internet to scientists, students, and the public worldwide. This project website, includes details on the four prototypes.

The most recent Networked Observatory is at the Welder Wildlife Refuge in Texas. CONE-Welder was designed to gather photographic and quantitative data for a biological study of avian activity and hypothesized range change for selected subtropical bird species. Since the system was deployed on 12 May 2008, over 600 users have participated online. Players have requested over 2.2 million camera frames and captured over 29,000 photographs. Within these photos, citizen scientists have classied 74 unique species, including eight avian species previously unknown to have breeding populations within the region. The collected dataset quanties seasonal presence of birds of particular interest, e.g., the Green Jay (Cyanocorax incas).

Updates, hardware designs, CAD models, schematics, source code, experimental data, and documentation will be  posted on this website as they emerge.

  • CONE Sutro Forest  
              Collaborative observatory to identify birds in sutro forest, California, April 2007 to November 2007
  • ACONE 1.0  
               Automated Observatory to assist in the search for the Ivory-Billed Woodpecker, Arkansas, Fall 2006-Fall 2007
  • CONE 1.0  
              Collaborative Observatory for Natural Environments, Audobon Nature Preserve, Mill Valley, CA, Fall 2005

 

 NSF Award 0534848/0535218/0643298
Robotics and Robust Intelligence Program
Division of Information and Intelligent Systems
Directorate for Computer Science and Engineering
National Science Foundation
Media coverage
Related projects

 
Related Publications

  1. Dezhen Song and Yiliang Yu, A Low False Negative Filter for Detecting Rare Bird Species from Short Video Segments using a Probable Observation Data Set-based EKF Method, IEEE Transactions on Image Processing, vol. 19, no. 9, Sept. 2010, pp. 2321-2331 [Download from IEEE]

  2. Yiliang Xu and Dezhen Song, Systems and Algorithms for Autonomous and Scalable Crowd Surveillance Using Robotic PTZ Cameras Assisted by a Wide-Angle Camera, Autonomous Robots, Volume 29, Number 1 / July, 2010, pp.53-66 [Download from Springer]

  3. Dezhen Song, Yiliang Xu, and Ni Qin, Aligning Windows of Live Video from an Imprecise Pan-Tilt-Zoom Camera into a Remote Panoramic Display for Remote Nature Observation, Journal of Real Time Image Processing, Vol. 5, Issue 1, 2010, pp. 57-71. [Download from Springer]

  4. Yiliang Xu, Dezhen Song, and Jingang Yi, Exact Algorithms for Non-Overlapping 2-Frame Problem with Non-Partial Coverage for Networked Robotic Cameras, the 6th annual IEEE Conference on Automation Science and Engineering (CASE 2010), August 21-24, 2010, Toronto, Ontario, Canada [pdf 911k]

  5. Dezhen Song and Yiliang Yu, A Low False Negative Filter for Detecting Rare Bird Species from Short Video Segments using a Probable Observation Data Set-based EKF Method, Special Track on Physically Grounded AI (PGAI), the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI-10), Atlanta, Georgia, USA, July 11–15, 2010 [pdf 362k]

  6. Yiliang Xu and Dezhen Song, Systems and Algorithms for Autonomously Simultaneous Observation of Multiple Objects Using Robotic PTZ Cameras Assisted by a Wide-Angle Camera, The 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), St. Louis, USA, Oct. 11-15, 2009 [pdf 510k]

  7. Siamak Faridani, Bryce Lee, Selma Glasscock, John Rappole, Dezhen Song, Ken Goldberg, A Networked Telerobotic Observatory for Collaborative Remote Observation of Avian Activity and Range Change, the IFAC workshop on networked robots, Oct. 6-8, 2009, Golden, Colorado [pdf 1.4M]

  8. Dezhen Song, Sharing a Vision: Systems and Algorithms for Collaboratively-Teleoperated Robotic Cameras, a Monograph in Springer Tracts on Advanced Robotics, Vol. 51, ISBN: 978-3-540-88064-6, 2009, Springer.

  9. Dezhen Song, Ni Qin, and Ken Goldberg, Systems, Control Models, and Codec for Collaborative Observation of Remote Environments with an Autonomous Networked Robotic Camera, Autonomous Robots, May 2008, Vol. 24, No. 4, pp. 435–449 [Download from Springer]

  10. Dezhen Song and Ken Goldberg, Approximate Algorithms for a Collaboratively Controlled Robotic Camera, IEEE Transactions on Robotics, Vol. 23, No. 5, Oct. 2007, pp. 1061-1070 [Download from IEEE]

  11. Dezhen Song, A Frank van der Stappen, and Ken Goldberg, Exact Algorithms for Single Frame Selection on Multi-Axis Satellites, IEEE Transactions on Automation Science and Engineering, Vol. 3, No. 1. January 2006, pp.16-28.[Download from IEEE].

  12. Yiliang Xu, Dezhen Song, Jingang Yi, and A.Frank van der Stappen, An Approximation Algorithm for the Least Overlapping p-Frame Problem with Non-Partial Coverage for Networked Robotic Cameras, IEEE International Conference on Robotics and Automation (ICRA), Pasadena, CA, May 2008. [pdf 310k]

  13. Ni Qin and Dezhen Song, On-Demand Sharing of a High-Resolution Panorama Video from Networked Robotic Cameras, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct, 2007, San Diego, CA [pdf 260k]

  14. Dezhen Song and Ken Goldberg, Networked Robotic Cameras for Collaborative Observation of Natural Environments, The 12th International Symposium of Robotics Research (ISRR 2005), October 12th-15th, 2005, San Francisco, CA, USA [pdf 480k]

  15. Dezhen Song, Ni Qin, and Ken Goldberg, A Minimum Variance Calibration Algorithm for Pan-Tilt Robotic Cameras in Natural Environments, IEEE International Conference on Robotics and Automation (ICRA), May. 2006, Orlando, Florida [pdf 250k]

  16. Ni Qin, Dezhen Song, and Ken Goldberg, Aligning Windows of Live Video from an Imprecise Pan-Tilt-Zoom Robotic Camera into a Remote Panoramic Display, IEEE International Conference on Robotics and Automation (ICRA), May. 2006, Orlando, Florida [pdf 380k]

  17. Dezhen Song, Qiang Hu, Ni Qin, and Ken Goldberg, Automating Inspection and Documentation of Remote Building Construction using a Robotic Camera, IEEE International Conference on Automation Science and Engineering (CASE) 2005,  August 1 & 2, 2005, Edmonton, Canada [pdf 550k]

  18. Dezhen Song, A. Frank van der Stappen, and Ken Goldberg, An Exact Algorithm Optimizing Coverage-Resolution for Automated Satellite Frame Selection, IEEE International Conference on Robotics and Automation (ICRA) 2004 [pdf 809k]

  19. Dezhen Song, Algorithms and Systems for Shared Access to a Robotic Streaming Video Camera, Doctoral Symposium, ACM Multimedia 2003 (MM2003), Berkeley, California, Nov., 2003 [pdf  125k]

  20. D. Song and K. Goldberg, ShareCam Part I: Interface, System Architecture, and Implementation of a Collaboratively Controlled Robotic Webcam, IEEE/RSJ International Conference on Intelligent Robots and Systems, Las Vegas, Nevada, Oct. 2003. [pdf 737k]

  21. D. Song, K. Goldberg, and A. Pashkevich, ShareCam Part II: Approximate and Distributed Algorithms for a Collaboratively Controlled Robotic Webcam, IEEE/RSJ International Conference on Intelligent Robots and Systems, Las Vegas, Nevada, Oct.2003. [pdf 326k]

  22. S. Har-Peled, V. Koltun, D. Song, and K. Goldberg, Efficient Algorithms for Shared Camera Control, In Proceedings of the 19th ACM Symposium on Computational Geometry, 2003. [pdf 380k]

  23. D. Song, A.F. van der Stappen, and K. Goldberg, Exact and Distributed Algorithms for Collaborative Camera Control, the Fifth International Workshop on Algorithmic Foundations of Robotics. Nice, France, Dec 15~17, 2002. [pdf 600k]

"Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation."

Last modified: Friday Sep. 24 15:28:33 CDT 2010