NYS ITS GIS Program Office
Geographic Information Systems Clearinghouse
An Innovative New Model for Data Sharing and Partnerships
NYS Department of Transportation
October 2, 1997
- What is the NYS GIS Data Sharing Cooperative?
- Why is a GIS Cooperative needed?
- What are the benefits?
- How is it being implemented?
What is the NYS GIS Data Sharing Cooperative?
It is a mechanism that encourages public agencies in New York to share in the creation, use, and maintenance of GIS datasets at the least possible cost while providing citizens, educational institutions, and the news media easy access to data for non-commercial uses. The idea for the GIS Cooperative came about in response to the issues which discourage investments in data creation, sharing, and maintenance in New York State.
Agencies join the Cooperative by executing the NYS GIS Cooperative Data Sharing Agreement which defines the parameters of data sharing and the obligations of the members. The Cooperative is administered by the NYS GIS Coordinating Body which is responsible for maintaining the member registry, promotion of the Cooperative, coordination with the NYS GIS Clearinghouse website to catalog the available datasets, promulgation of standards, mediation of disputes among Cooperative members, and the setting of policy to the extent required.
While it may be helpful to think of the GIS Cooperative as a data warehouse where agencies contribute data and access data created by others, in practice it is a virtual warehouse. That is, datasets continue to be owned and managed by their originating agencies, and other users obtain the datasets from these owners. Each dataset has a single recognized owner known as the Primary Custodian of that dataset. The Cooperative makes known the existence of the datasets (via the GIS Clearinghouse) so that other members can readily obtain datasets, as Secondary Custodians, as their needs dictate. Both Primary and Secondary Custodians have obligations regarding redistribution and maintenance of the datasets.
A key characteristic is the shared data maintenance responsibilities for members of the GIS Cooperative. The Cooperative recognizes that users want and need up-to-date data, and that users will discover errors or perform updates as a normal consequence of using the data. As use of members' datasets increases, resolution of inconsistencies and errors in them will improve their quality for the owners of the datasets as well as others who obtain the data from Primary Custodians.
Why is a GIS Cooperative needed?
Geographic Information Systems are powerful and useful tools only when populated with a wide array of spatial datasets. Perhaps more than any other computer application, the value of the system to its user increases dramatically with the ability to combine and explore relationships among and between many spatial data layers. Unfortunately, the most valuable and beneficial datasets (i.e. those created at the highest spatial resolutions) are the most costly to create and maintain. It has been estimated that as much as 80% of the cost of operating a GIS is associated with creating and maintaining the data. Ironically, local governments, those most in need of high resolution spatial data, are generally least able to undertake the investment alone. Only a small number have undertaken the effort and expense of developing a high resolution database for their jurisdiction. State agencies, with bigger budgets for data development, can often deliver their mission with lower resolution (lower cost) data. Still, only a relatively small number of agencies at any level of government are making significant spatial data development investments. Even these agencies need data from other agencies to realize the full benefits of GIS. Truly, partnerships are needed to share in the creation and coordinated use of GIS datasets between government and private entities at all levels. The GIS Cooperative is envisioned as the mechanism to foster these partnerships and invite wide participation in GIS efforts.
Many consider the current GIS data sharing environment in New York State to be dysfunctional. In response to New York's uncertain legal environment for GIS, individual agencies have enacted a wide range of data sharing policies. Copyright, fees, licensing, liability, and privacy protection have been handled in a seemingly infinite variety of combinations. The net effect is a cumbersome and restrictive data sharing environment -- just the opposite of what is needed! As a result, some agencies develop datasets that duplicate data already available through another agency. Smaller agencies may have little or no direct budget for GIS data development, and may seek to obtain free data, whether or not it truly meets their needs. These free datasets (such as the USGS's DLG files or the Census Bureau's TIGER file) frequently need significant effort to bring them up-to-date, correct errors, or otherwise adapt them to meet user needs. Repairing or adapting datasets is a 'hidden cost' that public agencies can sometimes absorb without notice since it does not require a purchase justification. It is not hard to imagine that more than one agency may independently and unknowingly improve the same dataset.
Those wishing to exchange data typically do so with little more than ad hoc arrangements. This approach affords no consistency; a single agency may have numerous data sharing agreements (formal or informal) with other agencies, each tailored separately. This inconsistency serves to complicate widespread use of datasets that might otherwise have broad applicability, and makes it difficult for an agency actively using GIS to keep track of and honor all of the separate data sharing arrangements in which it may be participating. The important issue of data maintenance is often overlooked in these arrangements, in spite of the fact that the usefulness of the data is often directly related to its temporal quality. Liability and privacy concerns are also inadequately addressed in many GIS data sharing situations today. In addition, state agencies and local governments have expended time and effort to craft unique licenses for data sharing and distribution, thereby wasting valuable resources.
Finally, we need to recognize that data is a commodity that has real value. In this so-called 'Information Age', new categories of businesses have emerged that acquire data from public agencies through FOIL, then repackage and resell datasets and derivative datasets at a profit, often to other public agencies. Some would argue that the expenditure of scarce resources in the public sector to comply with these types of FOIL requests represents a subsidy to the commercial sector and is beyond the spirit and intent of FOIL.
In summary, the current situation is wasteful of resources as agencies duplicate efforts or fail to coordinate their efforts and hinders the development of partnerships needed to finance high resolution datasets.
What are the benefits?
- Efficiencies - Cooperative members will be able to readily and easily obtain datasets from other members of the Cooperative. The use of a standardized data sharing agreement means that the 'rules' for sharing within the Cooperative will be consistent. The GIS Clearinghouse will facilitate easy searching for datasets of interest, thereby minimizing missed opportunities to use existing datasets.
- Improved Data - Since users will be passing updates, corrections, and revisions back to Primary Custodians, data quality will increase as use of member's datasets increase. All Cooperative members, as well as the public and private sector users of the data, will benefit from these improvements in data quality. As a result of data improvements being provided to Primary Custodians by Secondary Custodians, those obtaining datasets from the Primary Custodian can be confident that they are getting the 'best' version of the file available.
- Savings - Duplication of effort in creating datasets already available from other agencies will be minimized. Datasets will be available between Cooperative members at a cost not to exceed the marginal cost of dissemination. The use of a standard data sharing agreement will eliminate the need for communities across the state to pay their attorneys to develop new agreements from scratch.
- Organization - The GIS Cooperative is administered by the GIS Coordinating Body, which also operates a GIS Clearinghouse to organize and make known the availability of datasets statewide. It will be easy for government agencies, citizens, and commercial entities to determine what is available and who is the Primary Custodian.
- Ownership - Each dataset has a designated Primary Custodian, who remains the owner of the data. All members of the Cooperative share in maintaining the data by reporting updates and corrections of datasets they use back to the Primary Custodian. The frequency of error reports and update submissions will help Primary Custodians prioritize areas in the database needing blanket updates, such as from orthophotos. Since Primary Custodians will be able to keep track of where they distribute their datasets, it will be possible to notify all users of corrected versions, new updates, or known problems with the dataset. The Primary Custodian role reinforces the notion of true custodianship of the data.
- Protections - As established in the pending legislation (amendments to FOIL, soon to be introduced in the Legislature by the Governor's Office as part of the Omnibus Technology Act), Cooperative members will have the confidence that their datasets will not be redistributed without their consent.
- Flexibility - Members wishing to release data into the public domain are free to do so with their own datasets. They cannot, however, release datasets owned by other members of the Cooperative.
- Standards - All members of the Cooperative will agree to comply over time with standards for metadata, data exchange formats, and other characteristics, as developed and endorsed by the GIS Coordinating Body.
- Ability to Charge Fees for Commercial Use - Pending passage of the amended FOIL legislation, Primary Custodians will have the ability, at their option, of charging right-to-use fees not to exceed fair market value, for commercial uses of their datasets.
How is it being implemented?
The NYS GIS Coordinating Body has endorsed the GIS Cooperative concept. Two of the Working Groups have developed materials needed for implementation. The Legal Working Group has created the standard license, called the NYS GIS Cooperative Data Sharing Agreement, which has also been reviewed and approved by the attorney for the Office For Technology, as well as the State Comptroller's office and the Attorney General's office. The Data Sharing Agreement has recently been sent out to all NYS agencies to sign.
The Data Coordination Working Group developed an overall GIS Data Sharing Policy for GIS, which has been accepted by the GIS Coordinating Body and issued as new state policy by Director of State Operations, Jim Natoli, as Technology Policy 97-6, on July 17, 1997. This policy directs that the NYS GIS Data Sharing Cooperative be established and further directs all NYS agencies to join the Cooperative. Local governments are encouraged to join.
The timetable for implementing the Cooperative includes an initial period in the Fall of 1997 for state agencies to sign the Data Sharing Agreement, thereby joining the Cooperative. Agencies have been asked to sign the Agreement by October 15, 1997. They are also being asked to create metadata describing their GIS datasets, for inclusion on the NYS GIS Clearinghouse website, hosted by the NYS Library. The Clearinghouse will serve as the information gateway for the Cooperative. The second phase of implementation will commence in January, 1998, when local governments will be asked to join the Cooperative. This phase is expected to be concluded by the end of June, 1998.