Proposed Internet Integration Business Process



Summary

The focus of the data integration process is facilitate accurate and timely data integration to the Framework database while minimizing the amount of time that has to be spent by the Integrator on validating data and resolving errors. A Cadastral Data Transfer Standard is being developed to allow consistent exchange of data over the Internet. Procedures are also being written to validate data by transaction, authority, jurisdiction, and history. Data Developers will have to write procedures to move data into the data transfer standard template and retrieve data from the production database into their own proprietary database. The Integrator will be able to spend more time on helping Developers become Providers.



An overview of the integration process starts with the check-out of data from the production framework database.



Roles

Data Developer: Develops proprietary data sets.



Data Provider: Develops metadata for proprietary data sets.

Develops conversion procedures from proprietary data sets to the data transfer standard.

Shares data maintained within jurisdiction and authority.



Data Integrator: Assists Providers with developing conversion procedures to the data transfer standard.

Verifies data validation after data is checked in.

Notifies Providers of integration errors and conflict problems.



Data Steward: Reviews metadata for proprietary data sets.

Determines authority and jurisdiction of Provider for integrating data into the Framework database.

Resolves data conflicts.



Database Administrator: Maintains the Framework database.

Backs up the Framework database.

Recovers the Framework database from the back-up.



Data User: Obtains data from the Framework database.



Data Development

Data is developed by federal, state, county, city, tribal, and private organizations in the normal operations of their proprietary databases. A Data Developer should indicate an interest in providing data for integration into the Framework. Framework Data Stewards must evaluate the extent to which any proprietary data set may be integrated. To make this evaluation, metadata for the proprietary data set to be integrated should be developed by the Data Developer. The Data Integrator will provide information on how to put proprietary data into the data transfer standard which can be integrated into the Framework database. The Data Steward must also determine the authority and jurisdiction of the Data Developer. The Data Developer must write procedures based on Data Integrator input to convert their proprietary data into the transfer standard. The Data Developer is then ready to integrate their data using the data maintenance process. The data transfer standard should be the same as the data export format used for distributing data over the Internet. Encourage data development contracts with surveyors to include contribution to framework.



Business Process

Developer: Notifies the Data Integrator that there is data which may be included in the Framework database, and provides metadata.

Integrator: Provides information and recommendations on how to convert over to the Cadastral Data Transfer Standard from the proprietary data set.

Developer: Develops routines to convert/format data to the Cadastral Data Transfer Standard.



Framework Data Sets

Surveys Steward

Legal Description Areas

Boundary Lines

Corner Points

Geodetic Points

Land Records Steward

Parcel Transactions

Record Documents

Agency Steward

Governmental Units

Administrative Areas



Provider Certification

Data Developers must be certified for their authority and jurisdiction to become Data Providers. The authority determines which data sets may be updated. The jurisdiction determines the geographical extent of the data sets which may be updated. Data Stewards will have to make the determination of the extent of data which will be accepted by the Framework database on a regular basis. This will allow automation of the data maintenance and integration processes. Partners will be notified of Provider Certifications.



Business Process

Integrator: Passes on Developer interest and metadata to the Data Stewards.

Steward: Decides to accept data from the Developer.

Determines the authority and jurisdiction of the Provider.

Notify partners of Provider Certification

Partners Review provider certification for accuracy.





Jurisdictions are in terms of legal area descriptions.

Federal

State

County

City

Tribe

Own land holdings

Entrusted land holdings



Certification Requirements

Partner Agreement

Authority

Staff qualifications

Jurisdiction

Staff Qualifications

New information accepted without qualifications.

Professional Land Surveyor are required for the authority to update corner points and geodetic points.

Providers may update their own data allowed without qualifications.



Data Maintenance

Data maintenance can be done by Data Providers who have developed their proprietary databases based on the data transfer standard, or who have developed routines to format their data in the data transfer standard. Multiple transfer data standards should be developed including shape file, export coverage, and AutoCAD.



Maintenance will be on a 30 day update cycle. Data providers will have until the end of the cycle to complete their edits. The check out would automatically be canceled after 30 days. The data provider would have to check out the data set again to keep the check-out active. Data older than current would be retired on check-in. Longer maintenance cycles may be needed.



Business Process

Provider: Checks out a data set.

Framework: Locks the checked out area.

Notifies other data providers about the check-out.

Provider: Makes data set updates.

Must include data set metadata.

Checks in data set to staging area before the end of the cycle.

Notifies data integrator.

Framework: Checks the update cycle daily and cancels all data set check-outs that have expired

Unlocks all checked out areas.





Data Integration (Self Integration)

Validate the current update sets. Hierarchical approach to check-in and validation of features. It may be possible to fully automate acceptance of land record updates. Check plots and reports are specific to each data set. In self integration, the data provider would contact the data provider who is the current owner of dat in the database. The ownership information would come form the audit fields which show who updated a file row last and when it was last updated. The two data providers with conflicting data would have to decide which data should be retired and which should be included in the database.



Business Process

Framework: Validates the updated data set.

Creates check plots for legal description, boundary, and corner transactions.

Notifies Provider and previous Provider of data conflicts.

Provider: Contacts previous Provider to discuss conflict resolution.

Contacts Steward to arbitrate if necessary.

Steward: Arbitrates data conflicts if necessary.

Assigns temporary access keys to check conflict data in.

Provider: Marks data with override accept or decline.

Checks data in with the Steward key.

Integrator: Approves the data set check-in.

Cuts out any portion that has a problem and checks in remainder.

Framework: Retires old records to the off-line database.

Inserts new records to the Framework database.

Unlocks the checked in area.

Notifies users of the check-in.



Data Integration

Validate the current update sets. Hierarchical approach to check-in and validation of features. It may be possible to fully automate acceptance of land record updates. Updates to legal area descriptions, boundaries and corners require integrator review. Check plots and reports are specific to each data set.



Business Process

Framework: Validates the updated data set.

Prints check plots for legal description, boundary, and corner transactions.

Integrator: Approves the data set check-in.

Cuts out any portion that has a problem and checks in remainder.

If any integration or conflict problems exist, then extends the check-out.

Framework: Retires old records to the off-line database.

Inserts new records to the Framework database.

Unlocks the checked in area.



Data Integration Problem Resolution

Data integration problems occur where the data set that a Data Provider is checking in will not fit into the current data schema. Something is wrong with the Provider's data conversion routine, or the transfer standard has changed.



Business Process

Integrator: If any integration or conflict problems exist, then extends the check-out.

Notifies the Data Provider of any data set integration problems with passing standards.

Provider: Makes updates in the data set to pass standards.

Checks the data set back in.



Data Conflict Resolution

Data conflicts occur where the data set that the Data Provider is checking in is does not logically fit back into the Framework database. Examples include updates that are older than the most recent update currently in the Framework database, data sets which are outside the area of jurisdiction of the Data Provider, or data sets that are outside of the authority of the data provider. The Data Integrator must work with the Data Provider and other Providers/Data Stewards to resolve the data conflicts. Retire features that are not accepted as current to the historical database.



Business Process

Integrator: If any integration or conflict problems exist, then extends the check-out.

Notifies the Data Provider of any data set conflict problems.

Works with the Data Provider and others to resolve data conflicts.

Provider: Explains the significance of providing data outside of the expected allowed range of values.

Steward: Determines whether the update is allowable, must be retired, or must be changed before the update will be allowed.





Conflict Situations

Accepted corner challenged.

Older data to supercede newer.



Data Backup and Recovery

The Framework Database Administrator will incrementally back up the Framework database on a transaction basis, fully back up the database on a monthly basis, and test recovery of the back-up every 6 months.



Administrator: Backs up the Framework database.

Tests the recovery of the Framework database.

Recovers the Framework database if it goes down.



Cadastral Data Transfer Standard and Framework Database Change Control

Once formalized, the Washington Cadastral Data Transfer Standard and the Framework database will not change without review by the Project Partners. There must be a sound business or technical need to change the standard. A business need might be additional attributes necessary to conduct business between partners; a technical need might be changes in the GIS vendor's software or file configuration. Change proposals will be published on the partner web page and distributed by e-mail. Change reviews will be allowed 30 days. The change may also be reviewed by the Technical Coordination Group once other projects are developed to a point where they are relying on cadastral data. Coordination of framework projects schedules will help determine when that transition will happen.



Partner: Proposes a change in the data transfer standard.

Integrator: Distributes the change request to other Partners and/or Technical Coordination Task Group for review.

Partners: Approve/disapprove the change.