Thursday, October 22, 2015

Data Gathering, Interoperability and Projections

Introduction


This is the first step in our analysis of sand mines in Wisconsin.  For this portion of the assignment I will be gathering and preparing data for sand mine analysis.  To accomplish this task, I first will become proficient downloading data from a multitude of sources across the Internet.  Secondly, I will become efficient at importing and compiling the data into a well organized geodatabase.  The final step will have me projecting data from different sources into the same projection and clipping the data to my area of analysis.

Trempealeau County, Wisconsin (Fig. 1) is the the main focus area for our analysis.  All of the data I will download and prepare will be for Trempealeau County.

(Fig. 1) Trempealeau County is located in the west central portion of Wisconsin (Map Source)



Methods

The initial objective to my assignment was to download data from the following 5 different state and federal agencies:
  • US Department of Transportation (USDT).
    • Railway Network Data

  • US Geological Survey (USGS)
    • National Land Cover Data 2011
    • National Elevation Dataset

  • US Department of Agriculture (USDA)
    • Cropland Cover Data 

  • Trempealeau County Land Records
    • Entire Geodatabase from Trempealeau County
  • Natural Resources Conservation Service (NRCS) sub-organization of USDA
    • SSURGO Web Soil Survey Data

I navigated to the appropriate websites using links provided to me by my professor (see sources below).  The focus of this study is with in Trempealeau County, so when possible I downloaded only the data for the county.  Those images or feature classes downloaded will be clipped or extracted by mask (raster images) to the Trempealeau County boundary.

Each data set with the exception of the USDA was downloaded directly from the website.  The USDA data was sent to me in a email attachment.  All of the data sets were delivered in a .zip file format.  From the .zip files I extracted the actual data and placed the data in organized individual folders.

Steps in preparation for each dataset

USDT

From the USDT I downloaded the polyline, Railway Network 2015 data  for the entire United States.  After I downloaded the entire geodatabase from Trempealeau County, I clipped the railroad network to only those rail lines which fell within Trempealeau county.

USGS

From the USGS I downloaded the following 3 different datasets: National Land Cover Database 2011 (NCL),  National Elevation Dataset (2 different coverage areas).  Using the box/point tool within the National Map Viewer I made a box which enclosed all of Trempealeau County.  After setting the box as the area of interest (AOI) I selected each data set separately and downloaded them to my computer.

USDA

From the USDA I downloaded the Cropland Land Cover data set for the state of Wisconsin.  As stated before the data has to be processed and emailed to you for downloading.

Treampealeau County Land Records

From Treamplealeau County I downloaded the entire geodatabase from their land records divsion.

NRCS

From the NRCS I downloaded the SSURGO web soil survey database for Trempealeau County.  The SSURGO data is a very comprehesive dataset which takes a little more work to properly display need information.  I will not be going through the step by step tutuorial, just understand it is more than just a drop in shapefile/feature class with all the information.

Organization & Transformations

Before I organized everything into one nice neat package I had to combine my Digital Elevation Models (DEM) which I downloaded from the USGS.  The southern tip of Trempealeau County was in one image and the majority of the county was in another.  Using the Mosaic to New Raster tool within ArcMap I combined these two images to form one new raster image.

The next step in the process is to create a location to store all of our data.  For this instance I will be using the Trempealeau County Database as my location.  The database is very well organized already and will serve as a great base to build from.

However, I cannot just throw all of my data in the geodatabase.  All of the data I have retrieved is in varying projections and coverage areas.  I want to focus all of this data to the extent of Trempealeau County.  To assure accurate results in my analysis I want to have all of the data in the same projection.

After analyzing the Trempealeau County database I found they were using, NAD 1983 HARN WISCRS Trempealeau County_Feet. (NHWTCF)  This projection is the most accurate when analyzing data across Trempealeau county.

There are multiple approaches to accomplishing the projection and clipping of the data.  Our class had to use Python scripting to write a geoprocessing script to achieve the desired end result.  For this portion we were only dealing with the raster images we have extracted from our data.  To preform these objectives I will be using the Project tool and Extract by Mask (Clip) tool with in ArcMap via PyScripter.  The benefit of using a script, is it allows us to run the same tool across the 3 raster images at once.  If I was to process this by hand in ArcMap I would have to run the Project tool and the Extract by Mask (Clip) 3 times individually.  Though it may not seem like a great time savings in this instance, if I had 100 images to project and clip, it would save me a ton of time.  The exact Python script can be seen via my Python tab at the top of my blog or by click here.

The final step in this exercise was to make a map of all three of the raster images from the previous steps. (Fig. 2)

(Fig. 2) Comparison of the raster images with locator map.

Data Accuracy

Data accuracy is important to the quality and accuracy of your analysis.  Evaluating data accuracy can be a challenge.  Information about the data accuracy should be stored in the Metadata file which is usually provided when you download the data.  However, as Dr. C. Hupy has stated there is no "data police" to assure the information is provided.  Even if the information is provided, the chances of it being in the proper format is even less likely.  Industry standards have started to improve this issue with Metadata but it will take time for all of the available data to catch up and revise or create proper Metadata.

For our assignment we were asked to look at 7 different aspects of data accuracy.
  • Scale
  • Effective resolution
  • Minimum mapping unit
  • Planimetric Coordinate Accuracy
  • Lineage
  • Temporal Accuracy
  • Attribute accuracy
I examined the metadata for hours in addition to searching the websites the data was retrieved from for additional information.  In the end, I could not find many of the items I was requested to retrieve.  I created a table with the information I found to display the accuracy differences between datasets (Fig. 3).

(Fig. 3) Compilation of data accuracy components for downloaded data.
 
Conclusion

This exercise was a great introduction to the available data sources across different governmental agencies.  Being able to see the different formats the data is delivered in gives me a additional list of things to prepare for when starting a project.  Our introduction to Python adds another way to run geoprocessing tools within ArcMap while saving time.  The data accuracy examination is a great eye opener to understand that you really don't always know everything about your data.  I would not make a life or death decision based on this data due to lack of accuracy information available.


Sources

Cropland Cover. In United States Department of Agriculture. Retrieved October 14, 2015. http://datagateway.nrcs.usda.gov/

Trempealeau County Land Records. Retrieved October 14, 2015, from http://www.tremplocounty.com/tchome/landrecords/


United States Department of Transportation. Retrieved from http://www.rita.dot.gov/bts/sites/rita.dot.gov.bts/files/publications/national_transportation_atlas_database/index.html

United States Geological Survey. Retrieved October 12, 2015, from http://nationalmap.gov/about.html

Web Soil Survey. In United States Department of Agriculture. Retrieved October 14, 2015, from http://websoilsurvey.sc.egov.usda.gov/App/HomePage.htm

No comments:

Post a Comment