Raster Loading Instructions
Raster Data will be loaded in the GeoPortal instance of SDE (VM68)
The format for most raster loads will be jpeg2000 or .jp2.
Using ArcCatalog
Convert the raster into the desired format. Use JPEG2000, at 75%. Rasters cannot be converted
to other formats once they have been loaded. Also, if the raster is being compressed, a smaller
file is being loaded to the server. Large rasters will often fail to load, check the loading logs to
monitor for errors. Preprocessing has two steps; check for a world file and convert the raster
format.
A. Creating .tfw (world files)
If you know the pixel size and coordinates of a large set of tif files, you can quickly generate a
set of world files to go with them. In a spreadsheet, create a sheet that has Name, X-pixel size,
Y-pixel size, Longitude, and Latitude in decimal degrees. Save this as a .csv to serve as input to
auto_georef.exe. Run auto_georef and put the .csv file name in the first box and an output
directory in the second and click "write files". The .tfw file will be written to the directory.
World files for tifs, tfw, are identical to those for jpgs, jpw. They have the form
5.51451779168098E-07
0
0
-5.5145177916482861e-007
78.4634327622498
17.4016286081658
Where the longitude and latitude are the upper left corner of the image (at least in the Northern,
eastern quadrant).
B. Converting the format.
Navigate to the file in ArcCatalog. Right-click and choose "Export --> Raster to Different
Format...". Enter the appropriate extension for the new format into the "Output Raster Dataset"
text box. The extension for JPEG2000 is '.jp2' . Click 'OK'. ArcCatalog should create a copy of
your raster in the new format.
Once the raster is in the desired format, it may be imported into the Geoportal dataset. Right-
click on the Geodata Repository ( "GISOWNER01@gisportal@VM68.atech.tufts.edu.sde" ) and
C. Import the Raster.
Select "Import --> Raster Datasets...". Navigate to find the input raster. Click the
"Environments..." button at the bottom of the dialog window. Click "Raster Storage Settings"
and scroll down to "Compression". Select the appropriate compression ( JPEG2000 ) and
compression quality (75). Click "OK", then "OK" again. The raster will be imported into the
Geodata Repository with the specified compression. If it is a single image and not a mosaic,
check calculate statistics, add colormap and build pyramids( as necessary).
Metadata must still be loaded into the metadata tables so that the raster is accessible to users.
Grant 'SELECT' privileges to SDE, GEOSERVER2x by right-clicking on the layer in the
Geodata Repository and selecting "Privileges". Type "Geoserver2x" in the text box and click the
check-box next to "SELECT. The user may query the selected object(s) data." before clicking
"OK". Repeat this for “SDE”
Using the Command Line
1. Create a list of the files to import
a. Create a text file listing all of the layers
i. Open a command prompt
ii.Change directory in the command prompt Type
cd M:\datasets\State\MA\MassGIS\Imagery\Orthos2008\15cm_mosaics
iii. Type "dir /M *.sid > layers.txt" (.sid is the file type, change this to .tiff or .jpg as
appropriate). This will create a text file called layers.txt in that directory that lists all of the
metadata files.
iv. Edit the text file or excel file so that you have an sderaster command for each layer
as Text
sderaster –o import –l ,image –f .jp2 -g -G -i
geodata -s sde:sqlserver:vm68.atech.tufts.edu -u gisowner01 -p >
C:\Logs\LoadLog.txt
sderaster –o import –l US_MA_MetrowestOrthos_2008 ,image –f
M:\datasets\State\MA\MassGIS\Imagery\Orthos2008\15cm_mosaics\metrowest1.jp2 –N
–c jp2 –g -G 102686 –L 0 -i geoportal -s sde:sqlserver:vm68.atech.tufts.edu -u
gisowner01 -p > C:\Logs\LoadLog.txt
( Building the pyramids and colormap after the load)
1a. When building Mosaic, after the first line,all the following lines should mosaic the
images to the first image
sderaster -o mosaic -f metrowest2.jp2 -l US_MA_MetrowestOrthos_2008,image -v 1 -N -L 0
-Align -i geoportal -s sde:sqlserver:vm68.atech.tufts.edu -u gisowner01 -p >
C:\Logs\LoadLog.txt
If you use Excel to create the script save as txt
c. Change the .txt to.bat extension (i.e. import.bat)
d. Run the file from the command line
2) Check for errors
Open the loadlog.txt file and search for errors where the file did not full load. Check
the expected record count to the final record count.
Load input features into Spatial Index Batch mode, click on check mark If
Spatial Grid 1 has a number other than 0 a spatial grid is present. Delete any
layers that have a spatial grid in place then hit ok to generate new spatial
indices for the layers.
3) Define the projection
a. Copy import.bat to project.bat
b. Note the projection ID for each layer
i. Open the metadata file for the layer
ii. Find the name of the projection
iii. Go to http://spatialreference.org/ref/esri/
iv. Find the appropriate projection and note the numeric code example ESRI 102686
or EPSG:26986: for Massachusetts data
c. Edit each line to the correct command to define the projection
sderaster -o alter -i geoportal -s sde:sqlserver:vm68.atech.tufts.edu -u GISOWNER01 -
p -l ,footprint -G 102686 >> C:\Logs\projectionsLog.txt
d. Run the file from the command prompt
4) Build Colormap( as necessary)
sderaster -o colormap -l US_MA_MetrowestOrthos_2008,image -L -1 -v 1 -i geoportal -s
sde:sqlserver:vm68.atech.tufts.edu -u gisowner01 -p > C:\Logs\LoadLog.txt
5) Builds pyramids( as necessary)
sderaster -o pyramid -l in_hyderabad_g45survey_1915,image -L -1 -v 1 -i geoportal -s
sde:sqlserver:vm68.atech.tufts.edu -u gisowner01 -p > C:\Logs\LoadLog.txt
6) Calculate Stats for processing
sderaster -o stats -l US_MA_MetrowestOrthos_2008,image -v 63 -i geoportal -s
sde:sqlserver:vm68.atech.tufts.-u gisowner01-p [pwd]
----------------------------------------------------------------
Usage: sderaster -o stats -l -v
[-d] [-L ]
[-i | ]
[-s ] [-D ]
[-u ] [-p ]
-d Deletes image statistics and histogram.
7) Grant appropriate permissions
a. You are granting select privileges to users GEOSERVER2X and SDE
b. Open the project.bat file in Notepad or some other text editor
c. Save as grant.bat
d. Edit the commands in the file to be grant commands
i. Change the sderaster commands to read each of these users
sdetable-o grant -i geoportal -s sde:sqlserver:vm68.atech.tufts.edu -u
GISOWNER01 -p -l ,footprint -A select -U sde >
C:\Logs\permissionsLog.txt
sdetable -o grant -i geoportal -s sde:sqlserver:vm68.atech.tufts.edu -u
GISOWNER01 -p -l ,footprint -A select -U geoserver2x >
C:\Logs\permissionsLog.txt
ii. Note that the -o argument changes to "grant
iii. Repeat all of the lines, substituting "-U sde" for "-U geoserver2x"
e. Run the batch file from the command prompt
8. Rename ( As necessary)
sdetable -o rename -t -T
[-i ] [-s ] [-D ]
[-u ] [-p ] [-N]
[-q]
sdetable -o rename -t in_hyderabad_G45survey_1915 -T in_hyderabad_g45survey_1915 -i 5150
-s mapmaker.mit.edu -u sde_data -p [pwd]
9) Import layer metadata
Following the appropriate Metadata steps either in ArcCatalog or GeoNetwork . If no metadata
exists, start with Step 1 Standard Metadata Creation. If metadata exists, start with Step 2 Batch
Metadata Loading
a. 1_Standard_ArcCatalog_Metadata_Creation
2_Batch_ArcCatalog_Metadata_Loading
3_Checking_ArcCatalog_Metadata_
b. 1_GeoNetwork_Metadata_Creation
2_Batch_GeoNetwork_Metadata
3_Checking_GeoNetwork_Metadata
c. Enter user name (gisowner01) and password in the dialog box, then click connect
e. Make sure the proper options are selected for "Raster dataset" and "Publicly Accessible"
10) Rebuild indices
a. Open index.sql from the desktop
Or by Command line
AddSpatialIndex_management {spatial_grid_2}
{spatial_grid_3}
AddSpatialIndex_management Somerville_Buildings 4.1 16.4 0
b. Run each command separately. You do not need to edit anything.
For more information:
ArcSDE Help 9.3: http://edndoc.esri.com/arcsde/9.3/
ArcSCE Command Reference: http://edndoc.esri.com/arcsde/9.1/admin_cmd_refs/sderaster.htm
ESRI Knowledgebase Articles: such as
http://support.esri.com/index.cfm?fa=knowledgebase.techarticles.articleShow&d=28110
ArcGIS Desktop Help 9.3: Image and raster data management :
http://webhelp.esri.com/arcgisdesktop/9.3/index.cfm?TopicName=About_raster_dataset_properti
es