ASTER DEM Production Procedures

Document Sample
ASTER DEM Production Procedures Powered By Docstoc
					A. Pulling ASTER scenes (individually or multiple) .... 2
B. Setting up a project in PCI ..................................................... 2
C. (Absolute DEM only) Importing GCPs: ......................... 3
E. Collecting Tie Points (TPs) ........................................................ 5
F.Creating epipolar images ............................................................. 6
G.Processing the DEM ....................................................................... 6
H. Editing the DEM................................................................................. 7
I. Geocoding the DEM ......................................................................... 9
J.Filtering the DEM .............................................................................. 10
K. Outputting the .hdf, .met, and .pdr files ........................... 10
L. Generate a Geotiff from the DEM ........................................ 11
M. Ftp files to polling directory and datapool. ................... 12
N. Inserting the completed DEM files manually ............. 12
O. Sending Failure Notices............................................................................ 13

      ASTER DEM Production Procedures

A. Pulling ASTER scenes (individually or multiple)

1. Find the granule ID(s) in the DEM database.
   Copy the granule ID(S) to an open notepad on your PC desktop. Save
   this Granule list as (name).txt.
2. Open Secure File Transfer. Open profile to e0ais03 AsterScenes. Open
   AsterScenes folder. Drag the name.txt file into the AsterScenes folder.
3. Open x-win to e0mos01. Pick and XDM host that has a low number of
   connections and use that one.
4. Login and open a window. ssh2 to e0ais03. cd to AsterScenes and do
   an ls to be sure your (name).txt file is there. vi the (name).txt file to see if
   the granule list looks correct. Exit the vi.
5. Type in EdcStageDap.csh OPS (name).txt This will pull the granules and
   put them into your home/AsterScenes directory on e0ais03.
6. Create project folders on your g: drive. Name these folder for the DEM
   order #’s in the DEM database. Highlight and drag the .hdf and .met files
   to their appropriate folders and delete from the AsterScenes folder.

B. Setting up a project in PCI

1. Start OrthoEngine
2. Click File -> new
3. Browse to the project folder on your PC and double click it.
4. Use the project folder number and a .prj extension in the File name:
   space, for example: 10031.prj. Hit enter or open.
5. In the Name slot, type #<project name> - <> -
   <Geographic Location of scene> Ex. #972 – ramsey – Ethiopia. The

   customer name can be found in the DEM database. The Geographic
   location of the scene can be determined by opening the project.met file in
   the project folder on your PC with notepad. Page down until you locate
   the GRingPointLatitude and GringPointLongitude. Consult maps to
   determine what country the scene lies in.
6. Rename the hdf file to <project folder number>.hdf ie: 1459.hdf
7. Select “Satellite Orbital Modeling”, (General High Resolution should be
   selected as a default option). Click Accept.
8. Set the Output Projection to (UTM).
9. Set the Output Pixel Spacing to 15m and press Enter (Output Line
   Spacing will automatically set to 15m). For an Absolute DEM, set the
   GCP projection based on the ODPR email (GCPCoordSys= “Long/Lat” or
   “UTM”, GCPDatum= “NAD83” or “NAD84”). For a relative DEM, click Set
   GCP Projection based on Output Projection. Click Accept.
10. Change Processing Step to Data Input.
11. Click the Read CD-ROM icon. Change CD Format to ASTER
12. Select the ASTER HDF file from your PC.
13. Select VNIR Band 3N
14. Type in a PCI file name to output to (g:\972\3N.pix) and hit Enter
15. Click Read
16. When 3N is imported, deselect VNIR Band 3N, Select VNIR Band 3B,
   type in a PCI file name for output (g:\972\3B.pix), hit Enter, click Read.
17. Save project.
18. Update the following fields in the record that corresponds to this project
   in the DEM database: date begun, produced by, and DEM status (in
   progress). Update your book to show that the project is in progress, and
   add customer name and location to the comments section.

C. (Absolute DEM only) Importing GCPs:

1. Login to the Lotus EDCAsterdem account and open the e-mail for the
   Absolute DEM.
2. Open notepad.
3. Copy and paste the text of the e-mail to notepad, and save the text file
   from notepad as request.txt in the project folder of the Absolute DEM
   on your PC.
4. Open a command prompt. Cd to the folder in step 3.
5. Parse request.txt to create GCP files for the N and B images and the .dpr
   file using a perl script as follows: request.txt
6. Change the Processing Step on Orthoengine to GCP/TP Collection
7. Click the “Open new or existing image” icon
8. Select the band 3N image, click “Quick open and close”
9. Click the “Import GCPs from file” icon on the main OrthoEngine bar.
10. Click “Select” GCP input file; select the .N file you created in Section A
    Step 5. g:\request.txt.N)
11. For format description, type IPLYXE, then add DDDDD, i.e.,
    IPLYXEDDDDD (If GCPs are submitted in UTM, the format description
    will be IPLXYEDDDDD). The Ds blank out extra information that we
    require the customer to put in and then don’t use!
12. Click “Apply Format”
13. Make sure the GCPs were extracted; click OK on the warning.
14. Click Accept
15. Repeat steps 7 through 12 for the 3B image and GCPs.
16. To check residuals, click on one image to make it the working image.
    Click on Collect GCPs manually on the toolbar. Click Bundle Update.
    Residuals should be <1 or as low as possible. Mark any GCPs with
    residuals that are obviously out of line as Checkpoints. Repeat for the
    second image. Note in Access database how many GCP’s were used.
17. Save project.

E. Collecting Tie Points (TPs)

  1. Change Orthoengine processing step to GCP/TP Collection.
  2. Click “Open new or existing image”, “quick open” for both images. If
     your image has a dropped line, go to instruction for fixing dropped
     sensor lines in the miscellaneous section.
  3. Click the “Automatically collect tie points” icon. Type an approximate
     elevation that is somewhere in the middle of your image elevations (1000
     to 1500 usually works).
  4. Click “Start Auto Tie Point Matching” then “Yes” on the question box.
  5. Click OK on the Automatic Tie Point Collection report box, then close
     the Automatic Tie Point Collection window.
  6. Click on the “Collect tie points” icon.
  7. Turn off the “Auto Locate” feature.
  8. Click on the “Bundle Update” box (it will stay “on” for all of the TPs you
  9. Double check the tie points collected automatically for accuracy, then
     collect additional Tie Points to fill in a 3x3 grid utilizing the procedures that
  10. Pick an easily identifiable point on the working image, click on it and
     magnify it and then click on “Use as TP” on the image menu bar.
  11. Click on the same point on the reference image, and then click on “Use
     as    TP” on the image menu bar.
  12. On the Tie Point Collection window, Click Accept and then turn off
  13. Repeat steps 10-12 for as many TPs as you want to collect.
  14. Close the Tie Point Collection window and both image windows, and click
     File, then Save on the main menu bar.

F.Creating epipolar images

  1. Change Processing Step to “DEM from stereo”
  2. Click on “Create epipolar image”
  3. Highlight the left-looking (N) image in the left window, then the right-
     looking (B) image in the right window.
  4. Click Create, then OK on the pop-up box. Wait for epipolar images to be
     generated. Close the epipolar generation window.
  5. Check that the edges of the epipolar images are straight by clicking File
     -> Image view… and selecting the el_3N image. If there is a noticeable
     curve to the edges, add more tie points along the part that is curved.
     (Step E) and recreate the epipolar images, naming them el_3N2 and

  G.Processing the DEM

  1. Click on “Extract DEM automatically”.
  2. Highlight the line containing the epipolar images created in step F.
  3. Type in a minimum and maximum elevation to give the software a
     starting point. The minimum should be below any actual elevation on the
     images, and the maximum should be above any actual elevation on the
     images. Generally leave a 500-to-1000 meter cushion on either side.
     (Example: Minimum = 0, Maximum = 3000)
  4. Leave the default failure and background values - -100 and -150
     (unless working with imagery where elevations are below sea level, in
     which case you will lower them to values slightly below the minimum).
     Leave the Pixel spacing on 2, Fill Holes & Filter on Yes, and Score
     Channel on No.
  5. Set the DEM detail to High
  6. Browse to the project folder and Type in output file names for the DEM
     and the report. (i.e. 972Dem.pix, 972Dem.rpt).

7. Leave Extraction Start Time on Start now.
8. Click “Start DEM Extraction”, Click Yes in the “are you sure” box, and
    wait for the DEM to process. This will take approximately 45 minutes.
    When processing for a second or third time use Dem2.pix and Dem2.rpt,
    Dem3.pix and Dem3.rpt etc.

H. Editing the DEM

1. ***Warning***. There is no “undo” button in editing. Save often!! If you
    make a mistake, close without saving and reopen. The DEM will revert to
    it’s status the last time you saved.
2. Make sure the Processing Step is set to DEM From Stereo. Click on the
    Manually edit generated DEM icon. Navigate to the correct project
    folder (if necessary). Select the <project>dem.pix file, and click Open.
    click default, then “Load & Close”.
3. The DEM image box contains a toggle button that allows you to switch
    between image view and DEM view. Mask all water in your image first by
    executing the following steps. Water that touches the edge of the image
    will be masked and set to –150 (background). Inland Lakes will be
    masked and set to a value 1 meter below the lowest valid shoreline value.
    If there is a lot of water in the image or some lakes have already been set
    to –150 by modeling then the inland lakes can also be set to –150. If t he
    lake opens into water outside the DEM it is set at –150.
   Switch to Show image. Look for large lakes and rivers (generally > 4
    pixels for rivers). Place cursor over this feature, and switch back to Show
    DEM. Decide if the feature needs to be masked.
   For Lakes. Zoom in to a value of 4. Click Trace&Close , trace around
    the visible section of the lake and release the mouse button. Click
    Fill@Cursor and click inside the area you just traced. Continue to trace

    and fill the lake until it is all covered. Set the Fill Using Value to the
    appropriate number, (one value below the surrounding area) then click
    Fill Using Value. Toggle Show Mask to off, and inspect the DEM to see
    if your value matches well. If so, toggle Show Mask back to yes, then
    Clear Mask. You may reset the Value, click Fill Using Value, re-inspect
    the DEM, and clear the mask when you are satisfied.
   Click Save DEM back to file. Click “No” on the pop-up box as you do not
    wish to create a new channel.
   For Rivers. Place cursor on the river at an edge of the image. Zoom in
    to a value of 2. In the Bulldoze a Line section, set the Value: to –150 (or
    your background value if it is different), Change Bulldoze Line Width:
    to an appropriate value. Click on Bulldoze Using Value. Hold the left
    mouse button down while coloring a section of the river. Place the cursor
    at the front edge of the section you have bulldozed. Zoom out one level,
    then back in to 2. Click on Bulldoze Using Value and continue coloring
    until you reach the end of the river or the edge of the DEM.
   Click Save DEM back to file. Click “No” on the pop-up box as you do not
    wish to create a new channel.
   To mask failed areas. Set Fill Value to –150 (or your failure value if it’s
    different). Zoom in to 1 near a failed area. Click Polygon, click around
    the bad section, click Polygon again to close the polygon, click
    Fill@Cursor and click inside the polygon you just created. Click Fill
    Using Value, then Clear Mask. You may also use the trace feature as
    follows: Click Trace&Close , trace around the failed area and release
    the mouse button. Click Fill@Cursor and click inside the area you just
    traced. Click Fill Using Value, then Clear Mask. Repeat for all large
    failure areas in the DEM. Click Save DEM back to file. Click “No” on the
    pop-up box as you do not wish to create a new channel.
4. To mask and interpolate SMALL holes (generally <= 4 pixels high),
    use the following sequence of steps:

           Set Fill Value to –100 (or your failure value if it’s different) and
            toggle “Interpolate” to ON.
           Using the Polygon feature, create a polygon around a small
            section of the hole you wish to interpolate. Click Fill@Cursor, click
            inside the polygon, then click Fill Using Value, Apply Under
            Mask, and clear mask. Repeat the preceding step until you have
            interpolated the entire error area.
           Click Save DEM back to file. Click “No” on the pop-up box as you
            do not wish to create a new channel.
  5. Trim edges of DEM
           Create a polygon that frames the DEM. It should be approximately
            5 pixels inside the edges of the DEM. Set the Fill Using Value: to
            –150 (background). Click Fill@Cursor, click outside the polygon,
            then click Fill Using Value, and clear mask.
  6. Click Save DEM back to file. Click “No” on the pop-up box (if it appears)
     as you do not wish to create a new channel.

I. Geocoding the DEM

  1. Change Processing Step to “DEM from Stereo”
  2. Click on “Geocode Extracted DEM”
  3. Browse to the input DEM you just created. Accept the defaults for the
     DEM channel (2), failure and background values. For output file name,
     browse to your project folder and type g<project>dem.pix
     (i.e.g972Dem.pix) and click open.
  4. Set Pixel Spacing to 30.
  5. Change Interpolate Failure Pixels to No
  6. Click “Geocode DEM”
  7. The DEM has now been produced, edited, and geocoded (has
     geographics attached to each pixel).

J.Filtering the DEM

   1. Open PCI Geomatica.
   2. Click on the xpace icon.
   3. Open the task called FME (median filter).
   4. Click on the FILE tab and browse to the DEM you geocoded in step H.
   (From g: project folder). Values should be set as follows: DBIC = 1, DBOC =
   1,and FLSZ = 3,3
   5. Click Status, then Run
   6. Exit xpace.

K. Outputting the .hdf, .met, and .pdr files

   1. Navigate to g:\tools. Copy the request.txt, request.txt.N, and request.txt.B
   files and paste them into the current project directory. If this is an Absolute
   DEM – Do Not Copy over the files as they already exist in the project
   directory. Edit the request.txt file to contain the correct RequestID , Order ID
   and Input Granule ID(copy these from the ASTER DEM Database).
   2. On the main Orthoengine toolbar, Click “Options”; “Export”; “ASTER .met
   2. Select the .hdf file used to generate the DEM.
   3. Select the .dpr file listed.
   4. Select the .N text file
   5. Select the .B test file
   6. Select the geocoded DEM (g<project>dem.pix) Check if Database
      Channelbox says 3 X 3 median – If it says ortheon the Filtering the DEM
      step has been omitted. Go back and filter the DEM before outputting the
   7. Select the DEM Report that was generated when the DEM was created.

  8. Highlight the database channel (there should be only one for the
     geocoded DEM). This will cause the Elevation box to be filled in.
  9. Leave Edit Filter Type as the default: Median
  10. Select a quality rating (your own opinion)
  11. Select a DEM Type (relative or absolute)
  12. Copy and paste the export note from g:\tools into the Operational
     Comments box.
  13. Click on Create MET. Three files will be created in the output directory,
     an .hdf file (the DEM data), a .met file (metadata), and a .pdr file (product
     delivery record). Hit okay on message box – DEM 16-bit channel should
     be specified.

L. Generate a Geotiff from the DEM

  1. Click File, File Utility on OrthoEngine.
  2. Click File, File Open, and browse to the geocoded DEM.
  3. Highlight the geocoded Dem file and click OPEN
  4. Click File, Export To…
  5. Click select Destination File
  6. Navigate to the g:\output directory
  7. Paste the DEM .hdf filename with a .tif extension (ex.
     ASTER_DEM20020124151327.tif) in the File name: box. This may be
     done by starting to type in AST and the remainder should fill in. Change
     the ending file extension to .tif
  8. Click Save
  9. Highlight the file under Source Layer, click ADD
  10. Highlight the file under Destination Layer, click export
  11. Close file utility windows.

M. Ftp files to polling directory and datapool.

   1. Open a Command Prompt on your PC.
   2. Cd to the DEM output directory (g:\output).
   3. Type dem <project>.pdr. This script will ftp the .hdf, .met, and .pdr files
      to the appropriate directories on e0icg11 so that the polling ingest will
      take place. It will then ftp the geotiff file to e0dps01 – datapool. The
      script will also e-mail the customer.
   4. Move the .hdf, .met, .pdr and .tif files from the output directory into the
      project folder.
   5. Zip the project folder and delete the unzipped folder.
   6. Update the ACCESS Database and your book.

N. Inserting the completed DEM files manually

  1. Open ssh Secure File Transfer.
  2. Open profile to e0icg11 for the .pdr and the .met and the .hdf. If you need
      to get to datapool use e0dps01.
  3. Navigate to the directory that you need to be in. ie:
      /usr/ecs/OPS/CUSTOM/icl/e0icg11/data/pollASTERDEM/DATA for the
      .hdf and the .met files.
      /usr/ecs/OPS/CUSTOM/icl/e0icg11/data/pollASTERDEM/PDR for the .pdr
      files or /datapool/OPS/user/SAN/ASTT/AST14DEM_tif/Relative or
      Absolute (depending on the file) for the .tif file.
  4. Drag the appropriate file from the output directory to the correct profile in
      the Secure File Transfer.
  5. If the manual insert is due to the script having an error and not inserting
      the .met or the .hdf files, you will need to copy over whichever file did not
      insert to the DATA directory (see step 3 for the complete address). If the
      script shows that the .pdr was created you will have to rename the .pdr
      file before it will pull in the new .met or .hdf file. Navigate to the PDR

   directory (see step 3 for the complete address) and find the .pdr that was
   produced. Click on the pdr and rename it by adding a letter – or some
   other unique ending behind it. (13202.pdr would change to 13202a.pdr)
6. If the manual insert is due to a script error on the .tif file and it did not
   insert into datapool, you can copy the .tif file using the Secure File
   Transfer over to the datapool directory (see step 3 for the address).

O. Sending Failure Notices
1. Log into e0mss01. cd /home/aster/astod/OPS do an ls to view directory.
   Locate the (cancellation script). Do a ./ and copy
   and paste the filename in. Hit Enter. The first prompt is asking for the
   Request ID of the order you wish to cancel. Copy and paste the Request
   ID from the DEM order in the Access database. Hit Enter. The next
   prompt should be asking you to choose from the reasons for failure. Enter
   the appropriate cancellation code for the order you are canceling. The
   next prompt is asking if you would like to send the e-mail to the customer.
   The default is yes and you can answer no if for some reason you do not
   wish to notify the customer. The script should then tell you if the order
   was successfully updated. Sometimes the Request ID that is being
   cancelled has already been deleted from the pdps database and the script
   will try 15 times to send the e-mail before quitting. When the script has
   completed it will return you to the e0mss01 prompt. You can hit !! and the
   script name will be put in for you so that you can cancel another order.


Shared By: