Contracts Outline New York by wcd64786


Contracts Outline New York document sample

More Info
									   GIS Mapping of the New York
        City Sewer System
 Diagram
 Geodatabase                Design
 Topic       Outline
Entering spatial/non-spatial data - Brief Overview of Baker's process
  (scanning, digitizing and scrubbing)

Linking spatial and non-spatial data - Indexing Application, Sewer Database
   design (database structure, Feature Class, table attribute, relationship)

Data validation - QA process (Automated and detailed feature inspection) .
Entering spatial data
Types of DEP source Document
• As - built
• Contract
• Interceptor Plans
• Index Maps
• Emergency Contracts Drawings
• Infiltration and Inflow Maps (I/I maps)
• Connection Book
• Field Survey

Types of Non - DEP source document
• Department of Transportation
• Bridge and Port Authority
• NYC Orthophotos
• Sanbon - Property Maps
                        Methods of Data Inputs

   Scanning
    The process of translating hardcopy media into digital form that can be
    recognized by a computer. Create permanent records.
   Digitizing
    The method of data capture that involves the conversion of data in
    hardcopy or raster forms into Digital vector formats.
    Digitizing produces features (pipes and manholes) used in a GIS.
          * Features are digitized relative to the NYCMAP with 1 foot horizontal
          * Precision : Offset value (distance btw feature and the curb line).
                        Pipe Length

    Scanning differs from digitizing in that the entire pages of data or map
    sheet are captured all at once
   Geo-referencing
    The process of defining the position of a geographic object relative to a
    standard reference. Images with spatial data (I/I maps) were geo-
   Field Survey
                     Entering non-spatial data

   Scrubbing
    The act of cleaning a surface by rubbing it with a brush and soap and
    water   

    A meticulous process of reviewing and highlighting relevant attribute
    information for inclusion in the database.
          * Cleaned for inconsistency
          * Information retrieved are used to populate and link to their
            respective feature.
          * Standardization of domain and ranges (spatial and attribute)
                  Linking spatial and non-spatial data

   Indexing
    Creating a database of information for easy search and retrieval
    Indexing aid in filtering through a huge amount of information.
    *   Document Indexing
     A procedure that associates a source document with its respective street centerline
    *   Feature Indexing
        A procedure that associates a source document with its respective feature or

Where multiply source have been linked the most recent document acts as the default
Document are stored using a directories or folders and subfolders by their borough
Relationships are established within the database. Relationship require ID.
          Quality Assurance Procedures
QA Workflow

    The QA process consists of several phases that are defined in the
    procedures below:

   Check of digital data delivery and hard copy supporting documents

   Automated database checks using QA software

   Context check of plots and Engineering Review within the delivery area.

   Feature inspection based on a randomly selected sample set using ANSI
                                                                                  4. Conduct
                             2. Conduct
                                                                                  Sample Feature
  1. Check Data                Automated
  Delivery Package               Checks                3. Conduct
                                                       Context Check &
                                                       Engineering Review
                                            YES                                               YES
                                 Accept?                                       Accept?

                                       NO                                YES

                           6. Reject delivery                                            5. Accept delivery

                      7. Conversion Issue Resolution and Communication

Figure 1: QA Flow Chart
Quality Assurance Data Validation Techniques

The QA process ensures that database design specifications are adhered to
during conversion. The automated routine runs checks for data
completeness, spatial inaccuracies, context and attribution errors, scheme
and data structure and connectivity error. The process is divided into three
broad groupings – Automated Checks, Context Checks/ Engineering Review
and Feature Inspections Checks. The compliance level for automated checks
is 100% while the compliance level for feature inspection is 95%.

Automated Checks

Automated checks consists of three types of data evaluations—object
design, geometry, and connectivity checks. Table 1 summarizes the
Automated Validation Checks. The Automated application runs in batch
mode with pre-configured tables controlling the parameters. Most feature
characteristics and attributes can be checked through automated routines.
Object Design
    Review all features to ensure that all attribute values match domain tables
    Review all features to ensure that all mandatory values are populated and only
     permissible null values are populated in the database
    All unique values such as Facility ID are in fact unique both in a single delivery,
     as well as in the entire database
    Compare the attributes with the various area polygons such as Grid_no, etc., to
     determine if the attributes are correctly populated
    Check facilities with assigned linked annotation to assure they have at least one
     annotation feature linked to them.

Geometry test
   Review all junction features with elevation to ensure that the To and From
    attribution are correctly populated per pipeline
   Review all field-surveyed points in the Geodatabase located in the x,y
    coordinates captured in the field
   Review all features to identify any features that occupy the same x, y
    coordinate (i.e. duplication of geometry).

Connectivity Test.
The connectivity tests will:
     Validate that all features meet the topology business rules (i.e., identify all
      floating junction features and/or edge features that are not terminated with a
      coded junction feature).
     Conduct an audit and generate a report of all junction features and list features
      that have 0, 1, 2, and >2 edges connected to them.
     Per each junction type, generate a list of features that have 0, 1, 2, or > 2
      edges connected providing the ability to individually review each feature.
  Data Structure and schema          Type of Validation               Description                 Purpose/Notes

Spatial Attribute Check         Grid No, Maintenance Yard,   Features are validated          Quality Assurance
                                Drainage District,           based on there spatial          Layers
                                Treatment Plant              location within the

Document Link and Feature                                    Verifies that the each
Linked Annotation Check                                      feature is linked to a source
                                                             document and its related
                                                             piece of annotation.

Domain Validation Check:        *Variation between           Test for changes in defined     Ensure consistency btw
                                deliveries                   domain set. Detect attribute    deliveries
                                *Attribute Domain            values that fall outside of
                                                             the legal values.

Attribute Name Check            Attribute Name Check         Verifies attributes/field       An error with a coded
                                                             consistency between             value attribute error
                                                             deliveries.                     could point to an
                                                                                             uncontrolled editing
                                                                                             environment. Example,
                                                                                             digitized features
                                                                                             should have a value of
                                                                                             ―Active‖ for the
                                                                                             Operational_State field.

Null and Uniqueness Value       Attribute/fields             The nulls test checks for       Null values for an
Checks                                                       unpopulated values in an        attribute may signal an
                                                             attribute. The uniqueness       uncontrolled data
                                                             test seeks out duplicate        processing procedure.
                                                             values for an attribute field   Unique values for an
                                                             that is designed to be          attribute are critical for
                                                             unique, such as Facility-IDs.   referential integrity,
                                                                                             especially for primary
                                                                                             key attributes.

    These tests are critical for sewer dataset because invalid values can cause symbology and analysis errors..
Context and Attribute         Type of Validation          Description               Purpose/Notes
Error Descriptions

                          Flow                     Identify instances where   Review all junction
To and From Pipeline                               the From[Inv_Elevation     features with elevation
Invert Elevation Check                             of a pipe is lower than    to ensure that the To
                                                   the To_Inv_Elevation       and From attribution are
                                                                              correctly populated per

Manhole has two out       Multiple Exit            Identifies instance were   Check for only usually
sewer line                                         manhole have more than     configuration
                                                   one exit sewer on it
                                                   downstream section.

Isolated Junction Check   Connectivity             Identifies isolated        Ensure that all features
(Network Junction,                                 features e.g. manhole      are within the network
Simple Junction)                                   and sewer structure
                                                   nodes that have no
                                                   pipes connected to

Elevation Check at        Flow                     Compares incoming pipe     Ensure that incoming
Manhole                                            invert elevations to       pipes have higher or
                                                   outgoing elevations at     equal invert elevations
                                                   each manhole.              than outgoing pipes

No Outlet Sewer -         Connectivity             The Check identifies
                          (Dead End Check)         sewer sections that end
                                                   abruptly therefore
                                                   obstructing the
                                                   general effluent flow.
                          Context Checks
Check Plots

Reviews sewer configuration by comparing the geo-referenced I/I maps and
digitized sewers. This check is to make certain that no significant segments of
the sewer system are missing. Connectivity and flow tracing will support this
step. Other step include sewer length verification.

Engineering Plots

Baker Delivery Plots (Engineering Review) - performed by a team of DEP
Engineers/Staff. The check results in identification of configuration, attribute,
and annotation errors. Valid errors are entered in the QA application along
with other recorded errors

Infiltration and Inflow Maps (I/I Maps): Infiltration and Inflow maps (I/I)
were originally produced from as-built drawings. They cover 14 drainage
areas within New York City . The I/I maps show the following sewer
information: sewer type, size, shape, invert elevations, flow direction, and
year of as-built. Because they have not been maintained, they are
considered incomplete, and are generally inaccurate. They should be used
for reference only as a secondary source to be used for an understanding
of the sewer system within an area.
              Sample Feature Inspection
This process involves a detailed inspection on a selected set of
feature. The selection is chosen randomly and based on statistical
principals that establish an overall confidence on the quality of the
data delivered.

The ANSI test consists of four components:
 Calculate batch size,

 Select general inspection level,

 Calculate sample size and make a random selection from the
   batch size
 Summing the errors for acceptance/rejection of the batch.

Table 2 shows how the ANSI standard (American National Standard
Institute) is used in the Sewer Project.
     Table 2: ANSI Sample Size Code Letters14

             Lot or batch size                                     Special inspection levels                                  General inspection levels
                                                       S-1            S-2           S-3                    S-4              I            II             III
             2        to        8                       A              A              A                     A               A            A               B
             9        to        15                      A              A              A                     A               A            B               C
            16        to        25                      A              A              B                     B               B            C               D

            26        to        50                      A                 B                B                 C              C             D              E
            51        to        90                      B                 B                C                 C              C             E              F
            91        to        150                     B                 B                C                 D              D             F              G

          151         to        280                     B                 C                D                 E              E             G              H
          281         to        500                     B                 C                D                 E              F             H              J
          501         to        1200                    C                 C                E                 F              G             J              K

        1201          to        3200                    C                 D                E                 G              H             K              L
        3201          to        10000                   C                 D                F                 G              J             L              M
       10001          to        35000                   C                 D                F                 H              K             M              N

      35001           to        150000                  D                 E                G                 J              L             N              P
     150001           to        500000                  D                 E                G                 J              M             P              Q
     500001          and        over                    D                 E                H                 K              N             Q              R

Table is a copy of the American National Standards Institute: Sampling Procedures and Tables for Inspection by Attributes
Prepared by the American Society for Quality Control Standards Committee for the American National Standards Institute Committee
Z-1 on Quality Assurance
ANSI/ASQC Z1.4-1993
   Table 3. Single Sampling Plans for Normal Inspection

                                      TABLE Sample Single ers
                                     Table I -A-2: size code let tsampling plans for normal inspection (Master table)

Sampling                                     Acceptable Quality Levels (normal inspection)
size                0.010 0.015
             Sampling                0.025    0.040    0.065   0.10    0.15   0.25     0.40 0.65     1.0     1.5    2.5       4.0   6.5    10    15      25    40
code         size       AcRe AcRe    AcRe    AcRe     AcRe     AcRe AcRe AcRe AcRe AcRe AcRe AcRe AcRe AcRe AcRe AcRe AcRe AcRe AcRe
         A          2                                                                                                                01                 12    23
         B          3                                                                                                         01                 12     23    34
         C          5                                                                                              01                     12    23      34    56

         D          8                                                                                         01                     12    23    34     56    78
         E       13                                                                                   01                      12     23    34    56     78    10 11
         F       20                                                                           01                    12        23     34    56    78     10 11 14 15

         G       32                                                                  01                      12    23     34        56    78    10 11 14 15 21 22
         H       50                                                            01                     12      23    34        56     78   10 11 14 15 21 22
         J       80                                                     01                    12      23      34    56        78    10 11 14 15 21 22

         K     125                                             01                    12      23      34      56    78     10 11 14 15 21 22
         L     200                                    01                      12     23      34       56     78    10 11 14 15 21 22
         M     315                           01                         12     23      34     56      78     10 11 14 15 21 22

         N     500                     0 1                      12      23     34      56     78     10 11 14 15 21 22
         P     800           0 1                      12       23     34      56     78      10 11   14 15 21 22
         Q    1250 0 1                       12       23       34     56      78     10 11 14 15     21 22

         R    2000                     12    23         34      56      78    10 11 14 15 21 22

Table is a portion of a copy of the American National Standard: Sampling Procedures and Tables for Inspection by Attributes
Prepared by American Society for Quality Control Sandards Committee for American national Standards Committee
Z-1 on Quality Assurance
ANSI/ASQC Z1.4-1993
     Table 4: ANSI Interpolated Number of Errors Allowed per Sample Size for
                   95% Acceptance and Rejection Criteria

Sample Size Code   Sample     Interpolated Reject
    Letter          Size           Threshold          AC 95%         RE 95%
       A                  2                      1              0               1
       B                  3                      1              0               1
       C                  5                      1              0               1
       D                  8                    1.4              0               1
       E                 13                    2.4              1               2
       F                 20                    3.4              2               3
       G                 32                    4.8              4               5
       H                 50                    6.8              6               7
       J                 80                    9.2              8               9
       K                125                   10.6             12              13
       L                200                   17.8             17              18
       M                315                    22              21              22
       N                500                    22              NA              NA
       P                800                    22              NA              NA
       Q               1250                    22              NA              NA
       R               2000                    22              NA              NA
       S               3150                    22              NA              NA
Table 5: Feature Count and Sample Size

                                                 Table I             Table II-A
                             Total       Batch    Code     Sample    Allowable
                            Features    Number   Letter     Size       Errors
 Sewers, Force Mains            3,460     G1       L             200          17
 Manholes                        2785     M1       K             125          12
 Sewer Structures, Disch
 Pts, Pump Sta, Detention
 Systems                         345      P1       H             50           6
                                                                375          35
  Results of Random Sample Check

             Batch               (A)          Feature       Total     Attribute   (B)         (C)          Pass/Fail
                                 Total        Checked     Attribute         s     Weighted    Allowable
                                 Feature                   Errors     Checked     Attribute   Errors
                                 Errors                                           Errors

   G1 Sewer/Force Main               7           200        222         4600         9.7          17         Pass

   M1 Manholes                       3           125         82         2000          5           12         Pass

   P1     Other         Point        3           50          8          163          2.5          6          Pass

                                                               Not to                              Weighted

                                Feature    Sample      Sample           Feature Attribute Weight
                                                               Exceed                              Attribute
                                 Count      Letter      Size             Errors   Errors   Factor
                                                               Errors                               Errors
Manholes             M1            4848      L             200     17.8         0   39.96     0.06       2.50
Sewers/FM            S1            6570      L             200     17.8        11   33.98     0.05       1.62
Point Ftrs           P1            1125      J              80      9.2         0        0    0.15       0.00

Manholes             M2            5137      L             200        17.8           0        42.92       0.06         2.68
Sewers/FM            S2            7390      L             200        17.8           8        47.95       0.05         2.28
Point Ftrs           P2            1248      K             125        12.6           0            4       0.13         0.51
  Table 6: Computation of Attribute Weights

             A                  B              C              D               E               F

                            Attributes    Number of                        Average         Attribute
                               to be      features in        Total        attributes        weight
                            checked          batch        attributes     per feature     (1/average)
                                                             B*C              D/C             1/E
Sewer/Force Main
ss_pressurizedmain                   18               4           72
ss_gravitymain                       21            3456        72576
                Group G                            3460        72648           21.00              0.05

ss_manhole                           16            2785        44560
                 Group M                           2785        44560           16.00              0.06

Other Point Features
generic_node                          2             292            584
pump_station                         10               3             30
ss_detention_system                   9               1              9
ss_dischargepoint                    17              12            204
ss_sewer_structure                    8              38            304
                  Group P                           346           1131            3.27            0.31
           Data validation - QA process

Context Check
Baker Delivery Plots
Feature Inspection
Customized Tool for inspection
Feature Dialog
Attribute Dialog
Sample Tools

To top