COTS-Comments by akgame

VIEWS: 46 PAGES: 24

More Info
									Commenter and Clause/ Number Subclause

Paragraph Figure/ Table

Typ Comment e

Proposed Change

S. A. Klein

Doug Landol

Phil Scruggs

Sklein-007

5.6

Para 5.6.1.1

T

Unmodified COTS must be evaluated at the source code level to protect against the threats identified in 5.3.2.1 (A).

Delete Accept change as “Unmodified third-part of change party software is accepted not subject to elsewhere. code examination; however,” and replace it with “All third party software shall be subject to source code an d other examination to preclude the presence of trap doors, hard-coded passwords, vulnerabilities and other nondeliberate errors, deliberate errors allowing the introduction of malicious code, and malicious code of any kind, especially malicious code intended to trigger upon use of the software

I don't agree with the requirement for source code review. Delete all mentions. Furthermore, COTS should be treated no differently than any other software.

Do not accept the change. It is not practical or, in most cases, feasible to perform a line-byline review.

MercuriD50 078 (new)

7.13

T

Provision is made in the standard for update for COTS products releases, but there is no such provision for updating or decertifying nonCOTS voting system components if such have been revealed to be insecure.

System changes that have resulted from identification of insecure voting system components must be propagated to all systems currently deployed. (This might be more appropriate in the configuration management section, or a different section under maintenance.)

Lipsio-12

5.1.1

Para. 5

T

The treatment of COTS products contradicts section 5.1.2.2, “Elements of Security Outside of Vendor Control”.

Change “COTS product may” to “COTS products shall”. Mandate compliance with section 4.3.11 (“Previously developed or purchased software”) of IEEE Std 12281994, “IEEE Standard for Software Safety Plans”.

Systems are not either secure or insecure there are degrees of assurance along a continum. It is my understanding that each election official will be empowered to make a judgement. In practice you will find that sometime in the future a system about to be deployed will have a flaw that slightly increases the risk of not enforcing a security policy in some special circumstances. Whereas one election official may decide to not accept the voting system as secure, another may choose to perform a remedial Would need to see The section in cited IEEE question is a standard before discussion of how addressing the CC process proposed change. could be used in However, relevant the overall IEEE standards assessment of should generally voting machines. be applied where "may" seems fine. applicable in this Leave as is. standard because they reflect relevant IEEE expertise.

Covered by other changes or out of scope as administrative certification.

No to implement this change Should be left at the discression of the election officials.

No comment

Lipsio-14

5.1.1

Para. 7

T

There is implied a lack of testing in “COTS products require updates due to a detected security breach or vulnerability”; nothing that requires an update should pass testing.

Mandate that testing preclude any security breach or vulnerability; mandate compliance with section 4.3.11 (“Previously developed or purchased software”) of IEEE Std 12281994, “IEEE Standard for Software Safety Plans”. Mandate COTS be subject to the specifications of IEEE Std 1008™1987 (R1993), “IEEE Standard for Software Unit Testing”. Add reference to IEEE Std 982.1™-1988, “IEEE Standard Dictionary of Measures to Produce Reliable

Would need to see cited IEEE standard before addressing proposed change. However, relevant IEEE standards should generally be applied where applicable in this standard because they reflect relevant IEEE expertise.

Sounds reasonable Accept proposed (not familiar with change the standards mentioned.)

Lipsio-15

5.1.1

Para. 7

T

“The voting system vendor must provide a method to assess the impact of COTS updates on the voting system, as well as a method for providing notice and distribution of updates to purchasers” is inconsistent with IEEE Std 10121998.

Bring into Would need to see Don't know. conformance with cited IEEE Annex D (“V&V of standard before reusable addressing software“) of proposed change. IEEE Std 1012However, relevant 1998, “IEEE IEEE standards Standard for should generally Software be applied where Verification and applicable in this Validation”, standard because e.g., “Reusable they reflect software (in part relevant IEEE or whole) expertise. includes software from software libraries, custom software developed for other applications, legacy software, or commercial-offthe-shelf (COTS) software. The V&V tasks of Table 1 are applied to reusable software just as they are applied to newly developed

No comment

Simons - 002

5.1.1

the sentence that reads, "The security countermeasu res implemented by an IT system typically use functions of the underlying products and depend upon the correct operation of those products and their security functions."

G

This is a far too vague and does nothing to address the security issues.

Replace sentence with the following: "Underlying products, such as operating systems, database systems, firewalls, network devices, web browsers, smart cards, biometric devices, general purpose application components, libraries, and hardware platforms, that are crucial to the correct and secure operation of the entire system must be thoroughly tested. This includes COTS systems. In addition, there must be a line by

Accept change as part of change accepted elsewhere.

I believe the suggested rewrite goes too far. I do not think that all of the examples listed carry the same burden for enforcing the security policy. For example, web browers are just expected to perform as expected, whereas biometric devices are a core part of the identification system, and firewalls are relied upon to protect the system from modification etc. Remove all examples. Also the requirement for a code review is not supported elsewhere and is not currently support with a risk argument.

Do not accept change. It takes something that is too vague then caries the argument onto multiple tangents.

Sklein-056

5.1.3

All

T

Voter verified paper needs to be mandatory under certain circumstances

Simons - 017

5.1.3.4.2

the entire section

G

Add to the section created under comment SK4 above: A voter verified paper audit trail is mandatory for any system in which any of the following conditions is found: 1. Either the system software or any COTS used as either a system component or development tool, including compilers, libraries, and other tools, is too complex to clearly and thoroughly evaluate at the source code level to ensure absence of backdoors and other malicious code or means of There is no way Add the to adequately requirement that test against all all COTS used in possible bugs and any voting system malicious code in must be open COTS. source.

Accept change (it is mine).

Disagree. Sounds like an issue for the election official. I am not comfortable suggesting such a mandate. Many of the terms and phrases in the suggested rewrite (addition) are too subjective (too complex). Many requirements go beyond the assurance that is needed.

Do not accept change. This should be a county requirement not a standard.

Both open source and closed source are COTS. Both should be required to be fully examined at the source level, as recommended in the resolution to other comments. Otherwise, most experts agree that open source and closed source security are roughly equivalent.

Do NOT require Agree with S. that these be open Klein comment. source. I see no requirement no advantage of such a requirement. Resolution: Delete entire section. COTS should not be treated any differently than any other kind of software. The vulnerabilities mentioned in the section apply equally to an operating system that I develop myself as they do one I buy off the shelf.

Lipsio-80

5.1.3.6.5

E

COTS software was Eliminate “and already covered software” from in 5.1.1. the first paragraph and eliminate item “a”.

Reject change. Agree Section is explicitly focused on networking. Other sections cover software generally. This covers networking software. Already covered elsewhere

Agree with SAK

MercuriD50 - 5.6.1.1 064 (formerly mercuri-143)

Section

G

Concerns addressing use of COTS products need to be added.

COTS products, Covered by other especially changes. software libraries, are a vulnerable attack point and must be subject to risks assessment prior to use in voting products. Configuration management should include vendor updates and alerts when flaws are detected that could compromise election operations or cast ballot data integrity. Object code modules should be provided such that compiled versions of programs can be compared.

No not implement change - comment already addressed.

Alice - 001

5.6.1.1

T

"source code Delete this generated by COTS clause code development package and embedded in software modules for compilation or interpretation shall be provided in human readable form" Some newer programming tools do not necessary generate traditional source code as reference within this clause.

Depends on resolution of comments regarding software tools used in voting system development. If the tools are thoroughly inspected at the source code level and otherwise certified for use in voting system development, accept the change. Otherwise, leave the language as is.

Suggested change: Agree with DJL "Unmodified thirdparty software is not subject to code examination. Software that has been modified or generated since verification (e.g., source code generated by COTS code development package) shall be provided in human readable form." This same sentence is repeated in section 6.6.2. Please make it consistent if there are any changes.

Lipsio-89

5.6.2

E

“The software used by voting systems is selected by the vendor” appears to mean “COTS is selected”; else, it contradicts the subsequent sentence. Change the opening words from “The software” to “The COTS software”.

The word Agree “selected” is partly incorrect. Add “or developed” after “selected”.

Accept proposed change

Lipsio-3E

5.6.2.2

para. 1

T

Industry standard Require all COTS compiler and tools, including runtime compilers and interpreter both interpreters, to is not defined be validated and and assumes that, verified in the contrary to same manner as reality, application something is fail- software. safe and foolproof by virtue of being in common use.

Accept change as part of change accepted elsewhere.

Sklein-057

5.6.2.3

5.6.1.1

T

COTS evaluated should include compilers, libraries, and any other software tools used in system development and capable of introducing backdoors or other malicious code.

COTS to be evaluated shall include compilers, libraries, and any other software tools used in system development and capable of introducing backdoors or other malicious code.

Accept change as part of change accepted elsewhere.

I agree with the principle that something is not fool-proof just because it is in common use. However, it is impractical to require that compilers and interpreters are validated and verified. Information security has a long history of implicitly trusting these because of the impractibility of assessing them. Reasonable assurance can be gained from indirect review of these intrepreters and compilers (i.e., testing the resultant executable rigorously enough Disagree. These developmental and support tools are not relied upon to enforce the security policies of the system instead they merely need to perform as expected. These tools could be tested to ensure their performance is as expected, but this is typically a high assurance activity and seems beyond what is needed here.

Accept proposed change

Do not accept change. Not practical or necessary.

Lipsio-44

5.6.2.3

Para. 1

T

There is implied a lack of testing in “COTS products require updates due to a detected security breach or vulnerability”; nothing that requires an update should pass testing.

Mandate that testing preclude any security breach or vulnerability; mandate compliance with section 4.3.11 (“Previously developed or purchased software”) of IEEE Std 12281994, “IEEE Standard for Software Safety Plans”. Mandate COTS be subject to the specifications of IEEE Std 1008™1987 (R1993), “IEEE Standard for Software Unit Testing”. Add reference to IEEE Std 982.1™-1988, “IEEE Standard Dictionary of Measures to Produce Reliable

Would need to see cited IEEE standard before addressing proposed change. However, relevant IEEE standards should generally be applied where applicable in this standard because they reflect relevant IEEE expertise.

I don't understand No comment the statement "Nothing that requires updating should pass testing." Any part of the system may require an update at any time. Instead the concept of emerging threat analysis must be adopted. When a system is deployed it must be shown to be resistant to all "known" vulnerabilities. The vulnerabilities that are known change all the time. I suggest incorporating the following language to address this concern. "System deployment must include a review of the system

Lipsio-45

5.6.2.3

Para. 1

T

There is implied a lack of testing in “the most recent version of the COTS product incorporating all security patches” ”; nothing that requires an update should pass testing.

Bring into Would need to see See above conformance with cited IEEE Annex D (“V&V of standard before reusable addressing software“) of proposed change. IEEE Std 1012However, relevant 1998, “IEEE IEEE standards Standard for should generally Software be applied where Verification and applicable in this Validation”, standard because e.g., “Reusable they reflect software (in part relevant IEEE or whole) expertise. includes software from software libraries, custom software developed for other applications, legacy software, or commercial-offthe-shelf (COTS) software. The V&V tasks of Table 1 are applied to reusable software just as they are applied to newly developed

No comment

David Dill

Bob Oliver

Vince Lipsio

R. Mercuri

Accept.

NC - This proposed change is not practical and would put many vendors in a poor competitive position. Operating systems, drivers, etc. may not be available for line by line analysis.

Agree, but need to insert “, among other things,” between “presence” and “of”.

Any use of COTS that could impact security or functionality of the voting product must be thoroughly examined. Alternative certification (such as Common Criteria EAL4) that is independently recognized could be acceptable.

Accept.

Accept proposed change.

Language needs to be stronger; otherwise, accept. Any demonstrated lack of security or integrity in and component, COTS or otherwise, of the voting device shall result in decertification, the remedying whereof shall require that the V&V procedures shall be amended to flag the found defect, and full regression testing shall be performed before the corrected device be again certified.

Refer to Raba report as to why this is necessary (they describe exploits that could be used by Blaster if the OS is not updated, for example).

Accept.

NC - "may" allows (My comment) the case where existing data can help the evaluation process. Bring in the essence of the referenced specs if they apply. Many people, myself included, are not intimate with the references.

Accept proposed change

Accept.

Change to "COTS products may require.." Bring in the essence of the referenced specs if they apply. Many people, myself included, are not intimate with the references.

(My comment)

Accept proposed change

Accept.

NC - Bring in the essence of the referenced specs if they apply Many people, myself included, are not intimate with the references.

(My comment)

Would need to see cited IEEE standard.

Accept.

The first sentence is reasonable and can be accepted. A determination of what is "crucial to the corrct and secure operation" is key here. For example, voter verifiable paper ballots may obviate some or all software security concerns. The last sentence of the proposed change is not practical and would put many vendors in a poor competitive position. Operating systems, drivers, etc. may not be available for line by line analysis.

Assuming that the scope is explicitly limited to what is internal t the voting device, I agree; for external devices connected to the voting device, proof that these devices can not affect the functioning of the voting device should be sufficient.

Interaction with COTS components may not be readily understood until such assessment is performed. The change should reflect that all COTS products must be assessed in order to determine their involvement with the security functions, and those that are identified as having critical impact should be examined in detail. (Note: I would actually prefer an alternative, independent, official, security certification, such as an appropriate level common criteria evaluation for the COTS products,

I agree in principle with this, but this is another example of a high-level policy question that should be discussed before line-by-line editing. We need to grapple with the question of “What is an adequate level of security?”, and “How do we identify the components of a system that are critical to achieve that level of security?” I strongly question whether an adequate level of security can be achieved by anything short of a voter verified paper trail.

Out of scope to mandate VVPT here. However, please note that the stringency of V&V would not, in my opinion, be nearly so great in a system with an independent audit trail. As things stand where no such audit trail exists, I am inclined to test the systems to the standards of, at least, RTCA/DO178B, ("Software Considerations in Airborne Systems and Equipment Certification"), while with the presence of an independent audit trail, robustness is much less critical, and the main concern Open source should NC - This proposal Don’t know: Agree only be required is clearly antiin principle that for security competitive. open source critical should be components, COTS required but if or custom. we require all tools to be certified, to have undergone formal V&V, I don’t see know that we can mandate this. Disagree with the implication that open source will provide adequately testing; rather, it will improve the adequacy of testing.

Accept proposed change with the following modification: Replace "A voter verified paper audit trail..." with "A secure voter verifiable approach..."

The system audit section (5.1.3-4) here should be reworded to reflect the fact that auditability must pertain to the ballots as well -- anonymity of the ballots should not exempt auditing at the most critical point, that of the collection of the ballots. The COTS sections in this part of the standard 5.1.3.6.4 and 5.1.3.6.5 need to be augmented so that auditability is possible throughout. If this is explained well, it should not be necessary to specify a particular implementation. Open source does not provide any significant advantage in terms of security to closed source, in fact, it might even provide a false sense of security. I disagree that an open source requirement resolves this issue. But it does need rewriting as per sKlein-056.

Accept.

Accept the proposed change.

(My comment)

Networking components may contain embedded software modules that also need to be examined, as these can be critical to proper operation. Reject proposed change. Need to ensure that all of this is covered elsewhere in the latest draft of the standard.

Accept.

Accept the proposed change.

Proposed Change: COTS products, especially software libraries, are a vulnerable attack point and must be subject to risks assessment prior to use in voting products. My comment: Agree Proposed Change (continued): Configuration management should include vendor updates and alerts when flaws are detected that could compromise election operations or cast ballot data integrity. My comment: Agree with the following changes: 1) Change “should” to

Suggest we replace NC - Modified code with a requirement must be capable of that all files and evaluation. tools used to generate the COTS object code or executable by supplied. The build process should be executed at the ITA. The tools must either be approved by the ITA, or the generated files must be checked for consistency with the files from which they were derived at each step.

Accept proposed change.

No. Rather, require to be provided in human readable form the code written by a human, which would be the input to a COTS code development package that produces “source code”, and not necessarily the code produced by the package. If the input to a COTS code development package is not “traditional source code”, it still must be provided in a human readable form, even is that means a screen shot or whatever. (My comment)

Some code may be generated through other methods -table-driven packages, for example. In such cases, what must be provided are the original files and the programs used to generate code from them. This needs to be explained here, not exempted.

It is unclear what is meant -whether it is the COTS software or custom. Need clarification (perhaps by looking at earlier versions of this section, or the FEC document?).

Accept.

NC - This proposed (My comment) change is not practical and would put many vendors in a poor competitive position. Tools may not be available for line by line analysis.

All tools must be subject to validation, but even this does not provide full assurances (see Ken Thompson "Reflections on Trusting Trust" paper). Here again, it might be best to seek alternative security (like common criteria) certification, since that might be more stringent than what the voting system evaluators are capable of providing. The reason for this should be explained here as well.

Accept.

Accept proposed change.

Agree.

Change is necessary and should be accepted.

Accept.

NC - Bring in the essence of the referenced specs if they apply Many people, myself included, are not intimate with the references.

(My comment)

Need to see standard.

Accept.

NC - Bring in the essence of the referenced specs if they apply Many people, myself included, are not intimate with the references.

(My comment)

Need to see standard.


								
To top