Managing the development of secure electronic banking

Managing the development of secure electronic banking MSc Thesis GUC Tom Nilsen Problem • We have a security analysis for projects • The analysis should ensure compliance to our security requirements • We think that the analysis contributes to secure systems • We do not know the actual contribution • And security level varies across projects • We want to measure – The the security status of system at delivery – The contribution of the Sec. analysis to security Security requirements The Bank's Security Framework has approx 200 Security requirements Security Analysis Reccomended Security solution or approach Will the project use the solution? If not, please describe….. The relations that leads to secure systems Sec. requirements Security analysis Security metrics Security Status at delivery 200 requirements 60 questions &answ Research Question Research question • How to define a security metric that measures the security status of a new system at delivery • Security status: – Compliance to recommendations in the analysis – Compliance to the security framework and requirements The MSc work • • • • • • Review of previous work Choice of methods Defining a framework for the metric Defining the content of the metric Testing and improving the metric Analysing the project work and findings Review of previous work • Theory on security metrics – Security Metrics guide, 800-55 NIST – A method for security metric design, Frost/Snekkenes – Careful: "simplifying a complex security matter down to a number", McHugh/McCallam • Measuring security in practice – Risk analyses and Checklists – Security SLAs (I.e.the Banks own Security metric) – ISF metrics Choice of methods • The Authors role: The risk of being to involved and proprietary: – Reference group and external reviews • Defining a framework for the metric – Studying Previous work and apply principles – Qualitative Content analysis • Security Analysis, Security Requirements • ISF metrics and ideas • To define the topics to be measured – Analysing cross referencing and structuring findings from the Content analyses – Semi structured interview of relevant managers A method for security metric design (Frost/Snekkenes) RISK MGMNT, SECURITY REQ, COMPLIENCE CIA TO SELECTED Security Metric RISK OF NON-COMPLIANCE "A clear, known, agreed and understood definition…" Stakeholders Who needs the metric? STAKEHOLDER Project manager / Business developer System owner / Business owner REASON To document security status and learn from their own performance. To know the security status when they take over the responsibility from development. To manage risk and to quality assure facts before they report externally to reduce their personal liability. To collect statistics to see trends of systematic problems and flaws in development processes that lead to security problems. To assist top management in managing risk and to manage and improve the security process and deliverables (i.e. Security analyses). Top management Management of System development Head of IT Security "Clear line of sight" Accountability • A clear connection between the stakeholders actions and the measured result – Scope of project – Relevant security requirements • Selected requirements for Systems Security program maturity: Ambitions versus reality NIST 800-55 ASSESSMENT OF THE BANK - Security program since 1994 -Policy - Baseline Sec requirement 1995 - Security Analysis 1997 - > 400 conducted - Outsourcing and Security 2001 -Security program - Security testing, code review, IDS CONCLUSION: - between level 3 and 4 - data available at medium difficulties Maturity is is influenced by Mergers, Management +++ => focus on AUTOMATION of Data collection and testing NIST 800-55 Metric detail form IMPLEMENTATION EVIDENCE Does the Audit trail provide a trace of user action ? YES ? NO ? How can we help in assessing? ISF Metrics ASPECTS AREAS 7 SECTIONS CONTRO LS Ca 252 CtrlDETAILS Ca 745 SM SEC MGMNT 32 CB CRIT BUS CI COMP INST NW NETWORK 6 6 5 6 30 25 31 24 23 135 121 Ca 250 Ca 193 Ca 143 Ca 1000 348 Ca 600 Ca 455 Ca 399 Ca 2500 SD SYST DEV TOT Factors that increase # of incidents ISF FIRM Special circumstances for a system or information recourse:  Subject to a high degree of change  Widely extended geographically  Large in scale  Complex  Immature  Accessible to external parties  Used to support call centres The content of Security Analysis COVERAGE -Authentication & Identity management -Access control & role based access mgmnt -Password process # 30 -Access control for programs - Securing appl.data - Network & data exchange - Secure code & pentest - Audit, logs & incident mgt Sec. requirements Security analysis Security metrics Security Status at delivery - Resilience & contingency - Tech. Platform & infra - System maint.& Operation - Agreements (SLA, 3.party) - RISK SUMMARY # 80 6 200 requirements 0 questions &answ Research Question Why 30 Q on Access control? • The Banks approach: – Ease of use • Reduced sign-on – Ease of administration • Single point of admin – Advantage of scale • Standardisation • Re-use secure solutions and processes Sec. requirements Security analysis Security metrics Security Status at delivery 200 requirements 60 questions &answ Research Question EXTERNAL VALIDITY: INTERNATIONAL SECURITY STANDARD ISF SOGP AND METRICS HEALTH CHK Security metric work sheet Security Metric Prototype metric COVERAGE -Authentication & Identity management -Access control & role based access mgmnt -Password process # 50 -Access control for programs - Securing appl.data - Network & data exchange - Secure code & pentest - Audit, logs & incident mgt - Resilience & contingency # 90 - Tech. Platform & infra - System maint.& Operation - Agreements (SLA, 3.party) # 116 - RISK SUMMARY Security Metric Challenges • Are questions asked – On the most important topics – With the right approach to the mix of security and re-use of secure solutions – Right level of details (Health check 170 <> Survey 2500) • And in a way that we can – Collect answers and verify by testing? – Build on and compare to answers in Sec.Analysis – Use the measurement as a part of security documentation? Conclusions from the Bank • Final metric versus MSc version – "minimal"MSc version in English – Proposal and description for building: • new Security Analysis • and Metric on WEB • The bank needs – A metric that focus the weak security areas – With relevant info for risk management • The bank do not need a metric => 3.7 secure Security Metric Focus in the MSc report • For the sake of the reader: • We have chosen a few metric topics – Discuss them in more detail • Instead of "scratching the surface" of 115 metrics Security Metric Testing and improving the metric • The first test candidate: HR and Salary system run by an external SP • Responsible roles had a need for a status • Logistics – – – – Gather data from Sec.Analysis Arrange meeting with responsible roles Establish/Confirm facts Analyse non compliance areas Challenges measuring Implementation evidence NIST 800-55 Security Metric An evaluation of the metric • Value to the bank  • Validity: Internal , External  • Reliability: – improve by Implementation evidence – Trained ITS consultant to assist user • The workload: => automation  – Timing / project phase is important Evaluation of the MSc work • The authors role – Objectivity, the risk of being too involved: • Reference group but difficult to control  – Proprietary setting - "home grown" or knowledge of general interest? • Cross ref. to an open standard – SOGP • Applied principles from science and ISF practice => "How to generalize the metric" "How to generalize the metric" The business must have defined which Security requirements/standards • Select the requirements to comply with • Define stakeholders and their needs • Evaluate the security program maturity • Define a framework for the metric • Define the actual content of the metric with cross ref. to the requirements for validity • Use implementation evidence in forming questions • Test and improve the metric • Decide a process and IT-support for the metric • Remember that introducing a security metric is developing your organisation Further work • Finish a Norwegian version of the metric – Update the Sec.Analysis and metric to new version of ITS framework – Decide minimum version with final questions – Decide a process for the metric • Stakeholders support – IT-support/automation for the metric Conclusion • We have developed a prototype metric according to the assignment • We have tested and improved it • We have described the steps for a final Norwegian version of the metric • The process and the framework for designing the metric is based on scientific principles and "industry practice" • We hope to inspire others to build security metrics New generation ISF metrics • Web based in stead of Excel • XML cross reference – SOGP, 17799, Cobit • META Database – All controls from important standards New generation SA and S metrics • Web based – On ISF metric – With Bank extensions • XML cross reference – SOGP, 17799, CobiT – SA and metrics cr.ref • META Database – Also BSR controls