ENGR 482 Lecture 16

« previous | Monday, October 21, 2013 | next »

Ethics in Software Engineering

Guest Lecture by Dr. Ioerger

Most modern systems have software components

• Engineres should consider usability, reliability
• Generalization: engineering design (trans-disciplinary)

Cases

Where flaws in software design have had major negative impacts

Therac-25

Chemotherapy Machine produced by French company in 1985

Software bug caused 100× overdoses to 6 patients, resulting in radiation burns

Bug caused beam shield to be completely raised at high dosage settings

GM Emissions Controller (#4)

Chip didn't control emissions when climate control was being used

Engineers didn't think it was explicitly required in the regulations;

0.5 M vehicles recalled and millions of dollars in fines.

Not a bug, just costly design choice.

AT&T Long-Distance Network

Network crashed in 1990, causing 60,000 people to lose service for 9 hours

Due to missing break statement in a switch case:

• caused one node to send fail/congestion message to neighboring nodes
• cascaded to shut down entire network

Nasdaq Shutdown

A few weeks ago, Securities trading was shut down on Nasdaq for 3 hours.

Caused by flood of data from NYSE Arca exceeded capacity of marketwide feed known as Security Industry Processor, which exposed software bug.

Any piece of software, even if it's run for 100% for 10 yrs, there's still flaws in it.NASDAQ CEO

Responsibility for Documentation and Testing

• basic engineering ethics
• code should work, be reliable, and be robust
• Others should be able to understand your code so they can maintain it.
• shared responsibility of programmer, team, manager, company, etc.
• Respect software copyrights and licenses.

ACM

ACM Code of Ethics

Similar to other codes, but a little more down-to-earth

ACM code emphasizes safety of public over interests of employer.

Take responsibility for own work; keep informed; and honor laws, copyright, confidentiality, privacy, etc.

Contribute to Society and Human Well-Being

Minimize negative consequences of computing systems, including threats to health and safety.

General Moral Imperatives are very aspirational.

Interface Design

Design of software must match Cognitive Structures

How to clearly convey state/information:

• cancel buttons in a dialog box are notoriously ambiguous; does this action erase data?
• ATC GUI had "OK" light that used to flash twice per second; brain interpreted this as "alert", so changed to once per second.
• Airbus 300 crashed because A/P didn't clearly convey "go-around" to pilots

Social Impacts

More automation = more social reliability = less understanding

Hacking Cases:

• Reformed Hacker (#30)
• Morris Worm (1988)
• Kevin Mitnick (1995)

Final Thoughts

• Google's corporate motto: Don't be evil... Why? Because they have the power to be!
• Be ethical in engineering design; think about user and public consequences