Software security 1000-2M15BO

1. Introduction to hacking. Historical perspective and overview of selected exploits.
2. Stack layout in x86/x64. Calling conventions. Tracing programs with a debugger (assembly level).
3. Low level exploits that facilitate taking control over program execution. Buffer overflow attacks - basic idea. Static buffer overflow, stack smashing, heap overflow. Integer overflow attack. Use-after-free attack, double-free attack, etc. Heap spraying technique.
4. Buffer overflow protection. The art of writing secure programs. Hardware and software (compiler and OS level) countermeasures: canaries, bounds checking, Data Execution Prevention (DEP), Write XOR Execute (W^X), Safe Structured Exception Handling (SafeSEH), Vectored Exception Handling (VEH). “return-into-libc”/return-oriented programming (ROP) techniques to counter executable space protection mechanisms. Tools offering active protection, e.g., Enhanced Mitigation Experience Toolkit, StackGuard, ProPolice.
5. Reverse engineering of machines and software. Legal considerations. Overview of methods and tools.
6. Digital Rights Management (DRM). Protecting programs with code obfuscation. Anti-debugging protection. Code virtualization. Hardware solutions. DRM removal methods.
7. Web-based attacks. SQL Injection. Session hijacking and session riding aka Cross Site Request Forgery. Best protection practices. Rogue certificates and certificate pinning. DigiNotar case study.
8. Intrusion Detection System (IDS) tools like: TripWire, Samhain, etc. Penetration Testing.
9. Wi-Fi access protection. Wired Equivalent Privacy (WEP) and Wi-Fi Protected Access (WPA/WPA 2). Flaws in Wi-Fi Protected Setup (WPS) suite.
10. Cryptographic means. SSL/TLS protocol and its implementations. Basic attack schemes on network protocols: man-in-the-middle, reply attack, side-channel attack. Lucky Thirteen attack on TLS. Randomness in cryptography and its misuse. Attacks on OpenSSL and SecureRandom implementation from Java Class Library. Heartbleed - a buffer over-read attack on OpenSSL.
11. Backdoors. Kleptography.
12. Sandboxing in Linux. User-mode (chroot, cgroups, ptrace) and kernel-mode (seccomp). Privillege escalation. Jailbreaking.
13. Virtualization. Hardware support. Example of bypassing Supervisor Mode Execution Prevention (SMEP). Blue Pill rootkit.

Type of course

elective monographs

Requirements

Security of computer systems
Operating systems

Prerequisites (description)

One of the pivotal topics that will be discussed during this course covers subverting software with low level attacks. Being acquainted with an appropriate low level programming language, preferably C or x86 assembly language, is therefore highly desirable. All the essential information in this area however will be given/reviewed during the course. It is advised, yet not mandatory, to possess some experience with low level program debugging and debugging tools like: OllyDbg, gdb, WinDbg, or Visual Studio Debugger. Students are supposed to be familiar with rudiments of: SQL, web technologies, and Linux/Windows operating systems.

Learning outcomes

Knowledge:
1. Students are aware of the most wide spread kinds of software exploits and can distinguish between them.
2. Students understand executable space protections mechanisms and are conscious of consequences of turning those mechanisms off.
3. Students do know the general guidelines for writing secure software.
4. Students can name standard Wi-Fi access protection standards and point out those possessing security flaws.
5. Students are familiar with basic notions of code obfuscation.
6. Students are aware of fundamental cryptographic means for protecting information and communication.

Abilities:
1. Students understand the backgrounds of buffer overflow attacks and are able to write down a toy example illustrating the concept.
2. Students are able to identify a possible flaw in unsophisticated and unelaborate code provided the know the code is vulnerable to attacks of some given type in advance.
3. Students are capable of designing systems and implementing programs with increased resillience to the most popular types of exploits.
4. Students are able to determine whether an application issuing SQL queries is susceptible to SQL injection attacks.

Skills:
1. Students are aware of limitations of their own knowledge and is looking forward to broaden their horizons in the area of software security.
2. Students are capable of finding information concerning software security in the Internet.
3. Students are able to comprehend intermediate level articles covering software security topics and exploits.

Assessment criteria

Students are supposed to complete one or more individual projects (depending on project complexity) which will be subsequently graded. Projects will focus on testing in practice what students have learnt by performing source code analysis, nailing down possible vulnerabilities, as well as removing those and making programs more secure.

Bibliography

● J. Foster, V. Osipov, N. Bhalla, N. Heinen, Buffer overflow attacks: Detect, exploit, prevent. Syngress, 2005.
● J. Foster, V. Liu, Writing Security Tools and Exploits. Syngress, 2006.
● S. Mcclure, J. Scambray, G. Kurtz, Hacking Exposed 7: Network Security Secrets & Solutions. McGraw-Hill Osborne Media, 2012.
● D. Regalado, S. Harris, A. Harper, C. Eagle, J. Ness, B. Spasojevic, R. Linn, S. Sims, Gray Hat Hacking The Ethical Hacker's Handbook, Fourth Edition. McGraw-Hill Osborne Media, 2015.
● on-line materials, e.g., http://overthewire.org/wargames/, https://www.owasp.org/, or a Coursera course https://www.coursera.org/course/softwaresec

Additional information

Information on level of this course, year of study and semester when the course unit is delivered, types and amount of class hours - can be found in course structure diagrams of apropriate study programmes. This course is related to the following study programmes:

Additional information (registration calendar, class conductors, localization and schedules of classes), might be available in the USOSweb system:

Description of 1000-2M15BO in USOSweb