CS 6501 - Machine Learning in Systems Security

Spring 2025

This course explores the application of machine learning to system security, covering key topics such as intrusion detection systems, audit logging, adversarial machine learning, and graph representation learning. Students will engage with cutting-edge research on malware detection, ransomware, phishing, and enterprise security while addressing challenges like detection evasion, distributed threat detection, and real-time data analysis. Through class discussions, a hands-on project, and critical paper evaluations, the course emphasizes developing practical skills and exploring innovative solutions to enhance enterprise security.


Professor Wajih Ul Hassan (hassan@virginia.edu)
Office hours: Tue 4 PM
Office: Rice Hall 522
TA Mati Ur Rehman (mati@virginia.edu)
Office hours: Fri 5 PM.
Rice Hall 442
Lectures Tue/Thu: 5:00pm – 6:15pm
Rice Hall 340
Prerequisites No formal prerequisites. Some background in computer security and operating systems (e.g., equivalence of CS4630 and CS4414) will be helpful.
Communications We will make use of the following communication methods:

- Lecture slides and assignment submission: Collab
- Class discussion / Organization / Announcements: Piazza
Schedule/Paper Signup Google Sheet link

Grading

We will calculate your course grade based on these components:

Component Weight Description
In-class participation or paper discussion on Piazza 20% In each class, we will read and discuss two papers. You are required to critically analyze and discuss the strengths, weaknesses, scope, and potential future research directions of at least one of these papers.

You must contribute to the discussion in one of the following ways:
1. Before the class presentation: Participate in the Piazza discussions related to the paper.
2. During class: Engage in the in-class discussion following the presentation.

Your active participation in either format is essential and will contribute to your grade. Be prepared to make meaningful intellectual contributions by critically evaluating the paper and sharing your insights during class discussions or on Piazza.
Paper Presentations 20% Students will present research papers and lead the ensuing class discussion. The number of presentations required will be determined by the number of students enrolled in the course.
Course Project 60% Students will conduct a major research project in the area of system security, culminating in a conference-style paper at the end of the semester. Project topics will be discussed in class. Projects may be done individually or in pairs (larger groups need to do proportionately more). Your final grade will be based on milestone artifacts and the final report.

Course Project Timeline

Your course project should address an important, interesting open problem related to system security. I’m happy to discuss your project ideas individually and help you refine them.

Pre-Proposal Presentation — In class, Feb 13

Give a 5-minute presentation explaining the problem you want to work on, the most important related work, and your tentative approach. This will be an early opportunity to get feedback from the class.
Please use this template for the presentation:
Template link

Project Status Presentation — In class, March 18

Each group will give an in-class presentation about the status of their project. You’ll have 5 minutes to speak. This presentation should include:

Final Project Presentation — In class, April 24 and 29

Each group will give an in-class presentation about the status of their project, in the style of a brief conference talk.
You’ll have 12 minutes to speak. This presentation should include:

Final Paper — Due on May 4

Your group’s final project report should be written in the style of a workshop or conference submission, similar to the papers we have read this semester. Please include at least:

  1. An abstract that summarizes your work.
  2. An introduction that motivates the problem you are trying to solve.
  3. A related work section that differentiates your contributions.
  4. Section(s) describing your architecture or methodology.
  5. Results and/or evaluation section(s), with data/figures to support your claims.
  6. A brief future work section explaining what is left to do.
  7. Appropriate citations and references from the literature.

See also: advice on writing technical articles.

The length of your report should not exceed 6 typeset pages, excluding bibliography and well-marked appendices. You may include appendices, but graders are not required to read them. Format your text in two columns, using 10 point Times Roman type on 12 point leading, in a text block of 6.5” by 9”. I strongly encourage you to use Latex. You can use this Overleaf latex template.


Academic Integrity

Students are expected to be familiar with the university honor code, including the section on academic fraud.

Assignments or exams where Honor infractions occur will receive a zero grade, and infractions may be submitted to the Honor Committee. If you have questions on what is allowable, please ask.


Special Accommodations

The University of Virginia strives to provide accessibility to all students. If you anticipate or experience any barriers to learning in this course, please feel welcome to discuss your concerns with us.


Religious Accommodations

Students who wish to request academic accommodation for a religious observance should submit their request to the instructors by email as far in advance as possible. If you have questions or concerns about your request, you can contact the University’s Office for Equal Opportunity and Civil Rights (EOCR) at UVAEOCR@virginia.edu or (434) 924-3200.

Accommodations do not relieve you of the responsibility for completion of any part of the coursework you miss as the result of a religious observance.


Discrimination and Violence

The University of Virginia is dedicated to providing a safe and equitable learning environment for all students. Two values that the University and I hold as critically important:

  1. Power-based personal violence will not be tolerated.
  2. Everyone has a responsibility to do their part to maintain a safe community on Grounds.

If you or someone you know has been affected by power-based personal violence, please see the UVA Sexual Violence website (link) for reporting options and resources. I care about you and your well-being and stand ready to provide support and resources. As a faculty member, I am designated a “responsible employee,” meaning I am required by University policy and federal law to report what you share with me to the Title IX Coordinator. Use the Just Report It portal if you wish to report something yourself.


Ethics Statement

This course includes topics related to computer security and privacy. We may cover technologies whose abuse could infringe on the rights of others. As computer scientists, we rely on the ethical use of these technologies. Unethical use includes circumvention of any existing security or privacy mechanisms for any purpose, or the dissemination, promotion, or exploitation of vulnerabilities of these services. Any activity outside the letter or spirit of these guidelines will be reported to the proper authorities and may result in dismissal from the class and possibly more severe academic and legal sanctions.


Honor

I trust every student in this course to fully comply with the University’s Honor Code. By enrolling in this course, you have agreed to abide by and uphold the Honor System of the University of Virginia.

If you believe you may have committed an Honor Offense, you may wish to file a Conscientious Retraction by calling the Honor Offices at (434) 924-7602 before the act in question has come under suspicion by anyone. More information can be found at http://honor.virginia.edu. Your Honor representatives can be found at: http://honor.virginia.edu/representatives.