الخطوط العريضة للقسم

  • This seminar covers recent topics in Natural Language Processing. This term´s seminar is for BSc and MSc students and focuses on Mixture-of-Experts Architectures in Large Language Models.

    Goal:     Present and write a mini-survey paper about Mixture-of-Experts Architectures in Large Language Models focused on your subtopic.

    Description

    In this seminar, you will:

    • Select one of the offered subtopics (see Subtopics).
    • Read, understand and explore scientific literature (with listed papers as starting points for your analysis)
    • Organize the collected knowledge for a meaningful presentation about your topic (10-15 minutes + 5 minutes Q&A)
    • Summarize your topic in a concise report (6 - 8 pages + references)

    Feedback Sessions:

    • We offer 2 feedback sessions + on demand feedback via E-Mail
    • 1. Session - beginning of June: Present your structured current literature review + show high-level understanding of background concepts and papers
    • 2. Session - beginning of July: Shaping your presentation and the outline of your report
    • Come prepared: A few slides summarizing your current progress; a clear idea of what you want to discuss

    Subtopics 

    Each student explores and presents one of the below subtopics.
    For each topic, prominent papers are provided as a starting point for the seminar exploration.
    Your presentation and report are based on the provided papers and other work, which you are expected to discover on your own, starting from the given papers.
    Your goal is to clearly summarize the body of work on the selected subtopic, it's relevance and impact, and draw your own conclusions about the limitations and/or future efforts in this area.

    1. Architecture (Expert Design, Routing, Load Balancing)
    2. Upcyling
    3. Post-Training (e.g., Instruction Tuning, LoRA Fine-Tuning)
    4. Domain and Language Modularization

    General survey for an overview: https://ieeexplore.ieee.org/document/10937907/

    Deliverables:

    Presentation:
    • 10-15 Minutes
    • What, why, and how - (i) introduce &  motivate,  (ii) comprehensively cover (incl. related work),  (iii) and look at what lies ahead for your topic 
    • 5 Minutes Q&A
    • Target audience: Your fellow students
    Report

    Grading

    • Report and presentation are similarly important
    • Do not plagiarize

    Timeline

    • Attend the kick-off meeting: time and date to be announced soon
    • Send you topic preference by [to be announced] to benedikt.ebing@uni-wuerzburg.de; 
    • We will inform you by [to be announced], whether you got a spot in the seminar
    • Kick-Off Session [to be annunced]
    • 1. Feedback Session: beginning of June
    • 2. Feedback Session: beginning of July
    • Presentations: [to be announced]
    • Report: [to be announced]