Kursthemen

  • General

    Alles einklappen Alles aufklappen

  • Content/Schedule

    Block 1: Fundamentals

    17.4.

    L1: Languages of the world & Linguistic Universals; Course organization
    24.4. L2: Language modeling, word embedding models, tokenization & vocab building
    26.4 Ex1: Intro to Language Modeling & Tokenization
    15.5. L3: Deep Learning for (Modern) NLP — Perceptron/MLP, Backprop, Batching, Gradient Descent, Dropout…
    17.5 Ex2: Backprop & Training Models in Pytorch
    22.5. L4: Transformer Almighty & Pretraining Language Models (autoregressive, masked language modeling)
    24.5 (Online) Ex3: Transformer

    Block 2: Multilinguality

    29.5. (online)

    L5: Multilingual Word Embedding Spaces (and CL Transfer with them)

    31.5 (online)

    Ex4: Project Topics & Setup
    12.6. L6: Multilingual LMs and Cross-Lingual Transfer; Tasks, Benchmarks & Evaluation
    19.6. L7: Curse of Multilinguality, Modularization, and Language Adaptation
    26.6. L8: Transfer for Token-Level Tasks: Word Alignment & Label Projection (+ maybe translate train on sequence labeling task)
    28.6 Ex5: Modularization

    Block 3: Advanced Topics

    3.7.

    L9: Neural Machine Translation (incl. decoder only MT)

    5.7

    Ex6: Transfer for Token-Level Tasks
    10.7. L10: Multilingual Sentence Representations
    12.7 Ex7: Neural Machine Translation
    17.7. L11: Large Language Models, Instruction-Tuning and Generative NLP
    19.7 Ex8: Multilingual Sentence Representations

  • Lectures

  • Exercises