• Genome Data Science

    We develop methods and tools to work with tens of thousands of genomes and analyze and integrate the corresponding data.

    Model of DNA double helix in front of a student.
    © Universität Bielefeld
You are here: home » teaching » 2023summer » attention

Attention networks and diffusion models in big data analytics


392185 Schönhuth Summer 2023 Tue 10-12 in U10-146 (S)

Contents

Attention networks and diffusion models reflect two classes of deep learning techniques that have gained utmost recent attention. In this seminar, we will review recent, related contributions. Thereby, we will focus in particular on the algorithmic and statistical elements of the approaches. We will further discuss the corresponding applications in areas where big data plays a major role, such as text processing, biology or social networks.

Papers

If there is any problem with accesssing a paper, write an email to Johan Vérolet and Daniel Göbel.

Title Authors Year Journal
Beyond Self-Attention: External Attention Using Two Linear Layers for Visual Tasks Meng-Hao Guo et al. 2021 IEEE Transactions on Pattern Analysis and Machine Intelligence
Language Models are Few-Shot Learners Tom B. Brown et al. 2020 Advances in Neural Information Processing Systems
Equivariant 3D-Conditional Diffusion Models for Molecular Linker Design Ilia Igashov et al. 2022 arXiv
High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach et al. 2022 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding Chitwan Saharia et al. 2022 Advances in Neural Information Processing Systems
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya Ramesh et al. 2022 arXiv
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models Alex Nichol et al. 2021 CoRR
ResNeSt: Split-Attention Networks Hang Zhang et al. 2022 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops
Dual Attention Network for Scene Segmentation Jun Fu et al. 2019 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Denoising Diffusion Probabilistic Models Jonathan Ho et al. 2020 CoRR

Time table lecture

Date Topic
11.04.2023 Introduction (slides)
18.04.2023 Attention Networks II (slides)
25.04.2023 Transformers / How to present (transformer-slides / how-to-present-slides)
02.05.2023 no seminar
09.05.2023 Transformers II / How to write Reports (transformer-slides / how-to-write-slides)
16.05.2023
23.05.2023
30.05.2023
06.06.2023
13.06.2023
20.06.2023
27.06.2023
04.07.2023