====== Attention networks and diffusion models in big data analytics ======
\\
| [[https://ekvv.uni-bielefeld.de/kvv_publ/publ/vd?id=394224174  | 392185]] | Schönhuth | Summer 2023 | Tue 10-12 in U10-146 (S) |

==== Contents ====

Attention networks and diffusion models reflect two classes of deep learning techniques that have gained utmost recent attention. In this seminar, we will review recent, related contributions. Thereby, we will focus in particular on the algorithmic and statistical elements of the approaches. We will further discuss the corresponding applications in areas where big data plays a major role, such as text processing, biology or social networks.

==== Papers ====

If there is any problem with accesssing a paper, write an email to [[mailto:jverolet@aol.com,dgoebel@techfak.uni-bielefeld.de|Johan Vérolet and Daniel Göbel]].
\\
| **Title** | **Authors** | **Year** | **Journal** |
|[[https://arxiv.org/abs/2105.02358|Beyond Self-Attention: External Attention Using Two Linear Layers for Visual Tasks]] | Meng-Hao Guo et al. | 2021 | IEEE Transactions on Pattern Analysis and Machine Intelligence |
|[[https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html|Language Models are Few-Shot Learners]]| Tom B. Brown et al. | 2020 | Advances in Neural Information Processing Systems| 
|[[https://arxiv.org/abs/2210.05274|Equivariant 3D-Conditional Diffusion Models for Molecular Linker Design]] | Ilia Igashov et al. | 2022 | arXiv |
|[[https://openaccess.thecvf.com/content/CVPR2022/html/Rombach_High-Resolution_Image_Synthesis_With_Latent_Diffusion_Models_CVPR_2022_paper.html|High-Resolution Image Synthesis with Latent Diffusion Models]] | Robin Rombach et al. | 2022 | Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) |
|[[https://proceedings.neurips.cc/paper_files/paper/2022/hash/ec795aeadae0b7d230fa35cbaf04c041-Abstract-Conference.html|Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding]] | Chitwan Saharia et al. | 2022 | Advances in Neural Information Processing Systems |
| [[https://arxiv.org/abs/2204.06125|Hierarchical Text-Conditional Image Generation with CLIP Latents]] | Aditya Ramesh et al. | 2022 | arXiv |
| [[https://arxiv.org/abs/2112.10741|GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models]] | Alex Nichol et al. | 2021 | CoRR |
| [[https://openaccess.thecvf.com/content/CVPR2022W/ECV/html/Zhang_ResNeSt_Split-Attention_Networks_CVPRW_2022_paper.html|ResNeSt: Split-Attention Networks]] | Hang Zhang et al. | 2022 | Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops |
| [[https://openaccess.thecvf.com/content_CVPR_2019/html/Fu_Dual_Attention_Network_for_Scene_Segmentation_CVPR_2019_paper.html|Dual Attention Network for Scene Segmentation]] | Jun Fu et al. | 2019 | Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) |
| [[https://proceedings.neurips.cc/paper/2020/hash/4c5bcfec8584af0d967f1ab10179ca4b-Abstract.html|Denoising Diffusion Probabilistic Models]] | Jonathan Ho et al. | 2020 | CoRR | 

==== Time table lecture====

| **Date** | **Topic** |
|11.04.2023 | [[teaching:2023summer:attention:seminar01|Introduction]] ({{teaching:2023summer:attention:introduction-110423.pdf|slides}}) |
|18.04.2023 | [[teaching:2023summer:attention:seminar02|Attention Networks II]] ({{teaching:2023summer:attention:lecture2-180423.pdf|slides}}) |
|25.04.2023 | [[teaching:2023summer:attention:seminar03| Transformers / How to present]] ({{teaching:2023summer:attention:lecture3-250423.pdf|transformer-slides}} / {{teaching:2023summer:attention:howtopresent.pdf|how-to-present-slides}})|
|02.05.2023 | // no seminar //|
|09.05.2023 | [[teaching:2023summer:attention:seminar04|Transformers II / How to write Reports ]] ({{teaching:2023summer:attention:lecture4-090523.pdf|transformer-slides}} / {{teaching:2023summer:attention:howtowritereports.pdf|how-to-write-slides}}) |
|16.05.2023 | |
|23.05.2023 | |
|30.05.2023 | |
|06.06.2023 | |
|13.06.2023 | |
|20.06.2023 | |
|27.06.2023 | |
|04.07.2023 | |