Diffusion models are generating considerable interest in the AI community, particularly due to their unique approach and potential advantages over traditional autoregressive models. Key points include: 1. **Development and Scaling**: While current diffusion models may not yet match the performance of established models like GPT-2, there is optimism for future advancements as they are refined and scaled. 2. **Reduced Hallucination?**: Claims that diffusion models would hallucinate less are debated. Some users note that even in diffusion-based image generation, hallucinations can occur, suggesting caution against overestimating their capabilities. 3. **Editing Capabilities**: Diffusion models may offer the ability to edit early tokens, which could mitigate biases inherent in autoregressive methods. This suggests a potential for improved reasoning and text generation. 4. **Trade-offs and Compute**: There is an interesting dynamic of compute versus accuracy with diffusion models, as they may allow a more flexible approach compared to the fixed computational budgets of traditional models. Key questions arise regarding the relationship between model size and diffusion steps, as well as optimal configurations for context and diffusion windows. 5. **Future Possibilities**: At extreme scales, diffusion models may revolutionize creative processes, such as generating entire novels or sizeable codebases in seconds, thereby significantly impacting the productivity of AI-driven tasks.